Introduction

Overview

Talon aims to bring programming, realtime video gaming, command line, and full desktop computer proficiency to people who have limited or no use of their hands, and vastly improve productivity and wow-factor of anyone who can use a computer.

Join the Slack to talk, get hyped, or for help with the public beta.

NOTE: The Talon 0.1.x release is very new and is not fully documented yet! Please ask any questions in the #help channel on the Slack linked above.

Beta Info:

  • System requirements:
    • macOS High Sierra (10.13) or newer. Includes macOS Big Sur (11.0) and native Apple Silicon support.

    • Linux / X11 (Ubuntu 18.04+, and most modern distros), Wayland support is currently limited to XWayland

    • Windows 8 or 10

  • Powerful voice control - Talon comes with a free speech recognition engine (which requires downloading some extra files for now), and it is also compatible with Dragon for both macOS and Windows with no additional setup.

  • Multiple algorithms for eye tracking mouse control (depends on a single Tobii 4C or equivalent eye tracker). Tobii 5 support is preliminary.

  • Noise recognition system (pop and hiss). Many more noises hopefully coming soon.

  • Scriptable with Python 3.8 (via embedded CPython, no need to install or configure Python on your host system).

  • Talon will run fine without an eye tracker, or if you don’t have a speech recognition engine set up.

Getting Started

  1. Download and install Talon for your operating system.

  2. Run the Talon app.

  3. Open the Talon Home directory. This is %APPDATA%\Talon on Windows, and ~/.talon on macOS/Linux. (Talon has a menu in your system tray near the clock, you can use Scripting -> Open ~/.talon as a shortcut open Talon Home).

  1. Add some scripts to ~/.talon/user to add voice commands and other behaviour to Talon (see the Getting Scripts section below). Your user scripts control all of the voice commands in Talon, so Talon won’t recognize any commands until you add some scripts.

  2. If you have Dragon, start it up at the same time as Talon. If you don’t have Dragon, you will need to set up wav2letter, see wav2letter setup.

  3. Go to Scripting -> View Log in the menu for debug output, or Scripting -> Open REPL for a Python command line.

Getting Scripts

The best way to get started right now is to clone knausj_talon into your ~/.talon/user directory. Ask in #help on Slack if you run into any problems.

wav2letter setup

wav2letter requires speech recognition models. Until Talon has an automatic model downloader, you will need to download and put some files in the right spot for it to work, using the below instructions.

  1. Download and extract this zip: https://talonvoice.com/dl/talon-w2l-small3-dslm-en_US.zip

  2. Go to your Talon Home folder (Scripting -> Open ~/talon).

  3. Copy the contents of the talon-w2l zip file into your Talon Home folder. (Be careful not to completely overwrite w2l/ or user/ if you already put other files you care about in there).

  4. The resulting file structure should look roughly like this:

~/.talon/w2l/en_US/*
~/.talon/user/engines.py

.talon Files

Overview

Voice commands are defined in files with the .talon file extension, located in the user directory inside Talon Home.

.talon files can:

  • match specific applications, window titles, or other criteria

  • define voice commands

  • define global hotkeys (Mac-only at the moment)

  • reimplement global actions (for example, to change the behavior of Talon in a specific application)

  • set settings

  • enable tags

Creating a file in user named hello.talon with these contents will declare a voice command:

hello talon: "hello world"

This means when you say hello talon, Talon will type hello world.

This is a more advanced example:

# activate this .talon file if the current app name is "Chrome"
# you can find app names by running ui.apps() in the REPL
app.name: Chrome
-
# key_wait increases the delay when pressing keys (milliseconds)
# this is useful if an app seems to jumble or drop keys
settings():
    key_wait = 4.0

# activate the global tag "browser"
tag(): browser

# define some voice commands
hello chrome: "hello world"
switch tab: key(ctrl-tab)
go to google:
    key(ctrl-t) # note: use cmd-t on Mac
    insert("google.com")
    key(enter)

The - on the line after after app.name is important.

  • Lines above the - are used to set criteria for activating the file.

  • Lines below the - declare things or activate tags.

Any line beginning with # is considered to be a comment and ignored.

API Reference

talon

class talon.Context(name: str = None, *, desc: str = None)

Creating a Context:

from talon import Context
ctx = Context()
property enabled

Manually enable/disable this context:

ctx = Context()
ctx.enabled = False
ctx.enabled = True
Type

bool

action_class(path: str) → Callable[[Class], Class]
@ctx.action_class('prefix')
class Actions:
    def action_name():
        print("Running actions.prefix.action_name()")
action(path: str)
@ctx.action('prefix.action_name')
def action_name():
    print("Running actions.prefix.action_name()")
capture(path: Optional[str] = None, *, rule: Optional[str] = None) → Callable[[DecoratedT], DecoratedT]
@ctx.capture('number', rule='(one | two)')
def number(m) -> int:
    return 1 if m[0] == 'one' else 2
property matches

Describe when to activate this Context. If not specified, Context is always active.

ctx.matches = r"""
os: windows
app.name: Slack
"""
Type

str

property apps

apps:

ctx.apps["chrome"] = r"""
os: windows
app.name: Google Chrome
"""
Type

str

property lists

lists:

ctx.lists["user.listname"] = ["word", "word2"]
ctx.lists["user.listname"] = {
    "pronunciation": "word",
}
Type

Dict[str, Union[List[str], Dict[str, str]]]

property settings

settings:

ctx.settings = {
    "input_wait": 1.0,
}
Type

Dict[str, Any]

property tags

tags:

ctx.tags = ["user.terminal"]
Type

List[str]

property commands

Return the commands defined by this Context

Type

Dict[str, CommandImpl]

property hotkeys

Return the hotkeys defined by this Context

Type

Dict[str, HotkeyImpl]

class talon.Module(name: str = None, *, desc: str = None)

Creating a Module:

from talon import Module
mod = Module()
action_class(cls: Class) → Class
@mod.action_class
class Actions:
    def action_name():
        "Description of the action"

    def second_action(arg1: int, arg2: str='') -> str:
        "Action with arguments, return type, and body"
        return 'test'
action(func: DecoratedT) → talon.scripting.types.ActionDecl[DecoratedT]
@mod.action
def action_name():
    "Description of the action"
capture(*, rule: str) → Callable[[DecoratedT], DecoratedT]
capture(func: DecoratedT) → DecoratedT
@mod.capture
def capture_name() -> str:
    "Description of the capture"
scope(func: ScopeFunc) → ScopeDecl
@mod.scope
def scope():
    return {
        "key": "value",
    }

# call scope.update() at any time to force a scope update
setting(name: str, type: Type[T], default: Union[T, talon.scripting.types.SettingDecl.NoValueType] = <talon.scripting.types.SettingDecl.NoValueType object>, desc: Optional[str] = None) → talon.scripting.types.SettingDecl[T]
mod.setting("setting_name", int, default=0, desc="an example integer setting")
mod.setting("setting_name_2", str, default='', desc="an example string setting")
list(name: str, desc: Optional[str] = None) → talon.scripting.types.NameDecl
mod.list("list_name", desc="list description")
mode(name: str, desc: Optional[str] = None) → talon.scripting.types.NameDecl
mod.mode("mode_name", desc="mode description")
tag(name: str, desc: Optional[str] = None) → talon.scripting.types.NameDecl
mod.tag("tag_name", desc="tag description")
talon.actions

talon.registry

talon.scope

talon.settings

talon.storage

talon.app

talon.app.register(topic: Any, cb: Callable) → None

Register for an application event.

  • launch: Talon launched.

  • startup: Talon launched during system startup.

from talon import app

def app_launch():
    print("Talon launched")
app.register("launch", app_launch)
talon.app.unregister(topic: Any, cb: Callable) → None

Unregister a previously registered event:

app.unregister("launch", app_launch)
talon.app.notify(title: Optional[str] = None, subtitle: Optional[str] = None, body: Optional[str] = None, sound: bool = False) → None

Display a desktop notification, optionally playing a sound.

from talon import app

app.notify(body="Hello world")
app.notify(title="Hello world",
           subtitle="Welcome to Talon",
           body="Enjoy your stay.",
           sound=True)

talon.clip

talon.clip.text(*, mode: Optional[str] = None, tries: int = 3) → Optional[str]

Get the text contents of the clipboard.

talon.clip.set_text(s: str, *, mode: Optional[str] = None) → None

Set the text contents of the clipboard.

talon.clip.image(*, mode: Optional[str] = None) → Optional[talon.skia.image.Image]

Get the image contents of the clipboard.

talon.clip.set_image(image: talon.skia.image.Image, *, mode: Optional[str] = None) → None

Set the image contents of the clipboard.

talon.clip.clear(*, mode: Optional[str] = None) → None

Clear the clipboard.

exception talon.clip.NoChange
talon.clip.revert(old: str = None) → Generator[None, None, None]

Restore the old text of the clipboard after running a block:

from talon import clip

with clip.revert():
    clip.set_text("this will only be set temporarily")
talon.clip.capture(timeout: float = 0.5, inc: int = 0) → Generator[talon.clip.ChangePromise, None, None]

Capture a change in the clipboard, then restore the old text contents:

from talon import actions, clip

with clip.capture() as s:
    actions.edit.copy()
print(s.get())

talon.fs

from talon import fs

def on_change(path, flags):
    if flags.renamed:
        print("renamed", path)
    if flags.exists:
        print("changed", path)
    else:
        print("deleted", path)

fs.watch('/path/to/stuff', on_change)
class talon.fs.FsEventFlags(exists: bool, renamed: bool)
talon.fs.watch(path: str, cb: Callable[[str, talon.fs.FsEventFlags], None]) → None

Watch path for changes and call cb(path: str, flags: FsEventFlags) when changes occur.

talon.fs.unwatch(path: str, cb: Callable[[str, talon.fs.FsEventFlags], None]) → None

Remove cb from the set of callbacks being watched for path.

talon.noise

talon.noise.register(topic: Any, cb: Callable) → None

Register for a noise event.

  • "" - an empty string registers the callback for all noises.

  • "pop"

  • "hiss"

from talon import noise

def on_pop(active):
    print("pop")
noise.register("pop", on_pop)

def on_hiss(active):
    print("hiss", active)
noise.register("hiss", on_hiss)
talon.noise.unregister(topic: Any, cb: Callable) → None

Unregister a previously registered event:

noise.unregister("pop", on_pop)