0.4.0 beta-only features:
- New: Whisper hybrid speech recognition engine.
- New: Talon Menu -> Scripting -> Debug Window.
- New:
deck() support for Elgato Stream Deck in .talon files.
- New:
hotkey_wait setting to pause after complex hotkeys are pressed.
- New: "selection lists" API via
ctx.selections["user.listname"] = "string of text for selection"
- "Mixed Mode" simultaneous command and dictation mode.
- Faster speech recognition.
- Parrot noise recognition.
- Vosk multilingual engine.
- WebSpeech multingual engine.
- Mac: face expression input.
Sign up for the beta tier on patreon.com for earlier access to access to bug fixes, unreleased features, and higher priority support. Your support makes Talon possible.
0.4.0 (Jul 24, 2023)
- Deprecation: use the
@resource.watch(path) decorator instead of resource.open(path).
- Deprecation: CommandImpl now uses
.script instead of .target attribute.
- The following deprecated APIs have been removed:
from talon.track.geom import Point2d - import from talon.types instead.talon_plugins.speechtalon_plugins.menutalon.microphone
- New: the Talon app core is now Rust instead of Qt / C++.
- New: update Python to 3.11.4.
- Add new Conformer D speech model, which is around 20% more accurate.
- Add new Control Mouse settings: Gaze Control, Head Control, Mouse Jump, and Gaze Focus (focus follows gaze).
- Add new Control Mouse settings to use a specific eye (both, left, or right) for tracking.
- Add
gamepad() support for receiving input from gamepads and joysticks in .talon files.
- Add native Log Viewer window.
- Add
.talon-list file format.
- Add logrotate support for talon.log, and a maximum log file size.
- Add window get/set properties:
.minimized .maximized .fullscreen
- Add
browser.query and browser.fragment to scope matching.
- Add automatic dark/light theme switching for Talon's UI.
- Add
@resource.watch(path) decorator.
- Change Control Mouse behavior to resume more smoothly when rapidly toggled.
- Reduce memory allocation pressure during DFA compiles.
- Optimize
talon.experimental.locate performance.
- Fix: the "Sleep" item in the Talon Speech Recognition menu now calls
actions.speech.disable().
- Fix type inference in .talon files for nested containers.
- Fix an issue if a user script called
sys.exit().
- Fix action/capture docstrings only storing the first line of text.
- Fix parrot issue caused by incorrect timestamp offsets.
- Fix various clipboard problems.
- Fix downloading Conformer model through some corporate proxies.
- Fix
fs.watch() behavior for files that don't yet exist.
- Fix some issues where
knausj_talon might not work on first install until a Talon restart.
- Fix
talon.experimental.locate false positive matches.
- Fix issue where multiple microphones could be active at once.
- Fix issue where hiss noise could get stuck.
- Linux: fix issues opening a terminal for the REPL.
- Linux: run
udevadm trigger after adding udev rules.
- Windows: add Accessibility API.
- Windows: add OCR API.
- Windows: improve tray icon rendering.
- Windows: change key delay behavior to make modifier key presses more reliable.
- Windows: fix some issues related to WMI polling.
- Windows: improve app/window list/event quality.
- Windows: reduce CPU usage while idle.
- Windows:
ui.screens() / ui.main_screen() now have the correct .scale property for DPI scaling.
- Windows: no longer create Talon Console shortcut. Talon Console is deprecated for normal use. It is slower and less reliable than launching Talon normally.
- Windows: move venv from
.venv to venv/3.11 (exact path is based on the Python version).
- Mac: reduce crashes when using Accessibility API to access Talon's own windows from within Talon.
- Mac: fix some memory leaks.
- Mac: change key delay behavior to make modifier key presses more reliable.
- Mac: fix
visible_rect calculation edge case.
- Mac: fix issue attempting to track windows for some apps.
Resolved GitHub issues:
0.3.1 (Jul 28, 2022)
- Inputs such as voice commands and hotkeys are now disabled when your screen is locked.
- Add Python include/ directory to Talon's virtual environment.
- Add a compatibility shim for some
knausj_talon usage of the deprecated talon_plugins.eye_mouse.
- Fix
control1_item.toggle() action.
- Linux: fix Control Mouse 2 when primary monitor origin isn't (0, 0)
- Linux: fix libffi issue preventing launch on RHEL / Centos 8
- Windows: fix https://github.com/talonvoice/talon/issues/516 - pip install fails with no .egg-info
0.3.0 (Jul 7, 2022)
Breaking changes:
from __future__ import annotations does not work with runtime type checking. Talon now throws an error when you import annotations.- Remove
m._words[...] attribute in captures. You should just index m[...] directly.
- Language support: commands, lists, and captures are now implicitly scoped to
en_US (English US).
They will not be available in other languages by default.
You should reimplement them for your specific language, e.g. to match German, use:
language: de_DE at the top of a .talon file.ctx.matches = 'language: de_DE' in Python.
- Mac:
element.children.find(limit=) has been replaced with the less ambiguous element.children.find(max_results=).
Eye Tracking:
- Add a new Control Mouse mode. The old mode is still available as Control Mouse (Legacy).
- Add actions for the Eye Tracking menu, see
actions.list('tracking') for a list.
- Mouse control modes will now stay enabled if you restart Talon.
General changes:
- Significantly optimize Conformer.
- Significantly optimize grammar compilation.
- Significantly reduce Talon app size.
- Reduce Talon memory usage.
- Optimize
talon.experimental.locate API.
- All platforms now have hotkey support. You can bind hotkeys like
key(shift-f1): in a .talon file. You can also suffix with :up or :repeat, e.g. key(shift-f1:repeat): will run repeatedly while shift-f1 is held.
- Add MIME clipboard APIs.
- Add skia Path type.
- Constrain eye tracking calibration UI to a 27" monitor size when calibrating on larger monitors. This should make calibration easier.
- Add timing information to list-captured words.
- Fix issues with engine word timings.
- Fix some imgui rendering and placement issues.
- Fix issue where speech engines didn't show up in the menu on the very first launch.
- Fix command collision issue for non-Latin alphabets.
- Fix actions.key with multiple spaces between keys.
- Fix some cases where host Python environment variables could mess up the Talon Python environment.
- Mac: Add
max_visited, max_results, visible_only, and timeout params to find() and find_one().
- Mac: Add
ui.element_at(x, y).
- Mac: Add
talon.experimental.media API to interface with system-wide media playback.
- Mac: Add provisional
talon.experimental.ocr API.
- Mac: Add extended unicode / emoji key support to
actions.key() and ctrl.key_press()
- Mac: Improve performance and behavior of accessibility API and window management.
- Mac: Implement == and != equality for ui.Element objects.
- Mac: Fix glitchy imgui rendering.
- Mac: Fix screenshots to include embedded colorspace.
- Windows: prefer "Default Device" microphone over "Default Communications Device".
- Windows: add hotkey support in .talon files, e.g.
key(shift-f1): ...
- Linux: add hotkey support in .talon files, e.g.
key(shift-f1): ...
- Linux: add
talon.experimental.media API.
- Linux: hide "output monitor" audio sources.
- Linux: fix TextArea placement when no monitor is at y=0
- Linux: fix Start on Login
Resolved GitHub issues:
0.2.3 (Sep 30, 2021)
0.2.2 (Sep 23, 2021)
- Fix blank window appearing on Windows.
0.2.1 (Sep 22, 2021)
Now introducing the much more accurate Conformer b108 model, which you can install directly from Talon's Speech Recognition menu.
Upgrade to the Early-Access Talon Beta to get more Talon per Talon, including:
- Lighting fast (4-10x faster) speech recognition and command processing (coming soon!).
- Multi-language Vosk engine for dictation.
- Multi-language Webspeech engine for dictation.
- Parrot.py integration.
- Mac: face expression recognition on macOS Catalina and newer.
- Gratitude from the developers. Thank you, your support helps keep Talon possible!
Changes in v0.2.1:
- Add wav2letter Conformer model support.
- Add wav2letter downloader in the Speech Recognition menu.
- Add wav2letter no-LM mode, which uses much less memory.
- Add f0 freqeuency estimation for Parrot.
- Allow selecting version text in "About" window.
- Improve loading and selection of speech engines.
- Improve downloads over unreliable Internet connections.
- Fix parrot mic tracking.
- Fix
skia.Image.from_file() leaking file handles.
- Fix an edge case in the default
win.file_ext() action implementation.
- Fix screen state tracking on all platforms.
- Fix some eye tracking bugs.
- Mac: automatically toggle face expression recognition system based on context.
- Windows: fix potential Talon hang when showing a window, when the target window is hanging.
- Linux: support
_GTK_FRAME_EXTENTS when calculating window rectangle.
Resolved GitHub issues:
0.2.0 (Jul 17, 2021)
Changes in v0.2.0:
- Deprecate
action(): ... in .talon files for removal in Talon v0.3. You should reimplement actions on Context objects in Python now (which was already possible).
If you use knausj_talon, you should consider updating to the latest version so you have this PR: knausj85/knausj_talon/pull/482
Otherwise, from the menu under Scripting -> Console (REPL), you can run actions.migrate.v02_all() to perform an automatic migration of your eisting .talon actions.
This migration will first back up your user directory.
- Add extended Speech Recognition menu.
- Add on-screen subtitles for spoken phrases.
- Add experimental word-level timing information for wav2letter.
- Add model naming for wav2letter.
- Add web2letter geolocal server support.
- Add new
sim("phrase") command in the REPL.
- Add better flac comments for
speech.record_all = 1.
- Add options to events.tail().
- Add
speech.record_labels = 1 setting to record Audacity text labels. You can import these in Audacity by loading the corresponding flac file, then going to File -> Import -> Labels.
- Add actions for some Talon menu items, see
actions.list('menu').
- Change
storage API: add an implicit prefix (such as user.) to storage keys.
- Remove tray icon color, switch to a shape-based sleep indicator.
- Remove Eye Keyboard, Show Camera, and Show Debug Overlay (for now).
- Fix type checking for action implementations at load time.
- Fix
actions.find() decl attribution.
- Fix FLAC decoder leaking file descriptors.
- Fix wav2letter audio padding.
- Fix using a defaultdict for registry attributes. This will improve reported errors.
- Fix webspeech host/port arguments.
- Fix behavior of
settings.get('path', default=...) to not override module default.
- Fix bugs in events.tail() when event log was very long.
- Fix parser issue when capture rules were referenced but not defined.
- Fix DFA linker not writing missing rule messages to the log.
- Fix DFA linker not writing unknown token messages to the log.
- Fix line numbers in some TalonScript debug output.
- Fix fs watch recursing into VCS directories.
- Fix fs watch not working with file paths.
- Fix resource.open() not working outside the user/ directory.
- Fix readability of warnings printed to the log during startup.
- Fix log messages caused by code run in the REPL not being printed to the REPL.
- Fix automatic resource management from the REPL.
- Fix Start on Login checkbox tracking on Mac/Linux when you move the app.
- Fix update state machine not allowing Check for Updates after the "You're already up to date" dialog is dismissed.
- Fix issues related to incorrect microphone selection after unplugging a microphone or waking from sleep.
- Fix manual mouse override for Control Mouse on Linux/Windows.
- Optimize Talon app exit time.
- Mac: wav2letter speech engine is now Universal2 and runs natively on Apple Silicon.
- Mac: fix memory leak in wav2letter.
- Mac: fix accessibility attribute types (e.g. {$rect2d} -> Rect).
- Mac: fix accessibility element over-caching.
- Mac: fix errors related to empty ui events.
- Windows: add "Start on Login" support.
- Windows: add system sleep/wake events.
- Windows: fix issue where slow Talon exit may break update extraction.
- Windows: improve timing precision.
- Windows: vastly improve
cron precision and jitter.
- Windows: fix crash related to esoteric DLL load order.
- Windows: fix update taking too long if files are locked.
- Windows: fix crash when using RDP remote audio as a microphone.
- Windows: fix application names typically being the .exe name.
- Windows: add Z-ordering for ui.windows().
- Windows: fix detection of app names (MUI Cache).
- Windows: fix behavior issues focusing windows, including minimized or maximized windows.
- Windows: slightly improve app startup time and binary size.
- Linux: add "Start on Login" support.
- Linux: add system sleep/wake events.
- Linux: fix taking screenshots of secondary monitors.
- Linux: improve international keyboard layout support.
- Linux: rewrite X11 client to fix hangs and other bugs.
REPL improvements:
- Add support for reading from sys.stdin.
- Windows: repl.bat no longer spawns a secondary console window by default.
- Windows: add tab completion and history support.
- Linux: fix tab completion, history, and line editing hotkeys.
Resolved GitHub issues:
0.1.5 (Mar 10, 2021)
- Increase the default wav2letter speech timeout from 150ms to 300ms. This means Talon will respond to commands slightly more slowly, in exchange for more reliable speech recognition for longer commands and dictation. You can restore the old behavior by adding
settings(): speech.timeout = 0.150 to any global .talon file.
- Improve Talon startup time.
- Improve DFA minimization time.
- Improve error and watchdog tracebacks.
- Improve handling of eye tracking errors and calibration screen.
- Add new
app.register("ready", cb) event, which functions like app.register("launch", cb). However, unlike the launch event, ready will also be called immediately at registration time if Talon was ready.
- Update Skia, and improve Skia <--> numpy interoperability.
- "View Log" now shows the entire log from the current Talon run.
- Fix a race condition in cron.
- Fix clipboard mode support (e.g. "find" clipboard on Mac and "select" clipboard on Linux).
- Eye Tracking (Control Mouse): reset mouse movement timer when eye tracking is enabled.
speech.record_all: don't record when Talon is asleep.- Linux: Fix error when launching Talon under a virtualenv.
- Windows: Code sign all EXE and DLL files (instead of just signing the main exe).
- Windows: Rework key codes to fix #243 and #188
- Mac: Improve
canvas.focused = True behavior.
- Mac: Fix cmd modifier when used with accent keys (like ^ and ~)
- Mac: Optimize UI enumeration time on Talon startup.
- Mac: Add a Metal renderer for Canvas (which was 3x faster than OpenGL in simple tests).
REPL improvements:
- Tab completion should now work in more cases.
- Fix text offset issues when you press Up to go through history on a freshly loaded REPL.
- Handle KeyboardInterrupt while connecting.
- Mac: Pressing tab on an empty line should now indent without a 1-keystroke delay.
Resolved GitHub issues:
0.1.4 (Jan 6, 2021)
- Improve "Control Mouse" mode. It should now be usable on the Tobii 5. More improvements to come.
- Refactored all
Rect and Point2d types to live in talon.types, e.g. from talon.types import Rect, Point2d. This means rect.contains() and rect.center now use and support Point2d values, and ui.Rect is skia.Rect now.
speech.record_all = 1 now writes microphone name and spoken text to the file as FLAC comments, and better handles very long file paths.- Fix imgui flicker on show.
Resolved GitHub issues:
0.1.3 (Dec 25, 2020)
Added:
- Talon now uses Python 3.9.1.
- Talon is now a Universal 2 App on macOS, with native Apple Silicon support.
- Dragon recording is now supported with
speech.record_all = 1
- Added macOS engine for Dictation Mode (early-access beta only).
- Added Vosk engine for Dictation Mode (early-access beta only).
Added actions:
core.repeat_partial_phrase() action.core.cancel_phrase__unstable() action.speech.set_microphone(name: str) action.
Improved:
- Deprecation warnings are now turned on for Python.
- Documentation and error message updates.
- clip.capture(): s.get() will now throw an error if incorrectly called before block exits.
- .talon files can now use consecutive
and on identical match keys like mode and tag.
actions.next() now works if the next action in the chain is Module-implemented- TalonScript format strings are now properly type checked.
- TalonScript errors have been improved, which includes the path and line number of the erroring script.
- Add a check preventing rules (captures or commands) from being "completely optional". Parsing for such rules was already broken, so now there is a check preventing you from creating them.
This means a capture such as
@ctx.capture(rule="[word]") will no longer be allowed. The check also prevents [optional]: "hello" in a .talon file, which would have been a very bad idea but was previously possible.
- Change the
app.exe scope to match just the basename of the exe, and add app.exe_path to match the full exe path.
- Fix issues related to editing engines.py without restarting Talon.
- General parser reliability and performance improvements.
- General imgui improvements.
- Dragon word parsing improvements.
Optimizations:
- Optimize a quadratic rule parsing case, which should improve the performance of numbers in knausj.
- The Python interpreter lock was not being released during list DFA minimization, which may have affected eye tracking latency. The GIL is now released during all DFA minimization steps.
- Do even less work when updating
Context.lists and the underlying list hasn't changed.
- Implement differential context refresh. This is a significant optimization for the "context refresh" path. This means much less work is performed every time you switch apps, update a list, reload a Talon/Python file, and more.
- record.py is no longer slow for very large recordings/ folders.
Removed:
- Remove unused properties of
Context, such as wrapper actions.
- Remove
Context getters/setters. You are intended to use the property directly, for example: ctx.settings_set(value) should instead be ctx.settings = value.
- Remove
Context.defines as well as commands and hotkeys (these were intended as a private API - instead use .talon files to declare commands and hotkeys).
- Context.reload() is now a private API. With the differential context refresh feature, asking Talon to manually reload a context would have no effect.
- Remove some legacy "core" prefixed actions.
- Remove implicit Capture sharing. Only one instance of a capture is now active at a time.
- Remove inner context list mutation, such as
ctx.lists[name][key] = value and ctx.lists[name].update({'key': 'value'}), because it never worked properly. It now throws TypeError: Cannot mutate word lists.
Windows:
- Fix window rect coordinates.
- Add support for window resize/movement API.
macOS:
- Change Canvas window level.
- Fix some event tap issues.
Resolved these GitHub issues:
0.1.2 (Sep 23, 2020)
- Add global
<digit_string> capture definition (#124)
- Fix updater on bad internet connections.
- Fix View Log tailing beta.log instead of talon.log (#126)
- Fix a list parsing bug (#125)
0.1.1 (Sep 22, 2020)
- Fix canvas scale when dragging between monitors (#92)
- Fix incorrectly lowercasing
<phrase>.
- Improve context matching priority for the
os match.
0.1.0 (Sep 20, 2020)
- Cross-platform (macOS/Windows/Linux) public release of Talon.
- Works with the integrated wav2letter speech engine as well as Dragon.
- Supports the Tobii 4C eye tracker, and has preliminary support for the Tobii 5.
- See the Documentation for getting started information.
- Join the Slack for help and general discussion.
- Report issues here talonvoice/talon or ask about them on Slack in the #help channel.
- To use speech recognition, you need to put user scripts such as knausj_talon in your
~/.talon/user directory, and to set up wav2letter. Instructions for both of these can be found in the Documentation, and further information can be found on the Community Wiki.
Notes for existing Beta users:
- This release does not yet support sconv models, such as sconv-b5.
- This release does not yet support the WebSpeech engine.
- You will need to stay on the Beta for now to maintain support for those features.
Notes for existing Legacy (v0.0.8.42) users:
- Talon v0.0.8.42 will continue to work if you don't want to update.
- v0.0.8.42 will not automatically update to v0.1.0. You must download v0.1.0 from the website and plan to adapt your user configuration if you want to update. The new API is not compatible with scripts written for the old API.
- If you were using Talon v0.0.8.42 on Mac, you will need to back up your
~/.talon/user/ directory and either start fresh with knausj_talon or manually port your commands.
- There is no automatic command porting system to move from (v0.0.8.42) to (0.1.0) right now. We will consider making one if enough people ask for it on Slack.
- See the Unofficial Docs for information on the new .talon files for defining voice commands.
0.0.1 - 0.0.8.42 and beta versions (2017-2020)
- This is a historical note.
- The earlier Talon release notes have been removed as they were long and no longer relevant.
- Talon was previously not cross-platform, didn't have .talon files, and didn't have the wav2letter speech engine.