Global always-on event handlers?


(Nolan) #1

I have an odd use case. I’d like to use Amethyst for audio games targetted at blind/visually-impaired players. One feature I’d like to add is speech access to Amethyst UI components. Essentially, this will involve making the UI’s event model a bit richer, intercepting UI events, and speaking some meaningful text. For instance, hovering over a button labelled OK might speak “OK: button” via an as-of-yet-unwritten text-to-speech crate.

I’d like to make this work as generic and easy to integrate as possible. It looks like the current UI generates a handful of events. I imagine most of those will be handled in states specific to the UI (I.e. a launch menu will intercept events and perform actions on those) but what I’d really like is a way to globally intercept all UI events outside of a state, then perform actions on those events in ways that don’t have any side effects. I’d also like to make it as easy and as setup-and-forget as possible. IOW, I don’t want to have to integrate accessibility into every UI state. I’d rather grab the UI event channel on startup, integrate my handler, then have every UI become automatically accessible.

Is there any way to achieve this? I don’t think systems are what I’d want since I’d then need to integrate the handler into every system. Likewise with states. Ideally I could just create an amethyst_accessibility crate that exports a resource which, when the game launches, automatically injects accessibility helpers that work across states and systems when added to the world. But I don’t know that this is currently achieveable, or what I’d need to request to make it possible.

Thanks!


(Joël Lupien) #2

Hi! This is an excellent idea. The Ui is under rewrite, so there will be improvements coming that will make this easier. However, handling the events can already be done. For this, you want to use a System reading from the EventChannel. I didn’t really understand why you were against using systems for this, is it is their usage… You question made me realize that wr should add a Selected UiEvent variant. I will add this to me text.rs refactor pull request on github which should hopefully get merged somewhere next week.

As for TTS, I didn’t do any research on it yet. If you find a crate that does it, the best would be if it either

  1. Generate audio data (stream or Vec).
  2. Uses rodio as a backend.

Feel free to ask any question I missed or didn’t answered fully.


(Nolan) #3

Cool, thanks. I suppose I could use a system. What I was hoping for was something I wouldn’t necessarily have to add everywhere I need a UI (I.e. if I need a start menu, in-game UI, pause menu, etc. then I’d need to add the same system in each state.) If there’s a shortcut to keep a system active regardless of state, that would work as well. Essentially, in these games, every UI element would need to be accessible, so I wanted to achieve that without having to declare it in every state. :slightly_smiling_face: :
In terms of TTS, I’m writing my own crate for that. I just wanted a mechanism to globally receive every UI event, whether that’s a resource that can grab the event channel, a system that gets automatically added to every state, etc.

Thanks.


(Joël Lupien) #4

By default systems are global if registered on GameDataBuilder (what the tutorial shows in the book).
A system is definitely what you want.


(Nolan) #5

Ah, then you’re absolutely right. I thought systems were tied to individual states, so if I had multiple states with their own UIs, then each would need its own copy of the UIAccessibility system or whatever. Thanks, I’ll give that approach a shot!


(Hilmar Wiegand) #6

In an upcoming PR of mine it will also become easily possible to basically “react” to any event that gets dispatched with another one, conditionally or unconditionally. This would make it very easy for you to connect everything that’s supposed to have your functionality up to your text to speech service, I suppose.
All in all I agree with jojo though, you would want a system that always runs and reacts to ui events. I think the more complicated part will be the audio output, since optimally, you would want to share Amethysts audio output.