Skip to content

Wesxdz/ui-idm

Repository files navigation

User Interface Inverse Dynamics Model

IDM used in MineRL here https://github.com/openai/Video-Pre-Training General Inverse Dynamics Model for Linux desktop environments. Collecting generic desktop navigation data is a security and privacy hazard, yet this data is priceless to teach AI agents how to use computer user interfaces: coordinating between multiple applications over long periods of time, just like humans do.

Here I open source tooling for data collection to self-train user interface navigation for agents.

Use an up-to-date version of OBS with websocket/remote control builtin. Tools->WebSocket Server Settings-> Enable WebSocket server Disable Authentication (or update idm.py file)

About

User Interface Inverse Dynamics Model

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published