Pan/Tilt Module Controls

[logicbomb421]

I just wanted to get the ball rolling on figuring out how to control the pan/tilt module via an API.

Setup

I have determined that the pan/tilt module shows up in the response from GET /api/v3/accounts/{AccountID}/homescreen as an accessory of type rosie:

{
  ...
  "accessories": {
    ...
    "rosie": [
      {
        "id": 1881,
        "serial": "G7...GH",
        "type": "rosie",
        "connected": true,
        "calibrated": true,
        "target": "owl",
        "target_id": 377378,
        "created_at": "2023-06-20T04:38:48+00:00",
        "revision": "01"
      },
      ...
    ]
  }
}

Searching

I've captured all traffic that occurs when viewing a live stream where a pan and/or tilt action is executed, however, there does not appear to be a separate API call made when interacting with the pan/tilt D-pad. After running through the same test multiple times, the only additional API call I could find with "rosie" anywhere in the body is an initial POST /api/v1/accounts/{AccountID}/events/app, which contains an event of type rosie_lv_session (note that this call is made before any live-view command object has been returned):

{
  "events": [
    {
      "timestamp": "2023-06-23T21:14:05+0000",
      "event": "rosie_lv_session",
      "data": [
        {
          "value": "377590",
          "name": "target_id"
        },
        {
          "value": "online",
          "name": "status"
        },
        {
          "name": "360_button",
          "value": "false"
        },
        {
          "name": "d_pad_controls",
          "value": "true"
        },
        {
          "value": "false",
          "name": "set_home_button"
        },
        {
          "value": "owl",
          "name": "target"
        },
        {
          "value": "false",
          "name": "go_home_button"
        }
      ]
    },
    ...
  ]
}

I am currently unsure if this is the app telling the Blink backend what it is capable of, or if this is somehow actually related to being able to control the pan/tilt functionality. There doesn't appear to be any actual control/movement information anywhere, however, it does seem like the various key/value pairs in the request body above are indicating which buttons are active, as I only ever interacted with the D-pad.

Thoughts

When streaming from a live-view, a command object is returned containing a server key that contains a URI in the following format: immis://{IpAddress}:{Port}/{UniqueID}.

From what I can tell in related blinkpy (fronzbot/blinkpy#343) and homebridge-blink-for-home (colinbendell/homebridge-blink-for-home#27) issues, IMMI[S] seems to be an extension of RTSP[S], which was the original streaming protocol used on the Gen1 cameras. It does not appear the IMMI protocol is documented, or really mentioned anywhere, leading to the conclusion that it must be Blink proprietary. The developer of homebridge-blink-for-home seems to have been in the process of decoding the protocol to get video streaming working, but I have not been able to find anything aside from the code that branches on RTSPS/IMMIS.

My current hypothesis is that the pan/tilt commands are sent through this IMMIS connection that is established after initiating a live-view. This is further supported by viewing the log that is uploaded after a live-view. Located within the log.streamSource object is another object named immiStreamSource that contains two very interesting objects: receivedCountForMediaInfoType and sentCountForMediaInfoType. The former contains a counter named IMMI_DATA_FLAG_ACCESSORY_MSG, and the latter contains a counter named IMMI_DATA_FLAG_INLINE_LV_CMD which had 16 and 18 set, respectively.

---

I will be leaving comments on this issue with any additional findings I come across, and hopefully eventually be able to open a PR to add documentation for this feature.

If anyone has anything to add, or any suggestions, I'd love to hear them!