Getting snapshots of specified timestamp

[benjholla]

The protect API has a snapshot endpoint parameter of ts /cameras/{camera_id}/snapshot?ts={timestamp} but currently the snapshot command hardcodes the current time. It would be great if we could specify a timestamp for which to retrieve the snapshot.

Sadly, playing with the API it seems this parameter is actually ignored and I always get a latest camera snapshot (re-requesting the same timestamp returns new updated images from protect). Maybe this is why this project hardcodes the ts parameter to now? There are a few other approaches I thought of, but haven't had any success yet.

1. The timelapse API appears to be using web sockets to send specified snapshots.
   https://{ip}/proxy/protect/api/ws/timelapse?camera={camera_id}&channel=0&end={timestamp}&format=FMP4&reverse=true&start=0 returns JSON with a URL containing a unique id that resolves to the download URL {"url":"wss://{ip}:7443/ws/timelapse?uniqid={ws_unique_id}"}

2. Download a very short video using existing framework with timestamp set as start and minimum interval end from start then use ffmpeg or another utility to extract the first frame.

[danielfernau]

Great suggestion, thanks! Will definitely look into that at some point.

If I remember correctly, there is an endpoint somewhere that can return snapshots for specific timestamps – but it's not the snapshot one that is part of each individual camera path...

A first guess from my side would be that the ts parameter is not used for specifying a snapshot time but rather to circumvent browser/app content caches. By having an always changing part within the request URL they can make sure that the most recent image is returned in the browser/app view instead of one that might already be stored somewhere in the browser cache from a past request.

[benjholla]

Your theory about the ts parameter to avoid stale browser caches makes total sense. The timelapse endpoint does allow you to get a timestamped snapshot, but its using websockets and it looks like a bit more work to get. I tested by using chrome to watch the network requests and then opened one of the urls returned by the websocket json.

I haven't found anything easier yet. I was thinking it'd be neat to use this to make a retroactive timelapse of a camera (retroactive in the sense that it has already been recorded), instead of using a cron job to actively grab snapshots over time like https://lazyadmin.nl/home-network/unifi-protect-timelapse/.

[benjholla]

I poked around some more through a combination of inspecting network requests on the https://<server>/protect/timelapse/<camera_id> page and looking at the web sockets, but its not clear to me how to get at the snapshots for a given timestamp.

Moving the time bar slider around in the UI generates requests to a few "proxy" APIs.

• https://<server>/proxy/protect/api/ws/timelapse?camera=<camera>&channel=<channel>&format=<format>&start=<start>&end=<end> gives JSON to URL with a unique id to open a web socket (unique ID seems to act like a session token). The <start> and <end> parameters are not clear to me, I see one usually is set to 0 and the other is the timestamp of the slider bar. The <format> was FMP4 and channel was always 0 in what I observed. I'm not sure but I think this websocket gives a stream of previews for the time bar slider.

• https://<server>/proxy/protect/api/ws/playback?camera=<camera>&channel=<channel>&format=<format>&start=<start>&end=<end> gives JSON to URL with a unique id to open a web socket and all the parameters acted the same as what I saw on the timelapse API. I think the websocket for playback gives a constant stream of frames for the video player after the time slider is released.

For both the timelapse and playback websockets I was able to write some Java code that authenticates with Protect, requests the websocket url and connects to the web socket. On connect protect sends a byte array that is mostly JSON in a hex dump with something like {"success":true}�������X{"start":1619306343157,"end":0,"channelName":"E063DA011514_0:timelapse:0-1625959382546"}. Then watching the web socket in Chrome I see the client is supposed to send a reply, but it's some non-ASCII value I don't understand (I suspect something tied to the video player display syncing logic). It's not clear to me that there are standard image files in this data, but obviously the player knows how to decode this data.

Some code to reproduce these observations is here (https://github.com/benjholla/UnifiProtectClient/blob/f9f4b7e6c26d1e3008d7e181330ca917bee112af/UnifiProtectSnapshots/src/snapshots/UnifiProtectClient.java), but since I don't know what to send back after connecting to the server this path isn't looking promising. For my purposes I might just go with the easy way out of grabbing the smallest video clip I can starting at the timestamp I want and then just extracting the first frame. Ideally there would be a better way though.

• Also of interest is
  https://<server>/proxy/protect/api/events/<some_id_maybe_an_event_id>/thumbnail?h=35&w=62, but I'm worried this is only the Protect events like a motion detection and its not clear to me where the id comes from yet. The h and w specify the dimensions and not supplying these parameters gives a relatively small default, but interestingly the h and w parameters can be set values larger than the default to get a larger image.

[benjholla]

Ok the quick and dirty way of download video and extract first frame works pretty good. If you set start time and end time to be the same the resulting video ends up being a few seconds and around 2mb downloads. If you're local this probably isn't a big deal.

Here's some Java code that calls ffmpeg to get the first frame.
https://github.com/benjholla/UnifiProtectClient/blob/4cf48faa465a7a8a9195a11d5485a529a12ea6c5/UnifiProtectClient/src/upc/UnifiProtectClient.java#L137

[danielfernau]

Thanks a lot! 👍
I'll have a look at your discoveries and code soon and hope to be able to implement this functionality in the near future. Will post related updates here.