Skip to content
This issue has been moved to a discussionGo to the discussion

Integrate with Unifi Protect Video Events #164

@oronaldosantos

Description

@oronaldosantos

Is your feature request related to a problem? Please describe.
When using video analyzer there isn't a way to use the video snapshots from Unifi Protect.

Describe the solution you'd like
It would be amazing for more than 14,636 users of Unifi Protect to be able to use the video snapshots from recording events such as motion detections, veichle detections and etc togetter with LLM Vision.

Describe alternatives you've considered
I've tried to do some workaround using the API URL provided by Unifi Integration like /api/unifiprotect/thumbnail/{{ config_entry_id(trigger.entity_id) }}/{{ trigger.from_state.attributes.event_id }} but seems that some authentication is required and the video snapshot is not accessible for LLM Vision preventing me to use those videos on a Vídeo Analysis automation.

Activity

valentinfrlch

valentinfrlch commented on Jan 8, 2025

@valentinfrlch
Owner

Sounds like a good idea! I don't have UniFi Protect cameras or use their NVR, so I can't test this. If someone does, PRs are welcome!

It seems like the UniFi Protect Home Assistant Integration would expose events in the media browser (https://www.home-assistant.io/integrations/unifiprotect/#media-source).

Flintr

Flintr commented on Jan 16, 2025

@Flintr

I worked around it by capturing the video to the www folder, then passing it to Video Analyzer:

alias: Unifi LLM Vision
description: Sends a notification with video upon motion detection.
triggers:
  - entity_id:
      - binary_sensor.g5_bullet_motion
    trigger: state
    from: "on"
    to: "off"
    enabled: true
conditions: []
actions:
  - action: camera.record
    metadata: {}
    data:
      filename: www/g5.mp4
      duration: 3
    target:
      entity_id: camera.g5_bullet_low
  - action: camera.snapshot
    metadata: {}
    data:
      filename: www/g5.jpg
    target:
      entity_id: camera.g5_bullet_low
    enabled: true
  - action: llmvision.video_analyzer
    metadata: {}
    data:
      remember: false
      max_frames: 1
      include_filename: false
      max_tokens: 100
      temperature: 0.2
      generate_title: false
      expose_images: true
      expose_images_persist: false
      provider:
      model: llava-1.6-vicuna
      video_file: www/g5.mp4
      message: >-
        You are an AI model analyzing security camera footage. Your task is to
        generate a concise, plain-text notification describing the most relevant
        activity in the scene.


        Instructions:

        1. Analyze the input footage

        2. Identify the most important motion or activity (e.g., people,
        animals, objects, or actions).

        3. Respond with one sentence in plain text, formatted as a concise
        notification. Do not include any tags, symbols, or extra formatting.
        Output should only contain the notification text. Do not repeat any part
        of the prompt in your output.

        4. Do not prefix the sentence with any text

        5. Ignore the text in the top left corner of the frame.
      target_width: 1280
    response_variable: response
    enabled: true
  - action: notify.iphone
    metadata: {}
    data:
      title: Tool Room Motion
      message: "{{ response.response_text | replace('</s>', '') }}"
      data:
        image: /local/g5.jpg
        video: /local/g5.mp4
    enabled: true
mode: single
max_exceeded: silent

bf8392

bf8392 commented on Feb 16, 2025

@bf8392

Is there no way to integrate a unifi protect camera directly? I tried the blueprint with it, but it throws errors in the logs...

Repository owner locked and limited conversation to collaborators on Mar 21, 2025
converted this issue into a discussion #283 on Mar 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @oronaldosantos@Flintr@bf8392@valentinfrlch

      Issue actions

        Integrate with Unifi Protect Video Events · Issue #164 · valentinfrlch/ha-llmvision