Lags in 2k/4k resolution

[Konnor378]

Hi. I can't watch videos in 2k/4k format. When I watch a video even in 2k, I get video lags and the video itself is like 5fps. At the same time the backlight works correctly. As a server I use my personal server pc on windows server. I suspect that capturing video in 4k TV consumes too many resources. Is there any way to reduce the capture resolution in the app? I think 720p or lowest is enough for the backlight to work, because the backlight only needs to see the colors, not the picture itself.

Another hyperhdr developer suggested that the hardware acceleration feature is disabled when using hyperion android grabber, so videos lag in high quality. Kodi has hardware acceleration for amlogic processors, which helps for TV set-top boxes, but I have android tv embedded in my TV with Mstar processor (mi tv 4s). Will this ever be implemented?

[haze1986]

Encountered the same issue here for Mi TV 4S, wondering if you have found any workaround?

[Konnor378]

Encountered the same issue here for Mi TV 4S, wondering if you have found any workaround?

There is no solution yet. Video works only with a resolution of 1080p and less, otherwise you need to modify the backlight using an android set-top box. You can also connect a PC to a TV and use hyperion screen capture, but this is not a convenient option for viewing content, in games such a bundle works well.

[cr0wdelex]

I also have the same problem.
TCL TV CPU could not handle the capture in 4K,2K.

A decrease in the MEDIA_PROJECT resolution could help?

[Konnor378]

@cr0wdelex ,
Reducing the capture resolution should help, but this needs to be implemented in the interface. The author apparently abandoned the project, which is a pity. The application needs improvement, at least because hardware acceleration is disabled when it is running

[cr0wdelex]

@Konnor378
So the app automatically disable hardware acceleration? I don't remember if we need to manually switch it off anywhere else.

[Konnor378]

@cr0wdelex
Yes, when the application is running, hardware acceleration is automatically disabled, or it is simply engaged in image capture. In any case, because of it, it is impossible to watch in quality higher than 1080p, at least this is what the author of hyperhdr assumes.

[cr0wdelex]

@Konnor378
I don't think it disables hardware acceleration.
My TV when process 4k 24fps video in Smart Tube already took up to 80% CPU. So when plus with casting from Grabber with 4K it will be more than CPU could handle.
My TV (2023 model) use MediaTek Pentonic 700 (MT9653) which is base on a SoC launched in 2016 with only 1.4 GHz. Very weak compares to even a cheap smart phone at the moment.

Tested with phone could handle 4K easily because the CPU in phone is much more powerful compare to most of android TV.

I will try to grab a friend know Java and see if they could help me to modify the MEDIA_PROJECT resolution.

[Konnor378]

@cr0wdelex
I think 480p or 360p capture resolution is enough, some people use even lower resolution. If you fix the problem, I kindly ask you to post it in this thread.

[cr0wdelex]

@Konnor378
It sound so dumb if hardcoded the value but this is the only way I could do now.
Could you let me know which TV are you having? need the exact resolution and the Screen Diagonal in inch?
I will try to build an app will cast at 240p for your device and you could test it.

[Konnor378]

@cr0wdelex
I meant that when using a separate capture card and a wired connection to the server, you can change the screen capture resolution directly in hyperion/hyperhdr, but this only works if you have a separate TV box. TVs only have HDMI input, and the box has an output. My TV is a 43-inch Mi TV 4s on android tv 9.

[cr0wdelex]

@Konnor378 Sorry for disappoint you but I just test it could run normally then the lag come back.
Will check in the code again

[haze1986]

I can help test if you share the file here. Mi tv 4s and tcl c755 user

[krisstakos]

I have some idea(s) for increasing the performance:

1. What if we capture ridiculously low resolutions(60:34 for ration 16:9)? Realistically you don't even need 720p even 360p that's way too much data for lighting ~200leds. If we directly capture at very low res it will require zero post-processing hence further easing the load on the CPU. Now I see some operations happening for each pixels
2. Ease the pixel calculations. I didn't invest much time on the logic there but if we process any other pixel will also benefit the performance
3. Reducing framerate pulling. Again I didn't dive much in the code but the scenes doesn't change that much to capture 30/60fps, something like 3-5fps will sufficient I guess. Experiments will help. Even better if it's configurable through the app

Overall it matches what you are trying to do here: reducing the capture resolution so it's faster the processing

[Konnor378]

@krisstakos

1. This is what we are trying to do. The program simply does not have a line with changing the screen capture resolution. In addition, if you reduce the resolution so much, then the Anti-aliasing item will work very poorly or incorrectly, and anti-aliasing greatly improves the perception of the backlight
2. In this case, the accuracy of the LED reproduction will suffer.
3. Reducing the frequency will look very bad when the picture changes often. Even between 60 and 30 fps the difference is strongly felt, and at 3-5 fps the diodes will change too sharply

[krisstakos]

1. 2. Yeah, I see it doesn't have it I'm just saying how we can optimize it. I'm not saying to reduce resolution too much, matching roughly the NEOPIXELS count per side should be enough. Doing that GPU will handle it, not post-processing it on the CPU.
      How "Anti-aliasing" is working(not sure what you call that here)? Isn't just crunching the high res and averaging hundreds of TV pixels just to output single RGB value for the neopixel roughly in that place? Isn't that downscaling with extra steps?
2. Yeah probably that will be bad but it could be a desperate option to handle the high load

[mheinr]

Reducing the number of LEDs also reduces the resolution of the VirtualDisplay.

[Konnor378]

@krisstakos
Sorry, that's right, it's called smoothing. I don't speak English completely, so I use a translator. In my opinion, it's better to watch movies in 1080p with normal backlight operation than to use hacky methods with a strong deterioration of ambilight

[krisstakos]

Reducing the number of LEDs also reduces the resolution of the VirtualDisplay.

You are right, it actually does almost exactly what I tried to describe

EDIT: I have sorta similar spec TV as the OP and noticed slightly dropped frames under 4k HDR stream, so I guess the horse power is not quite enough. Imagine trying to do more processing on top of that :( bummer

[haze1986]

@krisstakos Sorry, that's right, it's called smoothing. I don't speak English completely, so I use a translator. In my opinion, it's better to watch movies in 1080p with normal backlight operation than to use hacky methods with a strong deterioration of ambilight

On the contrary, some of us rather 4k viewing and low resolution ambilight. Thanks.

[cr0wdelex]

I tried to set led number to 1x1 and my TV CPU still could not even handle 2K contents.
I don't have enough knowledge to work with Android MediaProjection.
Gave up, guys 😂 now we could only blame why they use so weak CPU in our TVs

[abrenoch]

Hey @cr0wdelex respect for taking a look 'under the hood' but you seem to have arrived at the same conclusion I have a while ago - there isn't really all that much that can be changed to make the grabber more efficient unfortunately. The MediaProjection API (last I checked at least) is the only way I'm familiar with to grab the content of the screen, and when you mix different manufacturers and DRM into the picture it results in an API that is completely inconsistent in how it actually behaves. I originally wrote this to run on a shield TV and I guess I just got lucky since that device seems to be a powerhouse.

I don't really use the app or hyperion these days, which is why this project has fallen into a state of neglect. Not to mention some life changes has left me without much time to work on this. I may look into a maintenance release, but if anybody has already gone through the trouble of updating the APIs I would welcome a PR!