IMX708: Add support for 4-lane operation

[peyton-howe]

This is just to get the ball rolling, will make sure to cleanup commits, descriptions, and break up the dtoverlay and driver into separate PRs.

I successfully have 4-lane operation automatically set by the overlay option inside the driver, dynamically switching the PLL registers as needed! I have tested max resolution @ 30fps and haven't run into any issues so far. rpicam-hello shows the following framerates and resolutions

• 1536x864 [125.69 fps - (768, 432)/3072x1728 crop]
• 2304x1296 [86.63 fps - (0, 0)/4608x2592 crop]
• 4608x2592 [31.60 fps - (0, 0)/4608x2592 crop]

Is 30fps the upper limit for max resolution on the RPI? I based the settings off what looks to be a 30fps mode for the X2 phone, but this datasheet indicates the max resolution of the sensor is 42fps in 10bit mode.

As for the other the 2304x1296 mode, I just followed 6by9's suggestion of halving the line length paired with the higher link frequency. Maybe more performance can be squeezed out of this one by tuning the PLL settings?

[pelwell]

Leave another comment when you think it is worth rerunning the build checks.

[6by9]

I'll have a more detailed look later.

The datasheet you link to is for IMX 078, not 708.

The main datasheet says max frame rate at full res if 60fps, but that will be at 2.5Gb/s per lane which no Pi supports. 30fps for your 1094Mbit/s data rate seems plausible (just).

Forcing the use of 547MHz link frequency in 4 lane mode is a little annoying. Although dtoverlay=imx708,4lane,link-frequency=450000000 would drop it back down. Does there need to be validation of 2 lane vs 4 lane link frequencies?

[6by9]

You don't want to be changing this register.

[peyton-howe]

I dropped the link frequency back to 450000000 and set the pixel rate to 777.6Mhz and I was able to get a stable output of 28.13fps. I also tried increasing the pixel clock to get closer to 30fps, but it just resulted in artificating and other sensor errors - I tried pixel rates of 854.4Mhz and 801.6Mhz.

I've also tried dropping the line length down to 5216 and tried all the current pixel rates in the driver but I just get artifcating and this libcamera error (still at the 450000000 link frequency):

Sensor minimum line length of 9.21us (500.378 MPix/s) is below the minimum allowable ISP limit of 12.13us (380 MPix/s)

[6by9]

Sensor minimum line length of 9.21us (500.378 MPix/s) is below the minimum allowable ISP limit of 12.13us (380 MPix/s)

will always result in corruption as the CFE overflows.

IMX708 is the first sensor where there have been such distinct line length values that could be supported. Unfortunately it means that the pixel rate and HBLANKing values do need to allow for sufficient time per line.

It's curious that the IVT multiplier is different between different modes of the sensor. It just makes the driver a little more complicated. I can't think of an obvious reason why the highest value can't be used for all modes, but I haven't looked at the full timings to ensure that it can offload the image data over CSI2 within the relevant line period.

There must be a setting of IVT that means the line length of 5216 comes in at >=12.13us

I'm afraid I simply don't have time to play with imx708 at present.
I'd be happy with the simple patch that added 4 lane mode in with the same timings. Likewise adding extra link frequencies. Trying to handle changing pixel rates and HBLANK is going to be more significant, and the risk of regressions will be high.