As many System-on-a-Chips (SoCs) do these days, recent Rockchip SoCs (namely, those of the RK35 generation) integrate dedicated IP blocks for video capture and image signal processing. These additions open the door to a wide range of interesting multimedia applications. However, support for these blocks in mainline Linux remains one of the last missing pieces in an otherwise well-supported SoC lineup. It is time to close that gap!

The mission

Over the last few years, Collabora has invested significant efforts into mainline support for the Rockchip RK3588 SoC. Our RK3588 upstream status matrix and the recent review blog post Running Mainline Linux, U-Boot, and Mesa on Rockchip: A year in review show the success of this mission. It is fair to say that together with the vibrant linux-rockchip community and Rockchip themselves, we were able to provide upstream support for most of the features of the RK3588. On the other hand, there are still a few unsupported hardware blocks that need to be taken care of.

The problem

Video capture blocks and image signal processors (ISPs) notoriously fall into the category of hardware blocks that gain mainline support at a late stage (if at all). The reasons for this are manifold: while the user base is substantial, it's smaller than that of other features, such as video output for instance. The hardware documentation is often not available, at least for ISPs, which seem to be considered special in terms of intellectual property. Then, the underlying hardware is not exactly trivial, making it difficult (if not impossible) for a hobbyist contributor to tackle in their free time. Even for companies, the initial efforts required to bring up that sort of hardware are a time-consuming and hence costly endeavor. As a consequence, users of these powerful hardware blocks tend to resort to a vendor Linux kernel, which can lead to regulatory compliance issues, such as under the Cyber Resilience Act (CRA).

(Towards) the solution

In a quest to overcome this deadlock, Collabora decided to start with the first step: bringing up the RK3588 video capture (VICAP) unit. It was obvious from the very start that this would not be a sprint, but a marathon. The plan was for the rkcif driver to support the RK3588 VICAP, and this driver had been under discussion on the respective mailing lists since 2020!

Collabora joined the mainlining discussion in early 2022 and the patch series went through nine iterations driven by Bootlin. Then, a major rework towards a media-controller centric V4L2 driver was requested. This led to a complete refactoring of the driver and a renewed push towards inclusion in mainline Linux.

After another seven iterations of the resulting patch series and renaming the driver twice, the work was presented at Open Source Summit Europe 2025 in Amsterdam. The talk "Towards Mainline Video Capture and Camera Support for Recent Rockchip SoCs" provided an overview of the software stack for modern multimedia SoCs and covered the contributions that had already landed in mainline and were currently in flight, respectively.

A few iterations of the rkcif driver later, the basic driver providing support for the PX30 VIP and the RK3568 VICAP was accepted (October 2025). After more than five years of development, including 25 iterations and three renamings, this was a major milestone. On the other hand, there was still a lot to do, of course. For instance, the Rockchip MIPI CSI-2 receiver unit that is coupled closely to the VICAP required a mainline driver as well. Again, the hardest part in software development proved to be naming things. After several iterations and three renamings and/or relocations, the driver found its name and place in the mainline Linux kernel (January 2026).

This came to pass right in time for FOSDEM 2026 in Brussels, where Collabora presented "Upstreaming Progress: Video Capture and Camera Support for Recent Rockchip SoCs" Apart from the refined and updated status matrix (see slides 15 and 17), the talk featured a first picture made with a Sony IMX415 sensor attached to the RK3588 VICAP. Of course, in this setup the raw image data from the sensor was debayered in software (which yielded an abysmal frame rate of 1 fps) and there was no image processing such as auto white balance (which explained the greenish image), but at that time it represented a significant stage of this journey.

A few extension patches to the rkcif driver were required to add RK3588 support. Those patches had been in development for some time, and were recently sent out for review.

The next step(s)

Naturally, we all would like to use the RK3588 ISP to process the images from the camera sensor in the dedicated hardware. To that end, we essentially need three things.

The first of the remaining issues is the support for the RK3588 VICAP MUX-TOISP unit, which constitutes a direct hardware connection that passes the data from the VICAP to the ISP(s). Technically, this is not strictly needed. As of now the RK3588 VICAP is able to stream the raw pictures into memory and the ISP can fetch the data from memory. This approach allows for more elaborate use cases, such as processing more than one camera stream per ISP (time-division multiplexing). However, for the simple use case with one camera stream per ISP, this has a couple of disadvantages: Amongst other things, it results in increased memory bandwidth requirements as well as increased latency. Also, it should be noted that the direct hardware connection affects the design of the software architecture for these (otherwise independent) hardware blocks. It is important to take this connection into consideration early on, otherwise it might be very hard to add support for it at a later step. Thus, apart from controlling the interaction with the hardware, this step requires considerations as to how to integrate the kernel drivers into the V4L2 media controller framework. Initial testing has already shown the first signs of life from this unit, though it will take some time until the code is ready to be integrated into mainline Linux.

The second issue is the missing Linux kernel driver for the RK3588 ISP, as the vendor kernel driver is not upstreamable for various reasons. In an effort to aid upstreaming, Rockchip provided a stripped down driver that is available here. This driver only supports memory-to-memory operation and debayering (and no further image processing capabilities), but it is very helpful for hardware bringup. For the inclusion in mainline Linux, however, the plan is for a new rkisp2 driver that is developed from scratch, aiming to support all ISPs of the RK35 generation. To achieve this ambitious milestone, Collabora teamed up with Rockchip and Ideas on Board with this new ISP as a shared goal in mind.

Based on that, the third issue centers on libcamera support for the RK3588 image processing pipeline including suitable image processing algorithms (IPA). The team at Ideas on Board is already active in that direction and has reported initial success with regard to the bring-up of the first image processing blocks. See their blog post for more details.

In the meantime, impatient users can try the libcamera software ISP as a stopgap solution. But for the real deal (full RK3588 ISP kernel driver and libcamera support), stay tuned! And don't miss our RK3588 ISP demo at Embedded Recipes in Nice (May 27-28)!