* [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
0 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Only the JPEG encoder is available for now, although there are patches
for the undocumented VP8 encoder floating around[0].
This has been tested on a rock-5b, resulting in four /dev/video*
encoders. The userspace program I’ve been using to test them is
Onix[1], using the jpeg-encoder example, it will pick one of these four
at random (but displays the one it picked):
% ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
% jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
[0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
[1] https://crates.io/crates/onix
Changes since v1:
- Dropped patches 1 and 4.
- Use the proper compatible form, since this device should be fully
compatible with the VEPU of rk356x.
- Describe where the VEPU121 name comes from, and list other encoders
and decoders present in this SoC.
- Properly test the device tree changes, I previously couldn’t since I
was using a too recent version of python-jsonschema…
Emmanuel Gil Peyrot (2):
media: dt-binding: media: Document rk3588’s VEPU121
arm64: dts: rockchip: Add VEPU121 to rk3588
.../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
2 files changed, 86 insertions(+), 2 deletions(-)
--
2.44.0
^ permalink raw reply [flat|nested] 33+ messages in thread
* [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
0 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Only the JPEG encoder is available for now, although there are patches
for the undocumented VP8 encoder floating around[0].
This has been tested on a rock-5b, resulting in four /dev/video*
encoders. The userspace program I’ve been using to test them is
Onix[1], using the jpeg-encoder example, it will pick one of these four
at random (but displays the one it picked):
% ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
% jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
[0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
[1] https://crates.io/crates/onix
Changes since v1:
- Dropped patches 1 and 4.
- Use the proper compatible form, since this device should be fully
compatible with the VEPU of rk356x.
- Describe where the VEPU121 name comes from, and list other encoders
and decoders present in this SoC.
- Properly test the device tree changes, I previously couldn’t since I
was using a too recent version of python-jsonschema…
Emmanuel Gil Peyrot (2):
media: dt-binding: media: Document rk3588’s VEPU121
arm64: dts: rockchip: Add VEPU121 to rk3588
.../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
2 files changed, 86 insertions(+), 2 deletions(-)
--
2.44.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
0 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Only the JPEG encoder is available for now, although there are patches
for the undocumented VP8 encoder floating around[0].
This has been tested on a rock-5b, resulting in four /dev/video*
encoders. The userspace program I’ve been using to test them is
Onix[1], using the jpeg-encoder example, it will pick one of these four
at random (but displays the one it picked):
% ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
% jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
[0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
[1] https://crates.io/crates/onix
Changes since v1:
- Dropped patches 1 and 4.
- Use the proper compatible form, since this device should be fully
compatible with the VEPU of rk356x.
- Describe where the VEPU121 name comes from, and list other encoders
and decoders present in this SoC.
- Properly test the device tree changes, I previously couldn’t since I
was using a too recent version of python-jsonschema…
Emmanuel Gil Peyrot (2):
media: dt-binding: media: Document rk3588’s VEPU121
arm64: dts: rockchip: Add VEPU121 to rk3588
.../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
2 files changed, 86 insertions(+), 2 deletions(-)
--
2.44.0
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
2024-03-27 13:41 ` Emmanuel Gil Peyrot
(?)
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
-1 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
This encoder-only device is present four times on this SoC, and should
support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
encoding).
According to the TRM[1], there is also the VEPU580 encoder which
supports H.264 and H.265, and various VDPU* decoders, of which only the
VDPU981 is currently supported. This patch describes only the VEPU121.
[1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
---
.../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
index 9d90d8d0565a..4c6cb21da041 100644
--- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
+++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
@@ -15,8 +15,12 @@ description:
properties:
compatible:
- enum:
- - rockchip,rk3568-vepu
+ oneOf:
+ - const: rockchip,rk3568-vepu
+ - items:
+ - enum:
+ - rockchip,rk3588-vepu121
+ - const: rockchip,rk3568-vepu
reg:
maxItems: 1
--
2.44.0
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
0 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
This encoder-only device is present four times on this SoC, and should
support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
encoding).
According to the TRM[1], there is also the VEPU580 encoder which
supports H.264 and H.265, and various VDPU* decoders, of which only the
VDPU981 is currently supported. This patch describes only the VEPU121.
[1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
---
.../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
index 9d90d8d0565a..4c6cb21da041 100644
--- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
+++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
@@ -15,8 +15,12 @@ description:
properties:
compatible:
- enum:
- - rockchip,rk3568-vepu
+ oneOf:
+ - const: rockchip,rk3568-vepu
+ - items:
+ - enum:
+ - rockchip,rk3588-vepu121
+ - const: rockchip,rk3568-vepu
reg:
maxItems: 1
--
2.44.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
0 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
This encoder-only device is present four times on this SoC, and should
support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
encoding).
According to the TRM[1], there is also the VEPU580 encoder which
supports H.264 and H.265, and various VDPU* decoders, of which only the
VDPU981 is currently supported. This patch describes only the VEPU121.
[1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
---
.../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
index 9d90d8d0565a..4c6cb21da041 100644
--- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
+++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
@@ -15,8 +15,12 @@ description:
properties:
compatible:
- enum:
- - rockchip,rk3568-vepu
+ oneOf:
+ - const: rockchip,rk3568-vepu
+ - items:
+ - enum:
+ - rockchip,rk3588-vepu121
+ - const: rockchip,rk3568-vepu
reg:
maxItems: 1
--
2.44.0
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH v2 2/2] arm64: dts: rockchip: Add VEPU121 to rk3588
2024-03-27 13:41 ` Emmanuel Gil Peyrot
(?)
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
-1 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
The TRM (version 1.0 page 385) lists five VEPU121 cores, but only four
interrupts are listed (on page 24), so I’ve only enabled four of them
for now.
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
---
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index 87b83c87bd55..510ed3db9d01 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -2488,6 +2488,86 @@ gpio4: gpio@fec50000 {
};
};
+ jpeg_enc0: video-codec@fdba0000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba0000 0x0 0x800>;
+ interrupts = <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc0_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc0_mmu: iommu@fdba0800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba0800 0x0 0x40>;
+ interrupts = <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc1: video-codec@fdba4000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba4000 0x0 0x800>;
+ interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc1_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc1_mmu: iommu@fdba4800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba4800 0x0 0x40>;
+ interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc2: video-codec@fdba8000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba8000 0x0 0x800>;
+ interrupts = <GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc2_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc2_mmu: iommu@fdba8800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba8800 0x0 0x40>;
+ interrupts = <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc3: video-codec@fdbac000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdbac000 0x0 0x800>;
+ interrupts = <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc3_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc3_mmu: iommu@fdbac800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdbac800 0x0 0x40>;
+ interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
av1d: video-codec@fdc70000 {
compatible = "rockchip,rk3588-av1-vpu";
reg = <0x0 0xfdc70000 0x0 0x800>;
--
2.44.0
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH v2 2/2] arm64: dts: rockchip: Add VEPU121 to rk3588
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
0 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
The TRM (version 1.0 page 385) lists five VEPU121 cores, but only four
interrupts are listed (on page 24), so I’ve only enabled four of them
for now.
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
---
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index 87b83c87bd55..510ed3db9d01 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -2488,6 +2488,86 @@ gpio4: gpio@fec50000 {
};
};
+ jpeg_enc0: video-codec@fdba0000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba0000 0x0 0x800>;
+ interrupts = <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc0_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc0_mmu: iommu@fdba0800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba0800 0x0 0x40>;
+ interrupts = <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc1: video-codec@fdba4000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba4000 0x0 0x800>;
+ interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc1_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc1_mmu: iommu@fdba4800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba4800 0x0 0x40>;
+ interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc2: video-codec@fdba8000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba8000 0x0 0x800>;
+ interrupts = <GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc2_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc2_mmu: iommu@fdba8800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba8800 0x0 0x40>;
+ interrupts = <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc3: video-codec@fdbac000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdbac000 0x0 0x800>;
+ interrupts = <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc3_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc3_mmu: iommu@fdbac800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdbac800 0x0 0x40>;
+ interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
av1d: video-codec@fdc70000 {
compatible = "rockchip,rk3588-av1-vpu";
reg = <0x0 0xfdc70000 0x0 0x800>;
--
2.44.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH v2 2/2] arm64: dts: rockchip: Add VEPU121 to rk3588
@ 2024-03-27 13:41 ` Emmanuel Gil Peyrot
0 siblings, 0 replies; 33+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-03-27 13:41 UTC (permalink / raw
To: linux-kernel
Cc: Emmanuel Gil Peyrot, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
The TRM (version 1.0 page 385) lists five VEPU121 cores, but only four
interrupts are listed (on page 24), so I’ve only enabled four of them
for now.
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
---
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index 87b83c87bd55..510ed3db9d01 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -2488,6 +2488,86 @@ gpio4: gpio@fec50000 {
};
};
+ jpeg_enc0: video-codec@fdba0000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba0000 0x0 0x800>;
+ interrupts = <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc0_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc0_mmu: iommu@fdba0800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba0800 0x0 0x40>;
+ interrupts = <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc1: video-codec@fdba4000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba4000 0x0 0x800>;
+ interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc1_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc1_mmu: iommu@fdba4800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba4800 0x0 0x40>;
+ interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc2: video-codec@fdba8000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdba8000 0x0 0x800>;
+ interrupts = <GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc2_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc2_mmu: iommu@fdba8800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdba8800 0x0 0x40>;
+ interrupts = <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
+ jpeg_enc3: video-codec@fdbac000 {
+ compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
+ reg = <0x0 0xfdbac000 0x0 0x800>;
+ interrupts = <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
+ clock-names = "aclk", "hclk";
+ iommus = <&jpeg_enc3_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ jpeg_enc3_mmu: iommu@fdbac800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdbac800 0x0 0x40>;
+ interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
av1d: video-codec@fdc70000 {
compatible = "rockchip,rk3588-av1-vpu";
reg = <0x0 0xfdc70000 0x0 0x800>;
--
2.44.0
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
2024-03-27 13:41 ` Emmanuel Gil Peyrot
(?)
@ 2024-03-27 17:23 ` Conor Dooley
-1 siblings, 0 replies; 33+ messages in thread
From: Conor Dooley @ 2024-03-27 17:23 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1: Type: text/plain, Size: 1501 bytes --]
On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> This encoder-only device is present four times on this SoC, and should
> support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> encoding).
>
> According to the TRM[1], there is also the VEPU580 encoder which
> supports H.264 and H.265, and various VDPU* decoders, of which only the
> VDPU981 is currently supported. This patch describes only the VEPU121.
>
> [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Thanks,
Conor.
> ---
> .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> index 9d90d8d0565a..4c6cb21da041 100644
> --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> @@ -15,8 +15,12 @@ description:
>
> properties:
> compatible:
> - enum:
> - - rockchip,rk3568-vepu
> + oneOf:
> + - const: rockchip,rk3568-vepu
> + - items:
> + - enum:
> + - rockchip,rk3588-vepu121
> + - const: rockchip,rk3568-vepu
>
> reg:
> maxItems: 1
> --
> 2.44.0
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-03-27 17:23 ` Conor Dooley
0 siblings, 0 replies; 33+ messages in thread
From: Conor Dooley @ 2024-03-27 17:23 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1.1: Type: text/plain, Size: 1501 bytes --]
On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> This encoder-only device is present four times on this SoC, and should
> support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> encoding).
>
> According to the TRM[1], there is also the VEPU580 encoder which
> supports H.264 and H.265, and various VDPU* decoders, of which only the
> VDPU981 is currently supported. This patch describes only the VEPU121.
>
> [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Thanks,
Conor.
> ---
> .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> index 9d90d8d0565a..4c6cb21da041 100644
> --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> @@ -15,8 +15,12 @@ description:
>
> properties:
> compatible:
> - enum:
> - - rockchip,rk3568-vepu
> + oneOf:
> + - const: rockchip,rk3568-vepu
> + - items:
> + - enum:
> + - rockchip,rk3588-vepu121
> + - const: rockchip,rk3568-vepu
>
> reg:
> maxItems: 1
> --
> 2.44.0
>
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-03-27 17:23 ` Conor Dooley
0 siblings, 0 replies; 33+ messages in thread
From: Conor Dooley @ 2024-03-27 17:23 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1.1: Type: text/plain, Size: 1501 bytes --]
On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> This encoder-only device is present four times on this SoC, and should
> support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> encoding).
>
> According to the TRM[1], there is also the VEPU580 encoder which
> supports H.264 and H.265, and various VDPU* decoders, of which only the
> VDPU981 is currently supported. This patch describes only the VEPU121.
>
> [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Thanks,
Conor.
> ---
> .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> index 9d90d8d0565a..4c6cb21da041 100644
> --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> @@ -15,8 +15,12 @@ description:
>
> properties:
> compatible:
> - enum:
> - - rockchip,rk3568-vepu
> + oneOf:
> + - const: rockchip,rk3568-vepu
> + - items:
> + - enum:
> + - rockchip,rk3588-vepu121
> + - const: rockchip,rk3568-vepu
>
> reg:
> maxItems: 1
> --
> 2.44.0
>
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
2024-03-27 13:41 ` Emmanuel Gil Peyrot
(?)
@ 2024-03-27 17:44 ` Sebastian Reichel
-1 siblings, 0 replies; 33+ messages in thread
From: Sebastian Reichel @ 2024-03-27 17:44 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Cristian Ciocaltea, Dragan Simic, Shreeya Patel,
Chris Morgan, Andy Yan, Nicolas Frattaroli, linux-media,
linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1: Type: text/plain, Size: 1518 bytes --]
Hi,
On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> This encoder-only device is present four times on this SoC, and should
> support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> encoding).
>
> According to the TRM[1], there is also the VEPU580 encoder which
> supports H.264 and H.265, and various VDPU* decoders, of which only the
> VDPU981 is currently supported. This patch describes only the VEPU121.
>
> [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
> ---
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
-- Sebastian
> .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> index 9d90d8d0565a..4c6cb21da041 100644
> --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> @@ -15,8 +15,12 @@ description:
>
> properties:
> compatible:
> - enum:
> - - rockchip,rk3568-vepu
> + oneOf:
> + - const: rockchip,rk3568-vepu
> + - items:
> + - enum:
> + - rockchip,rk3588-vepu121
> + - const: rockchip,rk3568-vepu
>
> reg:
> maxItems: 1
> --
> 2.44.0
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-03-27 17:44 ` Sebastian Reichel
0 siblings, 0 replies; 33+ messages in thread
From: Sebastian Reichel @ 2024-03-27 17:44 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Cristian Ciocaltea, Dragan Simic, Shreeya Patel,
Chris Morgan, Andy Yan, Nicolas Frattaroli, linux-media,
linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1.1: Type: text/plain, Size: 1518 bytes --]
Hi,
On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> This encoder-only device is present four times on this SoC, and should
> support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> encoding).
>
> According to the TRM[1], there is also the VEPU580 encoder which
> supports H.264 and H.265, and various VDPU* decoders, of which only the
> VDPU981 is currently supported. This patch describes only the VEPU121.
>
> [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
> ---
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
-- Sebastian
> .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> index 9d90d8d0565a..4c6cb21da041 100644
> --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> @@ -15,8 +15,12 @@ description:
>
> properties:
> compatible:
> - enum:
> - - rockchip,rk3568-vepu
> + oneOf:
> + - const: rockchip,rk3568-vepu
> + - items:
> + - enum:
> + - rockchip,rk3588-vepu121
> + - const: rockchip,rk3568-vepu
>
> reg:
> maxItems: 1
> --
> 2.44.0
>
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-03-27 17:44 ` Sebastian Reichel
0 siblings, 0 replies; 33+ messages in thread
From: Sebastian Reichel @ 2024-03-27 17:44 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Cristian Ciocaltea, Dragan Simic, Shreeya Patel,
Chris Morgan, Andy Yan, Nicolas Frattaroli, linux-media,
linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1.1: Type: text/plain, Size: 1518 bytes --]
Hi,
On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> This encoder-only device is present four times on this SoC, and should
> support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> encoding).
>
> According to the TRM[1], there is also the VEPU580 encoder which
> supports H.264 and H.265, and various VDPU* decoders, of which only the
> VDPU981 is currently supported. This patch describes only the VEPU121.
>
> [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
> ---
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
-- Sebastian
> .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> index 9d90d8d0565a..4c6cb21da041 100644
> --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> @@ -15,8 +15,12 @@ description:
>
> properties:
> compatible:
> - enum:
> - - rockchip,rk3568-vepu
> + oneOf:
> + - const: rockchip,rk3568-vepu
> + - items:
> + - enum:
> + - rockchip,rk3588-vepu121
> + - const: rockchip,rk3568-vepu
>
> reg:
> maxItems: 1
> --
> 2.44.0
>
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 2/2] arm64: dts: rockchip: Add VEPU121 to rk3588
2024-03-27 13:41 ` Emmanuel Gil Peyrot
(?)
@ 2024-03-27 17:45 ` Sebastian Reichel
-1 siblings, 0 replies; 33+ messages in thread
From: Sebastian Reichel @ 2024-03-27 17:45 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Cristian Ciocaltea, Dragan Simic, Shreeya Patel,
Chris Morgan, Andy Yan, Nicolas Frattaroli, linux-media,
linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1: Type: text/plain, Size: 4157 bytes --]
Hi,
On Wed, Mar 27, 2024 at 02:41:12PM +0100, Emmanuel Gil Peyrot wrote:
> The TRM (version 1.0 page 385) lists five VEPU121 cores, but only four
> interrupts are listed (on page 24), so I’ve only enabled four of them
> for now.
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
> ---
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
-- Sebastian
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++++++
> 1 file changed, 80 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> index 87b83c87bd55..510ed3db9d01 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> @@ -2488,6 +2488,86 @@ gpio4: gpio@fec50000 {
> };
> };
>
> + jpeg_enc0: video-codec@fdba0000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba0000 0x0 0x800>;
> + interrupts = <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc0_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc0_mmu: iommu@fdba0800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba0800 0x0 0x40>;
> + interrupts = <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc1: video-codec@fdba4000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba4000 0x0 0x800>;
> + interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc1_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc1_mmu: iommu@fdba4800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba4800 0x0 0x40>;
> + interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc2: video-codec@fdba8000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba8000 0x0 0x800>;
> + interrupts = <GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc2_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc2_mmu: iommu@fdba8800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba8800 0x0 0x40>;
> + interrupts = <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc3: video-codec@fdbac000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdbac000 0x0 0x800>;
> + interrupts = <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc3_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc3_mmu: iommu@fdbac800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdbac800 0x0 0x40>;
> + interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> av1d: video-codec@fdc70000 {
> compatible = "rockchip,rk3588-av1-vpu";
> reg = <0x0 0xfdc70000 0x0 0x800>;
> --
> 2.44.0
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 2/2] arm64: dts: rockchip: Add VEPU121 to rk3588
@ 2024-03-27 17:45 ` Sebastian Reichel
0 siblings, 0 replies; 33+ messages in thread
From: Sebastian Reichel @ 2024-03-27 17:45 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Cristian Ciocaltea, Dragan Simic, Shreeya Patel,
Chris Morgan, Andy Yan, Nicolas Frattaroli, linux-media,
linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1.1: Type: text/plain, Size: 4157 bytes --]
Hi,
On Wed, Mar 27, 2024 at 02:41:12PM +0100, Emmanuel Gil Peyrot wrote:
> The TRM (version 1.0 page 385) lists five VEPU121 cores, but only four
> interrupts are listed (on page 24), so I’ve only enabled four of them
> for now.
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
> ---
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
-- Sebastian
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++++++
> 1 file changed, 80 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> index 87b83c87bd55..510ed3db9d01 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> @@ -2488,6 +2488,86 @@ gpio4: gpio@fec50000 {
> };
> };
>
> + jpeg_enc0: video-codec@fdba0000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba0000 0x0 0x800>;
> + interrupts = <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc0_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc0_mmu: iommu@fdba0800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba0800 0x0 0x40>;
> + interrupts = <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc1: video-codec@fdba4000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba4000 0x0 0x800>;
> + interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc1_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc1_mmu: iommu@fdba4800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba4800 0x0 0x40>;
> + interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc2: video-codec@fdba8000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba8000 0x0 0x800>;
> + interrupts = <GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc2_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc2_mmu: iommu@fdba8800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba8800 0x0 0x40>;
> + interrupts = <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc3: video-codec@fdbac000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdbac000 0x0 0x800>;
> + interrupts = <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc3_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc3_mmu: iommu@fdbac800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdbac800 0x0 0x40>;
> + interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> av1d: video-codec@fdc70000 {
> compatible = "rockchip,rk3588-av1-vpu";
> reg = <0x0 0xfdc70000 0x0 0x800>;
> --
> 2.44.0
>
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 2/2] arm64: dts: rockchip: Add VEPU121 to rk3588
@ 2024-03-27 17:45 ` Sebastian Reichel
0 siblings, 0 replies; 33+ messages in thread
From: Sebastian Reichel @ 2024-03-27 17:45 UTC (permalink / raw
To: Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Cristian Ciocaltea, Dragan Simic, Shreeya Patel,
Chris Morgan, Andy Yan, Nicolas Frattaroli, linux-media,
linux-rockchip, devicetree, linux-arm-kernel, iommu
[-- Attachment #1.1: Type: text/plain, Size: 4157 bytes --]
Hi,
On Wed, Mar 27, 2024 at 02:41:12PM +0100, Emmanuel Gil Peyrot wrote:
> The TRM (version 1.0 page 385) lists five VEPU121 cores, but only four
> interrupts are listed (on page 24), so I’ve only enabled four of them
> for now.
>
> Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
> ---
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
-- Sebastian
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++++++
> 1 file changed, 80 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> index 87b83c87bd55..510ed3db9d01 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> @@ -2488,6 +2488,86 @@ gpio4: gpio@fec50000 {
> };
> };
>
> + jpeg_enc0: video-codec@fdba0000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba0000 0x0 0x800>;
> + interrupts = <GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc0_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc0_mmu: iommu@fdba0800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba0800 0x0 0x40>;
> + interrupts = <GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER0>, <&cru HCLK_JPEG_ENCODER0>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc1: video-codec@fdba4000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba4000 0x0 0x800>;
> + interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc1_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc1_mmu: iommu@fdba4800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba4800 0x0 0x40>;
> + interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER1>, <&cru HCLK_JPEG_ENCODER1>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc2: video-codec@fdba8000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdba8000 0x0 0x800>;
> + interrupts = <GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc2_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc2_mmu: iommu@fdba8800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdba8800 0x0 0x40>;
> + interrupts = <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER2>, <&cru HCLK_JPEG_ENCODER2>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> + jpeg_enc3: video-codec@fdbac000 {
> + compatible = "rockchip,rk3588-vepu121", "rockchip,rk3568-vepu";
> + reg = <0x0 0xfdbac000 0x0 0x800>;
> + interrupts = <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
> + clock-names = "aclk", "hclk";
> + iommus = <&jpeg_enc3_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + jpeg_enc3_mmu: iommu@fdbac800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdbac800 0x0 0x40>;
> + interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH 0>;
> + clocks = <&cru ACLK_JPEG_ENCODER3>, <&cru HCLK_JPEG_ENCODER3>;
> + clock-names = "aclk", "iface";
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> av1d: video-codec@fdc70000 {
> compatible = "rockchip,rk3588-av1-vpu";
> reg = <0x0 0xfdc70000 0x0 0x800>;
> --
> 2.44.0
>
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
2024-03-27 13:41 ` Emmanuel Gil Peyrot
(?)
@ 2024-04-04 17:41 ` Nicolas Dufresne
-1 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-04 17:41 UTC (permalink / raw
To: Emmanuel Gil Peyrot, linux-kernel
Cc: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner,
Joerg Roedel, Will Deacon, Robin Murphy, Sebastian Reichel,
Cristian Ciocaltea, Dragan Simic, Shreeya Patel, Chris Morgan,
Andy Yan, Nicolas Frattaroli, linux-media, linux-rockchip,
devicetree, linux-arm-kernel, iommu
Hi,
Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> Only the JPEG encoder is available for now, although there are patches
> for the undocumented VP8 encoder floating around[0].
[0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
removed. As Rockchip have remove the synthesize register while modifying the H1
IP, it is difficult to verify. Confusingly the H.264 specific registers are
documented in the TRM around VEPU121.
>
> This has been tested on a rock-5b, resulting in four /dev/video*
> encoders. The userspace program I’ve been using to test them is
> Onix[1], using the jpeg-encoder example, it will pick one of these four
> at random (but displays the one it picked):
> % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
I don't like that we exposing each identical cores a separate video nodes. I
think we should aim for 1 device, and then multi-plex and schedule de cores from
inside the Linux kernel.
Not doing this now means we'll never have an optimal hardware usage
distribution. Just consider two userspace software wanting to do jpeg encoding.
If they both take a guess, they may endup using a single core. Where with proper
scheduling in V4L2, the kernel will be able to properly distribute the load. I
insist on this, since if we merge you changes it becomes an ABI and we can't
change it anymore.
I understand that this impose a rework of the mem2mem framework so that we can
run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
which we don't have a driver yet is also multi-core, but you need to use 2 cores
when the resolution is close to 8K.
Nicolas
>
> [0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
> [1] https://crates.io/crates/onix
>
> Changes since v1:
> - Dropped patches 1 and 4.
> - Use the proper compatible form, since this device should be fully
> compatible with the VEPU of rk356x.
> - Describe where the VEPU121 name comes from, and list other encoders
> and decoders present in this SoC.
> - Properly test the device tree changes, I previously couldn’t since I
> was using a too recent version of python-jsonschema…
>
> Emmanuel Gil Peyrot (2):
> media: dt-binding: media: Document rk3588’s VEPU121
> arm64: dts: rockchip: Add VEPU121 to rk3588
>
> .../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
> 2 files changed, 86 insertions(+), 2 deletions(-)
>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-04 17:41 ` Nicolas Dufresne
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-04 17:41 UTC (permalink / raw
To: Emmanuel Gil Peyrot, linux-kernel
Cc: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner,
Joerg Roedel, Will Deacon, Robin Murphy, Sebastian Reichel,
Cristian Ciocaltea, Dragan Simic, Shreeya Patel, Chris Morgan,
Andy Yan, Nicolas Frattaroli, linux-media, linux-rockchip,
devicetree, linux-arm-kernel, iommu
Hi,
Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> Only the JPEG encoder is available for now, although there are patches
> for the undocumented VP8 encoder floating around[0].
[0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
removed. As Rockchip have remove the synthesize register while modifying the H1
IP, it is difficult to verify. Confusingly the H.264 specific registers are
documented in the TRM around VEPU121.
>
> This has been tested on a rock-5b, resulting in four /dev/video*
> encoders. The userspace program I’ve been using to test them is
> Onix[1], using the jpeg-encoder example, it will pick one of these four
> at random (but displays the one it picked):
> % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
I don't like that we exposing each identical cores a separate video nodes. I
think we should aim for 1 device, and then multi-plex and schedule de cores from
inside the Linux kernel.
Not doing this now means we'll never have an optimal hardware usage
distribution. Just consider two userspace software wanting to do jpeg encoding.
If they both take a guess, they may endup using a single core. Where with proper
scheduling in V4L2, the kernel will be able to properly distribute the load. I
insist on this, since if we merge you changes it becomes an ABI and we can't
change it anymore.
I understand that this impose a rework of the mem2mem framework so that we can
run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
which we don't have a driver yet is also multi-core, but you need to use 2 cores
when the resolution is close to 8K.
Nicolas
>
> [0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
> [1] https://crates.io/crates/onix
>
> Changes since v1:
> - Dropped patches 1 and 4.
> - Use the proper compatible form, since this device should be fully
> compatible with the VEPU of rk356x.
> - Describe where the VEPU121 name comes from, and list other encoders
> and decoders present in this SoC.
> - Properly test the device tree changes, I previously couldn’t since I
> was using a too recent version of python-jsonschema…
>
> Emmanuel Gil Peyrot (2):
> media: dt-binding: media: Document rk3588’s VEPU121
> arm64: dts: rockchip: Add VEPU121 to rk3588
>
> .../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
> 2 files changed, 86 insertions(+), 2 deletions(-)
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-04 17:41 ` Nicolas Dufresne
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-04 17:41 UTC (permalink / raw
To: Emmanuel Gil Peyrot, linux-kernel
Cc: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner,
Joerg Roedel, Will Deacon, Robin Murphy, Sebastian Reichel,
Cristian Ciocaltea, Dragan Simic, Shreeya Patel, Chris Morgan,
Andy Yan, Nicolas Frattaroli, linux-media, linux-rockchip,
devicetree, linux-arm-kernel, iommu
Hi,
Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> Only the JPEG encoder is available for now, although there are patches
> for the undocumented VP8 encoder floating around[0].
[0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
removed. As Rockchip have remove the synthesize register while modifying the H1
IP, it is difficult to verify. Confusingly the H.264 specific registers are
documented in the TRM around VEPU121.
>
> This has been tested on a rock-5b, resulting in four /dev/video*
> encoders. The userspace program I’ve been using to test them is
> Onix[1], using the jpeg-encoder example, it will pick one of these four
> at random (but displays the one it picked):
> % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
I don't like that we exposing each identical cores a separate video nodes. I
think we should aim for 1 device, and then multi-plex and schedule de cores from
inside the Linux kernel.
Not doing this now means we'll never have an optimal hardware usage
distribution. Just consider two userspace software wanting to do jpeg encoding.
If they both take a guess, they may endup using a single core. Where with proper
scheduling in V4L2, the kernel will be able to properly distribute the load. I
insist on this, since if we merge you changes it becomes an ABI and we can't
change it anymore.
I understand that this impose a rework of the mem2mem framework so that we can
run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
which we don't have a driver yet is also multi-core, but you need to use 2 cores
when the resolution is close to 8K.
Nicolas
>
> [0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
> [1] https://crates.io/crates/onix
>
> Changes since v1:
> - Dropped patches 1 and 4.
> - Use the proper compatible form, since this device should be fully
> compatible with the VEPU of rk356x.
> - Describe where the VEPU121 name comes from, and list other encoders
> and decoders present in this SoC.
> - Properly test the device tree changes, I previously couldn’t since I
> was using a too recent version of python-jsonschema…
>
> Emmanuel Gil Peyrot (2):
> media: dt-binding: media: Document rk3588’s VEPU121
> arm64: dts: rockchip: Add VEPU121 to rk3588
>
> .../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
> 2 files changed, 86 insertions(+), 2 deletions(-)
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
2024-03-27 17:23 ` Conor Dooley
(?)
@ 2024-04-04 17:47 ` Nicolas Dufresne
-1 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-04 17:47 UTC (permalink / raw
To: Conor Dooley, Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Le mercredi 27 mars 2024 à 17:23 +0000, Conor Dooley a écrit :
> On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> > This encoder-only device is present four times on this SoC, and should
> > support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> > encoding).
> >
> > According to the TRM[1], there is also the VEPU580 encoder which
> > supports H.264 and H.265, and various VDPU* decoders, of which only the
> > VDPU981 is currently supported. This patch describes only the VEPU121.
> >
> > [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
> >
> > Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
>
> Acked-by: Conor Dooley <conor.dooley@microchip.com>
I'd like to prevent this change until we fix the driver. It should not expose 1
video device per core, it should instead do schedule around these cores.
Nicolas
>
> Thanks,
> Conor.
>
> > ---
> > .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> > 1 file changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > index 9d90d8d0565a..4c6cb21da041 100644
> > --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > @@ -15,8 +15,12 @@ description:
> >
> > properties:
> > compatible:
> > - enum:
> > - - rockchip,rk3568-vepu
> > + oneOf:
> > + - const: rockchip,rk3568-vepu
> > + - items:
> > + - enum:
> > + - rockchip,rk3588-vepu121
> > + - const: rockchip,rk3568-vepu
> >
> > reg:
> > maxItems: 1
> > --
> > 2.44.0
> >
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-04-04 17:47 ` Nicolas Dufresne
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-04 17:47 UTC (permalink / raw
To: Conor Dooley, Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Le mercredi 27 mars 2024 à 17:23 +0000, Conor Dooley a écrit :
> On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> > This encoder-only device is present four times on this SoC, and should
> > support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> > encoding).
> >
> > According to the TRM[1], there is also the VEPU580 encoder which
> > supports H.264 and H.265, and various VDPU* decoders, of which only the
> > VDPU981 is currently supported. This patch describes only the VEPU121.
> >
> > [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
> >
> > Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
>
> Acked-by: Conor Dooley <conor.dooley@microchip.com>
I'd like to prevent this change until we fix the driver. It should not expose 1
video device per core, it should instead do schedule around these cores.
Nicolas
>
> Thanks,
> Conor.
>
> > ---
> > .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> > 1 file changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > index 9d90d8d0565a..4c6cb21da041 100644
> > --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > @@ -15,8 +15,12 @@ description:
> >
> > properties:
> > compatible:
> > - enum:
> > - - rockchip,rk3568-vepu
> > + oneOf:
> > + - const: rockchip,rk3568-vepu
> > + - items:
> > + - enum:
> > + - rockchip,rk3588-vepu121
> > + - const: rockchip,rk3568-vepu
> >
> > reg:
> > maxItems: 1
> > --
> > 2.44.0
> >
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121
@ 2024-04-04 17:47 ` Nicolas Dufresne
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-04 17:47 UTC (permalink / raw
To: Conor Dooley, Emmanuel Gil Peyrot
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Le mercredi 27 mars 2024 à 17:23 +0000, Conor Dooley a écrit :
> On Wed, Mar 27, 2024 at 02:41:11PM +0100, Emmanuel Gil Peyrot wrote:
> > This encoder-only device is present four times on this SoC, and should
> > support everything the rk3568 vepu supports (so JPEG, H.264 and VP8
> > encoding).
> >
> > According to the TRM[1], there is also the VEPU580 encoder which
> > supports H.264 and H.265, and various VDPU* decoders, of which only the
> > VDPU981 is currently supported. This patch describes only the VEPU121.
> >
> > [1] https://github.com/FanX-Tek/rk3588-TRM-and-Datasheet
> >
> > Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
>
> Acked-by: Conor Dooley <conor.dooley@microchip.com>
I'd like to prevent this change until we fix the driver. It should not expose 1
video device per core, it should instead do schedule around these cores.
Nicolas
>
> Thanks,
> Conor.
>
> > ---
> > .../devicetree/bindings/media/rockchip,rk3568-vepu.yaml | 8 ++++++--
> > 1 file changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > index 9d90d8d0565a..4c6cb21da041 100644
> > --- a/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > +++ b/Documentation/devicetree/bindings/media/rockchip,rk3568-vepu.yaml
> > @@ -15,8 +15,12 @@ description:
> >
> > properties:
> > compatible:
> > - enum:
> > - - rockchip,rk3568-vepu
> > + oneOf:
> > + - const: rockchip,rk3568-vepu
> > + - items:
> > + - enum:
> > + - rockchip,rk3588-vepu121
> > + - const: rockchip,rk3568-vepu
> >
> > reg:
> > maxItems: 1
> > --
> > 2.44.0
> >
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
2024-04-04 17:41 ` Nicolas Dufresne
(?)
@ 2024-04-05 14:21 ` Link Mauve
-1 siblings, 0 replies; 33+ messages in thread
From: Link Mauve @ 2024-04-05 14:21 UTC (permalink / raw
To: Nicolas Dufresne
Cc: Emmanuel Gil Peyrot, linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> Hi,
Hi,
>
> Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > Only the JPEG encoder is available for now, although there are patches
> > for the undocumented VP8 encoder floating around[0].
>
> [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> removed. As Rockchip have remove the synthesize register while modifying the H1
> IP, it is difficult to verify. Confusingly the H.264 specific registers are
> documented in the TRM around VEPU121.
Ah, the link became, and was indeed ST’s series:
https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
and it’s likely they didn’t remove just VP8 support since the codec
features are pretty close to H.264’s.
>
> >
> > This has been tested on a rock-5b, resulting in four /dev/video*
> > encoders. The userspace program I’ve been using to test them is
> > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > at random (but displays the one it picked):
> > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
>
> I don't like that we exposing each identical cores a separate video nodes. I
> think we should aim for 1 device, and then multi-plex and schedule de cores from
> inside the Linux kernel.
I agree, but this should be handled in the driver not in the device
tree, and it can be done later.
>
> Not doing this now means we'll never have an optimal hardware usage
> distribution. Just consider two userspace software wanting to do jpeg encoding.
> If they both take a guess, they may endup using a single core. Where with proper
> scheduling in V4L2, the kernel will be able to properly distribute the load. I
> insist on this, since if we merge you changes it becomes an ABI and we can't
> change it anymore.
Will it really become ABI just like that? Userspace should always
discover the video nodes and their capabilities and not hardcode e.g. a
specific /dev/videoN file for a specific codec. I would argue that this
series would let userspace do JPEG encoding right away, even if in a
less optimal way than if the driver would round-robin them through a
single video node, but that can always be added in a future version.
>
> I understand that this impose a rework of the mem2mem framework so that we can
> run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> which we don't have a driver yet is also multi-core, but you need to use 2 cores
> when the resolution is close to 8K.
I think the mediatek JPEG driver already supports that, would it be ok
to do it the same way?
>
> Nicolas
>
> >
> > [0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
> > [1] https://crates.io/crates/onix
> >
> > Changes since v1:
> > - Dropped patches 1 and 4.
> > - Use the proper compatible form, since this device should be fully
> > compatible with the VEPU of rk356x.
> > - Describe where the VEPU121 name comes from, and list other encoders
> > and decoders present in this SoC.
> > - Properly test the device tree changes, I previously couldn’t since I
> > was using a too recent version of python-jsonschema…
> >
> > Emmanuel Gil Peyrot (2):
> > media: dt-binding: media: Document rk3588’s VEPU121
> > arm64: dts: rockchip: Add VEPU121 to rk3588
> >
> > .../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
> > arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
> > 2 files changed, 86 insertions(+), 2 deletions(-)
> >
>
--
Link Mauve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-05 14:21 ` Link Mauve
0 siblings, 0 replies; 33+ messages in thread
From: Link Mauve @ 2024-04-05 14:21 UTC (permalink / raw
To: Nicolas Dufresne
Cc: Emmanuel Gil Peyrot, linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> Hi,
Hi,
>
> Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > Only the JPEG encoder is available for now, although there are patches
> > for the undocumented VP8 encoder floating around[0].
>
> [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> removed. As Rockchip have remove the synthesize register while modifying the H1
> IP, it is difficult to verify. Confusingly the H.264 specific registers are
> documented in the TRM around VEPU121.
Ah, the link became, and was indeed ST’s series:
https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
and it’s likely they didn’t remove just VP8 support since the codec
features are pretty close to H.264’s.
>
> >
> > This has been tested on a rock-5b, resulting in four /dev/video*
> > encoders. The userspace program I’ve been using to test them is
> > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > at random (but displays the one it picked):
> > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
>
> I don't like that we exposing each identical cores a separate video nodes. I
> think we should aim for 1 device, and then multi-plex and schedule de cores from
> inside the Linux kernel.
I agree, but this should be handled in the driver not in the device
tree, and it can be done later.
>
> Not doing this now means we'll never have an optimal hardware usage
> distribution. Just consider two userspace software wanting to do jpeg encoding.
> If they both take a guess, they may endup using a single core. Where with proper
> scheduling in V4L2, the kernel will be able to properly distribute the load. I
> insist on this, since if we merge you changes it becomes an ABI and we can't
> change it anymore.
Will it really become ABI just like that? Userspace should always
discover the video nodes and their capabilities and not hardcode e.g. a
specific /dev/videoN file for a specific codec. I would argue that this
series would let userspace do JPEG encoding right away, even if in a
less optimal way than if the driver would round-robin them through a
single video node, but that can always be added in a future version.
>
> I understand that this impose a rework of the mem2mem framework so that we can
> run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> which we don't have a driver yet is also multi-core, but you need to use 2 cores
> when the resolution is close to 8K.
I think the mediatek JPEG driver already supports that, would it be ok
to do it the same way?
>
> Nicolas
>
> >
> > [0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
> > [1] https://crates.io/crates/onix
> >
> > Changes since v1:
> > - Dropped patches 1 and 4.
> > - Use the proper compatible form, since this device should be fully
> > compatible with the VEPU of rk356x.
> > - Describe where the VEPU121 name comes from, and list other encoders
> > and decoders present in this SoC.
> > - Properly test the device tree changes, I previously couldn’t since I
> > was using a too recent version of python-jsonschema…
> >
> > Emmanuel Gil Peyrot (2):
> > media: dt-binding: media: Document rk3588’s VEPU121
> > arm64: dts: rockchip: Add VEPU121 to rk3588
> >
> > .../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
> > arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
> > 2 files changed, 86 insertions(+), 2 deletions(-)
> >
>
--
Link Mauve
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-05 14:21 ` Link Mauve
0 siblings, 0 replies; 33+ messages in thread
From: Link Mauve @ 2024-04-05 14:21 UTC (permalink / raw
To: Nicolas Dufresne
Cc: Emmanuel Gil Peyrot, linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> Hi,
Hi,
>
> Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > Only the JPEG encoder is available for now, although there are patches
> > for the undocumented VP8 encoder floating around[0].
>
> [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> removed. As Rockchip have remove the synthesize register while modifying the H1
> IP, it is difficult to verify. Confusingly the H.264 specific registers are
> documented in the TRM around VEPU121.
Ah, the link became, and was indeed ST’s series:
https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
and it’s likely they didn’t remove just VP8 support since the codec
features are pretty close to H.264’s.
>
> >
> > This has been tested on a rock-5b, resulting in four /dev/video*
> > encoders. The userspace program I’ve been using to test them is
> > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > at random (but displays the one it picked):
> > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
>
> I don't like that we exposing each identical cores a separate video nodes. I
> think we should aim for 1 device, and then multi-plex and schedule de cores from
> inside the Linux kernel.
I agree, but this should be handled in the driver not in the device
tree, and it can be done later.
>
> Not doing this now means we'll never have an optimal hardware usage
> distribution. Just consider two userspace software wanting to do jpeg encoding.
> If they both take a guess, they may endup using a single core. Where with proper
> scheduling in V4L2, the kernel will be able to properly distribute the load. I
> insist on this, since if we merge you changes it becomes an ABI and we can't
> change it anymore.
Will it really become ABI just like that? Userspace should always
discover the video nodes and their capabilities and not hardcode e.g. a
specific /dev/videoN file for a specific codec. I would argue that this
series would let userspace do JPEG encoding right away, even if in a
less optimal way than if the driver would round-robin them through a
single video node, but that can always be added in a future version.
>
> I understand that this impose a rework of the mem2mem framework so that we can
> run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> which we don't have a driver yet is also multi-core, but you need to use 2 cores
> when the resolution is close to 8K.
I think the mediatek JPEG driver already supports that, would it be ok
to do it the same way?
>
> Nicolas
>
> >
> > [0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885
> > [1] https://crates.io/crates/onix
> >
> > Changes since v1:
> > - Dropped patches 1 and 4.
> > - Use the proper compatible form, since this device should be fully
> > compatible with the VEPU of rk356x.
> > - Describe where the VEPU121 name comes from, and list other encoders
> > and decoders present in this SoC.
> > - Properly test the device tree changes, I previously couldn’t since I
> > was using a too recent version of python-jsonschema…
> >
> > Emmanuel Gil Peyrot (2):
> > media: dt-binding: media: Document rk3588’s VEPU121
> > arm64: dts: rockchip: Add VEPU121 to rk3588
> >
> > .../bindings/media/rockchip,rk3568-vepu.yaml | 8 +-
> > arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 80 +++++++++++++++++++
> > 2 files changed, 86 insertions(+), 2 deletions(-)
> >
>
--
Link Mauve
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
2024-04-05 14:21 ` Link Mauve
(?)
@ 2024-04-07 8:08 ` Nicolas Dufresne
-1 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-07 8:08 UTC (permalink / raw
To: Link Mauve
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Le vendredi 05 avril 2024 à 16:21 +0200, Link Mauve a écrit :
> On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> > Hi,
>
> Hi,
>
> >
> > Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > > Only the JPEG encoder is available for now, although there are patches
> > > for the undocumented VP8 encoder floating around[0].
> >
> > [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> > posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> > suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> > removed. As Rockchip have remove the synthesize register while modifying the H1
> > IP, it is difficult to verify. Confusingly the H.264 specific registers are
> > documented in the TRM around VEPU121.
>
> Ah, the link became, and was indeed ST’s series:
> https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
>
> But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
> and it’s likely they didn’t remove just VP8 support since the codec
> features are pretty close to H.264’s.
>
> >
> > >
> > > This has been tested on a rock-5b, resulting in four /dev/video*
> > > encoders. The userspace program I’ve been using to test them is
> > > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > > at random (but displays the one it picked):
> > > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
> >
> > I don't like that we exposing each identical cores a separate video nodes. I
> > think we should aim for 1 device, and then multi-plex and schedule de cores from
> > inside the Linux kernel.
>
> I agree, but this should be handled in the driver not in the device
> tree, and it can be done later.
As the behaviour we want is that these cores becomes a group and get schedule
together, its certainly a good time to slow down and evaluate if that part needs
to be improve in the DT too.
Hantro G1/H1 and VEPU/VDPU121 combos originally shared the same sram region. Its
not clear if any of these cores have this limitation and if this should be
expressed in the DT / driver.
>
> >
> > Not doing this now means we'll never have an optimal hardware usage
> > distribution. Just consider two userspace software wanting to do jpeg encoding.
> > If they both take a guess, they may endup using a single core. Where with proper
> > scheduling in V4L2, the kernel will be able to properly distribute the load. I
> > insist on this, since if we merge you changes it becomes an ABI and we can't
> > change it anymore.
>
> Will it really become ABI just like that? Userspace should always
> discover the video nodes and their capabilities and not hardcode e.g. a
> specific /dev/videoN file for a specific codec. I would argue that this
> series would let userspace do JPEG encoding right away, even if in a
> less optimal way than if the driver would round-robin them through a
> single video node, but that can always be added in a future version.
Might be on the gray side, but there is good chances software written for your
specific board can stop working after te grouping is done.
>
> >
> > I understand that this impose a rework of the mem2mem framework so that we can
> > run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> > which we don't have a driver yet is also multi-core, but you need to use 2 cores
> > when the resolution is close to 8K.
>
> I think the mediatek JPEG driver already supports that, would it be ok
> to do it the same way?
I don't know for JPEG, the MTK vcoder do support cascading cores. This is
different from concurrent cores. In MTK architecture, for some of the codec,
there is LAT (entropy decoder) and CORE (the reconstruction block) that are
split.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-07 8:08 ` Nicolas Dufresne
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-07 8:08 UTC (permalink / raw
To: Link Mauve
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Le vendredi 05 avril 2024 à 16:21 +0200, Link Mauve a écrit :
> On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> > Hi,
>
> Hi,
>
> >
> > Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > > Only the JPEG encoder is available for now, although there are patches
> > > for the undocumented VP8 encoder floating around[0].
> >
> > [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> > posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> > suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> > removed. As Rockchip have remove the synthesize register while modifying the H1
> > IP, it is difficult to verify. Confusingly the H.264 specific registers are
> > documented in the TRM around VEPU121.
>
> Ah, the link became, and was indeed ST’s series:
> https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
>
> But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
> and it’s likely they didn’t remove just VP8 support since the codec
> features are pretty close to H.264’s.
>
> >
> > >
> > > This has been tested on a rock-5b, resulting in four /dev/video*
> > > encoders. The userspace program I’ve been using to test them is
> > > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > > at random (but displays the one it picked):
> > > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
> >
> > I don't like that we exposing each identical cores a separate video nodes. I
> > think we should aim for 1 device, and then multi-plex and schedule de cores from
> > inside the Linux kernel.
>
> I agree, but this should be handled in the driver not in the device
> tree, and it can be done later.
As the behaviour we want is that these cores becomes a group and get schedule
together, its certainly a good time to slow down and evaluate if that part needs
to be improve in the DT too.
Hantro G1/H1 and VEPU/VDPU121 combos originally shared the same sram region. Its
not clear if any of these cores have this limitation and if this should be
expressed in the DT / driver.
>
> >
> > Not doing this now means we'll never have an optimal hardware usage
> > distribution. Just consider two userspace software wanting to do jpeg encoding.
> > If they both take a guess, they may endup using a single core. Where with proper
> > scheduling in V4L2, the kernel will be able to properly distribute the load. I
> > insist on this, since if we merge you changes it becomes an ABI and we can't
> > change it anymore.
>
> Will it really become ABI just like that? Userspace should always
> discover the video nodes and their capabilities and not hardcode e.g. a
> specific /dev/videoN file for a specific codec. I would argue that this
> series would let userspace do JPEG encoding right away, even if in a
> less optimal way than if the driver would round-robin them through a
> single video node, but that can always be added in a future version.
Might be on the gray side, but there is good chances software written for your
specific board can stop working after te grouping is done.
>
> >
> > I understand that this impose a rework of the mem2mem framework so that we can
> > run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> > which we don't have a driver yet is also multi-core, but you need to use 2 cores
> > when the resolution is close to 8K.
>
> I think the mediatek JPEG driver already supports that, would it be ok
> to do it the same way?
I don't know for JPEG, the MTK vcoder do support cascading cores. This is
different from concurrent cores. In MTK architecture, for some of the codec,
there is LAT (entropy decoder) and CORE (the reconstruction block) that are
split.
Nicolas
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-07 8:08 ` Nicolas Dufresne
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Dufresne @ 2024-04-07 8:08 UTC (permalink / raw
To: Link Mauve
Cc: linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
Le vendredi 05 avril 2024 à 16:21 +0200, Link Mauve a écrit :
> On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> > Hi,
>
> Hi,
>
> >
> > Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > > Only the JPEG encoder is available for now, although there are patches
> > > for the undocumented VP8 encoder floating around[0].
> >
> > [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> > posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> > suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> > removed. As Rockchip have remove the synthesize register while modifying the H1
> > IP, it is difficult to verify. Confusingly the H.264 specific registers are
> > documented in the TRM around VEPU121.
>
> Ah, the link became, and was indeed ST’s series:
> https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
>
> But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
> and it’s likely they didn’t remove just VP8 support since the codec
> features are pretty close to H.264’s.
>
> >
> > >
> > > This has been tested on a rock-5b, resulting in four /dev/video*
> > > encoders. The userspace program I’ve been using to test them is
> > > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > > at random (but displays the one it picked):
> > > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
> >
> > I don't like that we exposing each identical cores a separate video nodes. I
> > think we should aim for 1 device, and then multi-plex and schedule de cores from
> > inside the Linux kernel.
>
> I agree, but this should be handled in the driver not in the device
> tree, and it can be done later.
As the behaviour we want is that these cores becomes a group and get schedule
together, its certainly a good time to slow down and evaluate if that part needs
to be improve in the DT too.
Hantro G1/H1 and VEPU/VDPU121 combos originally shared the same sram region. Its
not clear if any of these cores have this limitation and if this should be
expressed in the DT / driver.
>
> >
> > Not doing this now means we'll never have an optimal hardware usage
> > distribution. Just consider two userspace software wanting to do jpeg encoding.
> > If they both take a guess, they may endup using a single core. Where with proper
> > scheduling in V4L2, the kernel will be able to properly distribute the load. I
> > insist on this, since if we merge you changes it becomes an ABI and we can't
> > change it anymore.
>
> Will it really become ABI just like that? Userspace should always
> discover the video nodes and their capabilities and not hardcode e.g. a
> specific /dev/videoN file for a specific codec. I would argue that this
> series would let userspace do JPEG encoding right away, even if in a
> less optimal way than if the driver would round-robin them through a
> single video node, but that can always be added in a future version.
Might be on the gray side, but there is good chances software written for your
specific board can stop working after te grouping is done.
>
> >
> > I understand that this impose a rework of the mem2mem framework so that we can
> > run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> > which we don't have a driver yet is also multi-core, but you need to use 2 cores
> > when the resolution is close to 8K.
>
> I think the mediatek JPEG driver already supports that, would it be ok
> to do it the same way?
I don't know for JPEG, the MTK vcoder do support cascading cores. This is
different from concurrent cores. In MTK architecture, for some of the codec,
there is LAT (entropy decoder) and CORE (the reconstruction block) that are
split.
Nicolas
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
2024-04-07 8:08 ` Nicolas Dufresne
(?)
@ 2024-04-12 15:07 ` Link Mauve
-1 siblings, 0 replies; 33+ messages in thread
From: Link Mauve @ 2024-04-12 15:07 UTC (permalink / raw
To: Nicolas Dufresne
Cc: Link Mauve, linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
On Sun, Apr 07, 2024 at 10:08:58AM +0200, Nicolas Dufresne wrote:
> Le vendredi 05 avril 2024 à 16:21 +0200, Link Mauve a écrit :
> > On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> > > Hi,
> >
> > Hi,
> >
> > >
> > > Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > > > Only the JPEG encoder is available for now, although there are patches
> > > > for the undocumented VP8 encoder floating around[0].
> > >
> > > [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> > > posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> > > suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> > > removed. As Rockchip have remove the synthesize register while modifying the H1
> > > IP, it is difficult to verify. Confusingly the H.264 specific registers are
> > > documented in the TRM around VEPU121.
> >
> > Ah, the link became, and was indeed ST’s series:
> > https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
> >
> > But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
> > and it’s likely they didn’t remove just VP8 support since the codec
> > features are pretty close to H.264’s.
> >
> > >
> > > >
> > > > This has been tested on a rock-5b, resulting in four /dev/video*
> > > > encoders. The userspace program I’ve been using to test them is
> > > > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > > > at random (but displays the one it picked):
> > > > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > > > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
> > >
> > > I don't like that we exposing each identical cores a separate video nodes. I
> > > think we should aim for 1 device, and then multi-plex and schedule de cores from
> > > inside the Linux kernel.
> >
> > I agree, but this should be handled in the driver not in the device
> > tree, and it can be done later.
>
> As the behaviour we want is that these cores becomes a group and get schedule
> together, its certainly a good time to slow down and evaluate if that part needs
> to be improve in the DT too.
>
> Hantro G1/H1 and VEPU/VDPU121 combos originally shared the same sram region. Its
> not clear if any of these cores have this limitation and if this should be
> expressed in the DT / driver.
The TRM on page 369 mentions that:
> Please note that VDPU121 and VDPU381 and VDPU720 and AV1 and VEPU121 and VEPU580
> is different IP cores, so these processing core can be work together.
I understand that as them not sharing any memory and being able to work
independently.
>
> >
> > >
> > > Not doing this now means we'll never have an optimal hardware usage
> > > distribution. Just consider two userspace software wanting to do jpeg encoding.
> > > If they both take a guess, they may endup using a single core. Where with proper
> > > scheduling in V4L2, the kernel will be able to properly distribute the load. I
> > > insist on this, since if we merge you changes it becomes an ABI and we can't
> > > change it anymore.
> >
> > Will it really become ABI just like that? Userspace should always
> > discover the video nodes and their capabilities and not hardcode e.g. a
> > specific /dev/videoN file for a specific codec. I would argue that this
> > series would let userspace do JPEG encoding right away, even if in a
> > less optimal way than if the driver would round-robin them through a
> > single video node, but that can always be added in a future version.
>
> Might be on the gray side, but there is good chances software written for your
> specific board can stop working after te grouping is done.
I will send a new series shortly which enables only one of these four
cores, the functionality should be the same, it will just expose only
one video node which can get a four times throughput upgrade later once
we implement multi-core support in the driver.
>
> >
> > >
> > > I understand that this impose a rework of the mem2mem framework so that we can
> > > run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> > > which we don't have a driver yet is also multi-core, but you need to use 2 cores
> > > when the resolution is close to 8K.
> >
> > I think the mediatek JPEG driver already supports that, would it be ok
> > to do it the same way?
>
> I don't know for JPEG, the MTK vcoder do support cascading cores. This is
> different from concurrent cores. In MTK architecture, for some of the codec,
> there is LAT (entropy decoder) and CORE (the reconstruction block) that are
> split.
Ah, that’s different then, I only had a cursory look at them back when I
implemented the JPEG decoder for sunxi.
>
> Nicolas
--
Link Mauve
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-12 15:07 ` Link Mauve
0 siblings, 0 replies; 33+ messages in thread
From: Link Mauve @ 2024-04-12 15:07 UTC (permalink / raw
To: Nicolas Dufresne
Cc: Link Mauve, linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
On Sun, Apr 07, 2024 at 10:08:58AM +0200, Nicolas Dufresne wrote:
> Le vendredi 05 avril 2024 à 16:21 +0200, Link Mauve a écrit :
> > On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> > > Hi,
> >
> > Hi,
> >
> > >
> > > Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > > > Only the JPEG encoder is available for now, although there are patches
> > > > for the undocumented VP8 encoder floating around[0].
> > >
> > > [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> > > posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> > > suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> > > removed. As Rockchip have remove the synthesize register while modifying the H1
> > > IP, it is difficult to verify. Confusingly the H.264 specific registers are
> > > documented in the TRM around VEPU121.
> >
> > Ah, the link became, and was indeed ST’s series:
> > https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
> >
> > But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
> > and it’s likely they didn’t remove just VP8 support since the codec
> > features are pretty close to H.264’s.
> >
> > >
> > > >
> > > > This has been tested on a rock-5b, resulting in four /dev/video*
> > > > encoders. The userspace program I’ve been using to test them is
> > > > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > > > at random (but displays the one it picked):
> > > > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > > > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
> > >
> > > I don't like that we exposing each identical cores a separate video nodes. I
> > > think we should aim for 1 device, and then multi-plex and schedule de cores from
> > > inside the Linux kernel.
> >
> > I agree, but this should be handled in the driver not in the device
> > tree, and it can be done later.
>
> As the behaviour we want is that these cores becomes a group and get schedule
> together, its certainly a good time to slow down and evaluate if that part needs
> to be improve in the DT too.
>
> Hantro G1/H1 and VEPU/VDPU121 combos originally shared the same sram region. Its
> not clear if any of these cores have this limitation and if this should be
> expressed in the DT / driver.
The TRM on page 369 mentions that:
> Please note that VDPU121 and VDPU381 and VDPU720 and AV1 and VEPU121 and VEPU580
> is different IP cores, so these processing core can be work together.
I understand that as them not sharing any memory and being able to work
independently.
>
> >
> > >
> > > Not doing this now means we'll never have an optimal hardware usage
> > > distribution. Just consider two userspace software wanting to do jpeg encoding.
> > > If they both take a guess, they may endup using a single core. Where with proper
> > > scheduling in V4L2, the kernel will be able to properly distribute the load. I
> > > insist on this, since if we merge you changes it becomes an ABI and we can't
> > > change it anymore.
> >
> > Will it really become ABI just like that? Userspace should always
> > discover the video nodes and their capabilities and not hardcode e.g. a
> > specific /dev/videoN file for a specific codec. I would argue that this
> > series would let userspace do JPEG encoding right away, even if in a
> > less optimal way than if the driver would round-robin them through a
> > single video node, but that can always be added in a future version.
>
> Might be on the gray side, but there is good chances software written for your
> specific board can stop working after te grouping is done.
I will send a new series shortly which enables only one of these four
cores, the functionality should be the same, it will just expose only
one video node which can get a four times throughput upgrade later once
we implement multi-core support in the driver.
>
> >
> > >
> > > I understand that this impose a rework of the mem2mem framework so that we can
> > > run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> > > which we don't have a driver yet is also multi-core, but you need to use 2 cores
> > > when the resolution is close to 8K.
> >
> > I think the mediatek JPEG driver already supports that, would it be ok
> > to do it the same way?
>
> I don't know for JPEG, the MTK vcoder do support cascading cores. This is
> different from concurrent cores. In MTK architecture, for some of the codec,
> there is LAT (entropy decoder) and CORE (the reconstruction block) that are
> split.
Ah, that’s different then, I only had a cursory look at them back when I
implemented the JPEG decoder for sunxi.
>
> Nicolas
--
Link Mauve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH v2 0/2] Enable JPEG encoding on rk3588
@ 2024-04-12 15:07 ` Link Mauve
0 siblings, 0 replies; 33+ messages in thread
From: Link Mauve @ 2024-04-12 15:07 UTC (permalink / raw
To: Nicolas Dufresne
Cc: Link Mauve, linux-kernel, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Joerg Roedel, Will Deacon,
Robin Murphy, Sebastian Reichel, Cristian Ciocaltea, Dragan Simic,
Shreeya Patel, Chris Morgan, Andy Yan, Nicolas Frattaroli,
linux-media, linux-rockchip, devicetree, linux-arm-kernel, iommu
On Sun, Apr 07, 2024 at 10:08:58AM +0200, Nicolas Dufresne wrote:
> Le vendredi 05 avril 2024 à 16:21 +0200, Link Mauve a écrit :
> > On Thu, Apr 04, 2024 at 01:41:15PM -0400, Nicolas Dufresne wrote:
> > > Hi,
> >
> > Hi,
> >
> > >
> > > Le mercredi 27 mars 2024 à 14:41 +0100, Emmanuel Gil Peyrot a écrit :
> > > > Only the JPEG encoder is available for now, although there are patches
> > > > for the undocumented VP8 encoder floating around[0].
> > >
> > > [0] seems like a broken link. The VP8 encoder RFC is for RK3399 (and Hantro H1
> > > posted by ST more recently). The TRM says "VEPU121(JPEG encoder only)", which
> > > suggest that the H.264 and VP8 encoders usually found on the VEPU121 are
> > > removed. As Rockchip have remove the synthesize register while modifying the H1
> > > IP, it is difficult to verify. Confusingly the H.264 specific registers are
> > > documented in the TRM around VEPU121.
> >
> > Ah, the link became, and was indeed ST’s series:
> > https://patchwork.kernel.org/project/linux-rockchip/list/?series=789885&archive=both
> >
> > But the TRM part 1 says the VEPU121 supports H.264 encoding (page 367),
> > and it’s likely they didn’t remove just VP8 support since the codec
> > features are pretty close to H.264’s.
> >
> > >
> > > >
> > > > This has been tested on a rock-5b, resulting in four /dev/video*
> > > > encoders. The userspace program I’ve been using to test them is
> > > > Onix[1], using the jpeg-encoder example, it will pick one of these four
> > > > at random (but displays the one it picked):
> > > > % ffmpeg -i <input image> -pix_fmt yuvj420p temp.yuv
> > > > % jpeg-encoder temp.yuv <width> <height> NV12 <quality> output.jpeg
> > >
> > > I don't like that we exposing each identical cores a separate video nodes. I
> > > think we should aim for 1 device, and then multi-plex and schedule de cores from
> > > inside the Linux kernel.
> >
> > I agree, but this should be handled in the driver not in the device
> > tree, and it can be done later.
>
> As the behaviour we want is that these cores becomes a group and get schedule
> together, its certainly a good time to slow down and evaluate if that part needs
> to be improve in the DT too.
>
> Hantro G1/H1 and VEPU/VDPU121 combos originally shared the same sram region. Its
> not clear if any of these cores have this limitation and if this should be
> expressed in the DT / driver.
The TRM on page 369 mentions that:
> Please note that VDPU121 and VDPU381 and VDPU720 and AV1 and VEPU121 and VEPU580
> is different IP cores, so these processing core can be work together.
I understand that as them not sharing any memory and being able to work
independently.
>
> >
> > >
> > > Not doing this now means we'll never have an optimal hardware usage
> > > distribution. Just consider two userspace software wanting to do jpeg encoding.
> > > If they both take a guess, they may endup using a single core. Where with proper
> > > scheduling in V4L2, the kernel will be able to properly distribute the load. I
> > > insist on this, since if we merge you changes it becomes an ABI and we can't
> > > change it anymore.
> >
> > Will it really become ABI just like that? Userspace should always
> > discover the video nodes and their capabilities and not hardcode e.g. a
> > specific /dev/videoN file for a specific codec. I would argue that this
> > series would let userspace do JPEG encoding right away, even if in a
> > less optimal way than if the driver would round-robin them through a
> > single video node, but that can always be added in a future version.
>
> Might be on the gray side, but there is good chances software written for your
> specific board can stop working after te grouping is done.
I will send a new series shortly which enables only one of these four
cores, the functionality should be the same, it will just expose only
one video node which can get a four times throughput upgrade later once
we implement multi-core support in the driver.
>
> >
> > >
> > > I understand that this impose a rework of the mem2mem framework so that we can
> > > run multiple jobs, but this will be needed anyway on RK3588, since the rkvdec2,
> > > which we don't have a driver yet is also multi-core, but you need to use 2 cores
> > > when the resolution is close to 8K.
> >
> > I think the mediatek JPEG driver already supports that, would it be ok
> > to do it the same way?
>
> I don't know for JPEG, the MTK vcoder do support cascading cores. This is
> different from concurrent cores. In MTK architecture, for some of the codec,
> there is LAT (entropy decoder) and CORE (the reconstruction block) that are
> split.
Ah, that’s different then, I only had a cursory look at them back when I
implemented the JPEG decoder for sunxi.
>
> Nicolas
--
Link Mauve
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2024-04-12 15:07 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-03-27 13:41 [PATCH v2 0/2] Enable JPEG encoding on rk3588 Emmanuel Gil Peyrot
2024-03-27 13:41 ` Emmanuel Gil Peyrot
2024-03-27 13:41 ` Emmanuel Gil Peyrot
2024-03-27 13:41 ` [PATCH v2 1/2] media: dt-binding: media: Document rk3588’s VEPU121 Emmanuel Gil Peyrot
2024-03-27 13:41 ` Emmanuel Gil Peyrot
2024-03-27 13:41 ` Emmanuel Gil Peyrot
2024-03-27 17:23 ` Conor Dooley
2024-03-27 17:23 ` Conor Dooley
2024-03-27 17:23 ` Conor Dooley
2024-04-04 17:47 ` Nicolas Dufresne
2024-04-04 17:47 ` Nicolas Dufresne
2024-04-04 17:47 ` Nicolas Dufresne
2024-03-27 17:44 ` Sebastian Reichel
2024-03-27 17:44 ` Sebastian Reichel
2024-03-27 17:44 ` Sebastian Reichel
2024-03-27 13:41 ` [PATCH v2 2/2] arm64: dts: rockchip: Add VEPU121 to rk3588 Emmanuel Gil Peyrot
2024-03-27 13:41 ` Emmanuel Gil Peyrot
2024-03-27 13:41 ` Emmanuel Gil Peyrot
2024-03-27 17:45 ` Sebastian Reichel
2024-03-27 17:45 ` Sebastian Reichel
2024-03-27 17:45 ` Sebastian Reichel
2024-04-04 17:41 ` [PATCH v2 0/2] Enable JPEG encoding on rk3588 Nicolas Dufresne
2024-04-04 17:41 ` Nicolas Dufresne
2024-04-04 17:41 ` Nicolas Dufresne
2024-04-05 14:21 ` Link Mauve
2024-04-05 14:21 ` Link Mauve
2024-04-05 14:21 ` Link Mauve
2024-04-07 8:08 ` Nicolas Dufresne
2024-04-07 8:08 ` Nicolas Dufresne
2024-04-07 8:08 ` Nicolas Dufresne
2024-04-12 15:07 ` Link Mauve
2024-04-12 15:07 ` Link Mauve
2024-04-12 15:07 ` Link Mauve
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.