* [PATCH net-next v3 0/8] Netlink protocol specs
@ 2023-01-19 0:36 Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs) Jakub Kicinski
` (8 more replies)
0 siblings, 9 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
I think the Netlink proto specs are far along enough to merge.
Filling in all attribute types and quirks will be an ongoing
effort but we have enough to cover FOU so it's somewhat complete.
I fully intend to continue polishing the code but at the same
time I'd like to start helping others base their work on the
specs (e.g. DPLL) and need to start working on some new families
myself.
That's the progress / motivation for merging. The RFC [1] has more
of a high level blurb, plus I created a lot of documentation, I'm
not going to repeat it here. There was also the talk at LPC [2].
[1] https://lore.kernel.org/all/20220811022304.583300-1-kuba@kernel.org/
[2] https://youtu.be/9QkXIQXkaQk?t=2562
v2: https://lore.kernel.org/all/20220930023418.1346263-1-kuba@kernel.org/
Jakub Kicinski (8):
docs: add more netlink docs (incl. spec docs)
netlink: add schemas for YAML specs
net: add basic C code generators for Netlink
netlink: add a proto specification for FOU
net: fou: regenerate the uAPI from the spec
net: fou: rename the source for linking
net: fou: use policy and operation tables generated from the spec
tools: ynl: add a completely generic client
Documentation/core-api/index.rst | 1 +
Documentation/core-api/netlink.rst | 99 +
Documentation/netlink/genetlink-c.yaml | 318 +++
Documentation/netlink/genetlink-legacy.yaml | 346 +++
Documentation/netlink/genetlink.yaml | 284 ++
Documentation/netlink/specs/fou.yaml | 128 +
.../userspace-api/netlink/c-code-gen.rst | 107 +
.../netlink/genetlink-legacy.rst | 96 +
Documentation/userspace-api/netlink/index.rst | 5 +
Documentation/userspace-api/netlink/specs.rst | 422 +++
MAINTAINERS | 3 +
include/uapi/linux/fou.h | 54 +-
net/ipv4/Makefile | 1 +
net/ipv4/fou-nl.c | 48 +
net/ipv4/fou-nl.h | 25 +
net/ipv4/{fou.c => fou_core.c} | 47 +-
tools/net/ynl/samples/cli.py | 47 +
tools/net/ynl/samples/ynl.py | 534 ++++
tools/net/ynl/ynl-gen-c.py | 2376 +++++++++++++++++
tools/net/ynl/ynl-regen.sh | 30 +
20 files changed, 4903 insertions(+), 68 deletions(-)
create mode 100644 Documentation/core-api/netlink.rst
create mode 100644 Documentation/netlink/genetlink-c.yaml
create mode 100644 Documentation/netlink/genetlink-legacy.yaml
create mode 100644 Documentation/netlink/genetlink.yaml
create mode 100644 Documentation/netlink/specs/fou.yaml
create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst
create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst
create mode 100644 Documentation/userspace-api/netlink/specs.rst
create mode 100644 net/ipv4/fou-nl.c
create mode 100644 net/ipv4/fou-nl.h
rename net/ipv4/{fou.c => fou_core.c} (94%)
create mode 100755 tools/net/ynl/samples/cli.py
create mode 100644 tools/net/ynl/samples/ynl.py
create mode 100755 tools/net/ynl/ynl-gen-c.py
create mode 100755 tools/net/ynl/ynl-regen.sh
--
2.39.0
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-19 15:48 ` Vladimir Oltean
2023-01-19 20:29 ` Johannes Berg
2023-01-19 0:36 ` [PATCH net-next v3 2/8] netlink: add schemas for YAML specs Jakub Kicinski
` (7 subsequent siblings)
8 siblings, 2 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski, Bagas Sanjaya
Add documentation about the upcoming Netlink protocol specs.
Reviewed-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
--
v3:
- spelling from Guillaume
v2:
- plenty -> plenty of
- type: unspec -> type: unused
- ECHO edits from Guillaume
---
Documentation/core-api/index.rst | 1 +
Documentation/core-api/netlink.rst | 99 ++++
.../userspace-api/netlink/c-code-gen.rst | 107 +++++
.../netlink/genetlink-legacy.rst | 96 ++++
Documentation/userspace-api/netlink/index.rst | 5 +
Documentation/userspace-api/netlink/specs.rst | 422 ++++++++++++++++++
MAINTAINERS | 2 +
7 files changed, 732 insertions(+)
create mode 100644 Documentation/core-api/netlink.rst
create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst
create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst
create mode 100644 Documentation/userspace-api/netlink/specs.rst
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index 77eb775b8b42..7a3a08d81f11 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -127,6 +127,7 @@ Documents that don't fit elsewhere or which have yet to be categorized.
:maxdepth: 1
librs
+ netlink
.. only:: subproject and html
diff --git a/Documentation/core-api/netlink.rst b/Documentation/core-api/netlink.rst
new file mode 100644
index 000000000000..7b98dd48a6af
--- /dev/null
+++ b/Documentation/core-api/netlink.rst
@@ -0,0 +1,99 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+
+.. _kernel_netlink:
+
+===================================
+Netlink notes for kernel developers
+===================================
+
+General guidance
+================
+
+Attribute enums
+---------------
+
+Older families often define "null" attributes and commands with value
+of ``0`` and named ``unspec``. This is supported (``type: unused``)
+but should be avoided in new families. The ``unspec`` enum values are
+not used in practice, so just set the value of the first attribute to ``1``.
+
+Message enums
+-------------
+
+Use the same command IDs for requests and replies. This makes it easier
+to match them up, and we have plenty of ID space.
+
+Use separate command IDs for notifications. This makes it easier to
+sort the notifications from replies (and present them to the user
+application via a different API than replies).
+
+Answer requests
+---------------
+
+Older families do not reply to all of the commands, especially NEW / ADD
+commands. User only gets information whether the operation succeeded or
+not via the ACK. Try to find useful data to return. Once the command is
+added whether it replies with a full message or only an ACK is uAPI and
+cannot be changed. It's better to err on the side of replying.
+
+Specifically NEW and ADD commands should reply with information identifying
+the created object such as the allocated object's ID (without having to
+resort to using ``NLM_F_ECHO``).
+
+NLM_F_ECHO
+----------
+
+Make sure to pass the request info to genl_notify() to allow ``NLM_F_ECHO``
+to take effect. This is useful for programs that need precise feedback
+from the kernel (for example for logging purposes).
+
+Support dump consistency
+------------------------
+
+If iterating over objects during dump may skip over objects or repeat
+them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.
+
+Netlink specification
+=====================
+
+Documentation of the Netlink specification parts which are only relevant
+to the kernel space.
+
+Globals
+-------
+
+kernel-policy
+~~~~~~~~~~~~~
+
+Defines if the kernel validation policy is per operation (``per-op``)
+or for the entire family (``global``). New families should use ``per-op``
+(default) to be able to narrow down the attributes accepted by a specific
+command.
+
+checks
+------
+
+Documentation for the ``checks`` sub-sections of attribute specs.
+
+unterminated-ok
+~~~~~~~~~~~~~~~
+
+Accept strings without the null-termination (for legacy families only).
+Switches from the ``NLA_NUL_STRING`` to ``NLA_STRING`` policy type.
+
+max-len
+~~~~~~~
+
+Defines max length for a binary or string attribute (corresponding
+to the ``len`` member of struct nla_policy). For string attributes terminating
+null character is not counted towards ``max-len``.
+
+The field may either be a literal integer value or a name of a defined
+constant. String types may reduce the constant by one
+(i.e. specify ``max-len: CONST - 1``) to reserve space for the terminating
+character so implementations should recognize such pattern.
+
+min-len
+~~~~~~~
+
+Similar to ``max-len`` but defines minimum length.
diff --git a/Documentation/userspace-api/netlink/c-code-gen.rst b/Documentation/userspace-api/netlink/c-code-gen.rst
new file mode 100644
index 000000000000..89de42c13350
--- /dev/null
+++ b/Documentation/userspace-api/netlink/c-code-gen.rst
@@ -0,0 +1,107 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+
+==============================
+Netlink spec C code generation
+==============================
+
+This document describes how Netlink specifications are used to render
+C code (uAPI, policies etc.). It also defines the additional properties
+allowed in older families by the ``genetlink-c`` protocol level,
+to control the naming.
+
+For brevity this document refers to ``name`` properties of various
+objects by the object type. For example ``$attr`` is the value
+of ``name`` in an attribute, and ``$family`` is the name of the
+family (the global ``name`` property).
+
+The upper case is used to denote literal values, e.g. ``$family-CMD``
+means the concatenation of ``$family``, a dash character, and the literal
+``CMD``.
+
+The names of ``#defines`` and enum values are always converted to upper case,
+and with dashes (``-``) replaced by underscores (``_``).
+
+If the constructed name is a C keyword, an extra underscore is
+appended (``do`` -> ``do_``).
+
+Globals
+=======
+
+``c-family-name`` controls the name of the ``#define`` for the family
+name, default is ``$family-FAMILY-NAME``.
+
+``c-version-name`` controls the name of the ``#define`` for the version
+of the family, default is ``$family-FAMILY-VERSION``.
+
+``max-by-define`` selects if max values for enums are defined as a
+``#define`` rather than inside the enum.
+
+Definitions
+===========
+
+Constants
+---------
+
+Every constant is rendered as a ``#define``.
+The name of the constant is ``$family-$constant`` and the value
+is rendered as a string or integer according to its type in the spec.
+
+Enums and flags
+---------------
+
+Enums are named ``$family-$enum``. The full name can be set directly
+or suppressed by specifying the ``enum-name`` property.
+Default entry name is ``$family-$enum-$entry``.
+If ``name-prefix`` is specified it replaces the ``$family-$enum``
+portion of the entry name.
+
+Boolean ``render-max`` controls creation of the max values
+(which are enabled by default for attribute enums).
+
+Attributes
+==========
+
+Each attribute set (excluding fractional sets) is rendered as an enum.
+
+Attribute enums are traditionally unnamed in netlink headers.
+If naming is desired ``enum-name`` can be used to specify the name.
+
+The default attribute name prefix is ``$family-A`` if the name of the set
+is the same as the name of the family and ``$family-A-$set`` if the names
+differ. The prefix can be overridden by the ``name-prefix`` property of a set.
+The rest of the section will refer to the prefix as ``$pfx``.
+
+Attributes are named ``$pfx-$attribute``.
+
+Attribute enums end with two special values ``__$pfx-MAX`` and ``$pfx-MAX``
+which are used for sizing attribute tables.
+These two names can be specified directly with the ``attr-cnt-name``
+and ``attr-max-name`` properties respectively.
+
+If ``max-by-define`` is set to ``true`` at the global level ``attr-max-name``
+will be specified as a ``#define`` rather than an enum value.
+
+Operations
+==========
+
+Operations are named ``$family-CMD-$operation``.
+If ``name-prefix`` is specified it replaces the ``$family-CMD``
+portion of the name.
+
+Similarly to attribute enums operation enums end with special count and max
+attributes. For operations those attributes can be renamed with
+``cmd-cnt-name`` and ``cmd-max-name``. Max will be a define if ``max-by-define``
+is ``true``.
+
+Multicast groups
+================
+
+Each multicast group gets a define rendered into the kernel uAPI header.
+The name of the define is ``$family-MCGRP-$group``, and can be overwritten
+with the ``c-define-name`` property.
+
+Code generation
+===============
+
+uAPI header is assumed to come from ``<linux/$family.h>`` in the default header
+search path. It can be changed using the ``uapi-header`` global property.
diff --git a/Documentation/userspace-api/netlink/genetlink-legacy.rst b/Documentation/userspace-api/netlink/genetlink-legacy.rst
new file mode 100644
index 000000000000..65cbbffee0bf
--- /dev/null
+++ b/Documentation/userspace-api/netlink/genetlink-legacy.rst
@@ -0,0 +1,96 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+
+=================================================================
+Netlink specification support for legacy Generic Netlink families
+=================================================================
+
+This document describes the many additional quirks and properties
+required to describe older Generic Netlink families which form
+the ``genetlink-legacy`` protocol level.
+
+The spec is a work in progress, some of the quirks are just documented
+for future reference.
+
+Specification (defined)
+=======================
+
+Attribute type nests
+--------------------
+
+New Netlink families should use ``multi-attr`` to define arrays.
+Older families (e.g. ``genetlink`` control family) attempted to
+define array types reusing attribute type to carry information.
+
+For reference the ``multi-attr`` array may look like this::
+
+ [ARRAY-ATTR]
+ [INDEX (optionally)]
+ [MEMBER1]
+ [MEMBER2]
+ [SOME-OTHER-ATTR]
+ [ARRAY-ATTR]
+ [INDEX (optionally)]
+ [MEMBER1]
+ [MEMBER2]
+
+where ``ARRAY-ATTR`` is the array entry type.
+
+array-nest
+~~~~~~~~~~
+
+``array-nest`` creates the following structure::
+
+ [SOME-OTHER-ATTR]
+ [ARRAY-ATTR]
+ [ENTRY]
+ [MEMBER1]
+ [MEMBER2]
+ [ENTRY]
+ [MEMBER1]
+ [MEMBER2]
+
+It wraps the entire array in an extra attribute (hence limiting its size
+to 64kB). The ``ENTRY`` nests are special and have the index of the entry
+as their type instead of normal attribute type.
+
+type-value
+~~~~~~~~~~
+
+``type-value`` is a construct which uses attribute types to carry
+information about a single object (often used when array is dumped
+entry-by-entry).
+
+``type-value`` can have multiple levels of nesting, for example
+genetlink's policy dumps create the following structures::
+
+ [POLICY-IDX]
+ [ATTR-IDX]
+ [POLICY-INFO-ATTR1]
+ [POLICY-INFO-ATTR2]
+
+Where the first level of nest has the policy index as it's attribute
+type, it contains a single nest which has the attribute index as its
+type. Inside the attr-index nest are the policy attributes. Modern
+Netlink families should have instead defined this as a flat structure,
+the nesting serves no good purpose here.
+
+Other quirks (todo)
+===================
+
+Structures
+----------
+
+Legacy families can define C structures both to be used as the contents
+of an attribute and as a fixed message header. The plan is to define
+the structs in ``definitions`` and link the appropriate attrs.
+
+Multi-message DO
+----------------
+
+New Netlink families should never respond to a DO operation with multiple
+replies, with ``NLM_F_MULTI`` set. Use a filtered dump instead.
+
+At the spec level we can define a ``dumps`` property for the ``do``,
+perhaps with values of ``combine`` and ``multi-object`` depending
+on how the parsing should be implemented (parse into a single reply
+vs list of objects i.e. pretty much a dump).
diff --git a/Documentation/userspace-api/netlink/index.rst b/Documentation/userspace-api/netlink/index.rst
index b0c21538d97d..be250110c8f6 100644
--- a/Documentation/userspace-api/netlink/index.rst
+++ b/Documentation/userspace-api/netlink/index.rst
@@ -10,3 +10,8 @@ Netlink documentation for users.
:maxdepth: 2
intro
+ specs
+ c-code-gen
+ genetlink-legacy
+
+See also :ref:`Documentation/core-api/netlink.rst <kernel_netlink>`.
diff --git a/Documentation/userspace-api/netlink/specs.rst b/Documentation/userspace-api/netlink/specs.rst
new file mode 100644
index 000000000000..8394d74fc63a
--- /dev/null
+++ b/Documentation/userspace-api/netlink/specs.rst
@@ -0,0 +1,422 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+
+=========================================
+Netlink protocol specifications (in YAML)
+=========================================
+
+Netlink protocol specifications are complete, machine readable descriptions of
+Netlink protocols written in YAML. The goal of the specifications is to allow
+separating Netlink parsing from user space logic and minimize the amount of
+hand written Netlink code for each new family, command, attribute.
+Netlink specs should be complete and not depend on any other spec
+or C header file, making it easy to use in languages which can't include
+kernel headers directly.
+
+Internally kernel uses the YAML specs to generate:
+
+ - the C uAPI header
+ - documentation of the protocol as a ReST file
+ - policy tables for input attribute validation
+ - operation tables
+
+YAML specifications can be found under ``Documentation/netlink/specs/``
+
+Compatibility levels
+====================
+
+There are four schema levels for Netlink specs, from the simplest used
+by new families to the most complex covering all the quirks of the old ones.
+Each next level inherits the attributes of the previous level, meaning that
+user capable of parsing more complex ``genetlink`` schemas is also compatible
+with simpler ones. The levels are:
+
+ - ``genetlink`` - most streamlined, should be used by all new families
+ - ``genetlink-c`` - superset of ``genetlink`` with extra attributes allowing
+ customization of define and enum type and value names; this schema should
+ be equivalent to ``genetlink`` for all implementations which don't interact
+ directly with C uAPI headers
+ - ``genetlink-legacy`` - Generic Netlink catch all schema supporting quirks of
+ all old genetlink families, strange attribute formats, binary structures etc.
+ - ``netlink-raw`` - catch all schema supporting pre-Generic Netlink protocols
+ such as ``NETLINK_ROUTE``
+
+The definition of the schemas (in ``jsonschema``) can be found
+under ``Documentation/netlink/``.
+
+Schema structure
+================
+
+YAML schema has the following conceptual sections:
+
+ - globals
+ - definitions
+ - attributes
+ - operations
+ - multicast groups
+
+Most properties in the schema accept (or in fact require) a ``doc``
+sub-property documenting the defined object.
+
+The following sections describe the properties of the most modern ``genetlink``
+schema. See the documentation of :doc:`genetlink-c <c-code-gen>`
+for information on how C names are derived from name properties.
+
+genetlink
+=========
+
+Globals
+-------
+
+Attributes listed directly at the root level of the spec file.
+
+name
+~~~~
+
+Name of the family. Name identifies the family in a unique way, since
+the Family IDs are allocated dynamically.
+
+version
+~~~~~~~
+
+Generic Netlink family version, default is 1.
+
+protocol
+~~~~~~~~
+
+The schema level, default is ``genetlink``, which is the only value
+allowed for new ``genetlink`` families.
+
+definitions
+-----------
+
+Array of type and constant definitions.
+
+name
+~~~~
+
+Name of the type / constant.
+
+type
+~~~~
+
+One of the following types:
+
+ - const - a single, standalone constant
+ - enum - defines an integer enumeration, with values for each entry
+ incrementing by 1, (e.g. 0, 1, 2, 3)
+ - flags - defines an integer enumeration, with values for each entry
+ occupying a bit, starting from bit 0, (e.g. 1, 2, 4, 8)
+
+value
+~~~~~
+
+The value for the ``const``.
+
+value-start
+~~~~~~~~~~~
+
+The first value for ``enum`` and ``flags``, allows overriding the default
+start value of ``0`` (for ``enum``) and starting bit (for ``flags``).
+For ``flags`` ``value-start`` selects the starting bit, not the shifted value.
+
+Sparse enumerations are not supported.
+
+entries
+~~~~~~~
+
+Array of names of the entries for ``enum`` and ``flags``.
+
+header
+~~~~~~
+
+For C-compatible languages, header which already defines this value.
+In case the definition is shared by multiple families (e.g. ``IFNAMSIZ``)
+code generators for C-compatible languages may prefer to add an appropriate
+include instead of rendering a new definition.
+
+attribute-sets
+--------------
+
+This property contains information about netlink attributes of the family.
+All families have at least one attribute set, most have multiple.
+``attribute-sets`` is an array, with each entry describing a single set.
+
+Note that the spec is "flattened" and is not meant to visually resemble
+the format of the netlink messages (unlike certain ad-hoc documentation
+formats seen in kernel comments). In the spec subordinate attribute sets
+are not defined inline as a nest, but defined in a separate attribute set
+referred to with a ``nested-attributes`` property of the container.
+
+Spec may also contain fractional sets - sets which contain a ``subset-of``
+property. Such sets describe a section of a full set, allowing narrowing down
+which attributes are allowed in a nest or refining the validation criteria.
+Fractional sets can only be used in nests. They are not rendered to the uAPI
+in any fashion.
+
+name
+~~~~
+
+Uniquely identifies the attribute set, operations and nested attributes
+refer to the sets by the ``name``.
+
+subset-of
+~~~~~~~~~
+
+Re-defines a portion of another set (a fractional set).
+Allows narrowing down fields and changing validation criteria
+or even types of attributes depending on the nest in which they
+are contained. The ``value`` of each attribute in the fractional
+set is implicitly the same as in the main set.
+
+attributes
+~~~~~~~~~~
+
+List of attributes in the set.
+
+Attribute properties
+--------------------
+
+name
+~~~~
+
+Identifies the attribute, unique within the set.
+
+type
+~~~~
+
+Netlink attribute type, see :ref:`attr_types`.
+
+.. _assign_val:
+
+value
+~~~~~
+
+Numerical attribute ID, used in serialized Netlink messages.
+The ``value`` property can be skipped, in which case the attribute ID
+will be the value of the previous attribute plus one (recursively)
+and ``0`` for the first attribute in the attribute set.
+
+Note that the ``value`` of an attribute is defined only in its main set.
+
+enum
+~~~~
+
+For integer types specifies that values in the attribute belong
+to an ``enum`` or ``flags`` from the ``definitions`` section.
+
+enum-as-flags
+~~~~~~~~~~~~~
+
+Treat ``enum`` as ``flags`` regardless of its type in ``definitions``.
+When both ``enum`` and ``flags`` forms are needed ``definitions`` should
+contain an ``enum`` and attributes which need the ``flags`` form should
+use this attribute.
+
+nested-attributes
+~~~~~~~~~~~~~~~~~
+
+Identifies the attribute space for attributes nested within given attribute.
+Only valid for complex attributes which may have sub-attributes.
+
+multi-attr (arrays)
+~~~~~~~~~~~~~~~~~~~
+
+Boolean property signifying that the attribute may be present multiple times.
+Allowing an attribute to repeat is the recommended way of implementing arrays
+(no extra nesting).
+
+byte-order
+~~~~~~~~~~
+
+For integer types specifies attribute byte order - ``little-endian``
+or ``big-endian``.
+
+checks
+~~~~~~
+
+Input validation constraints used by the kernel. User space should query
+the policy of the running kernel using Generic Netlink introspection,
+rather than depend on what is specified in the spec file.
+
+The validation policy in the kernel is formed by combining the type
+definition (``type`` and ``nested-attributes``) and the ``checks``.
+
+operations
+----------
+
+This section describes messages passed between the kernel and the user space.
+There are three types of entries in this section - operations, notifications
+and events.
+
+Operations describe the most common request - response communication. User
+sends a request and kernel replies. Each operation may contain any combination
+of the two modes familiar to netlink users - ``do`` and ``dump``.
+``do`` and ``dump`` in turn contain a combination of ``request`` and
+``response`` properties. If no explicit message with attributes is passed
+in a given direction (e.g. a ``dump`` which does not accept filter, or a ``do``
+of a SET operation to which the kernel responds with just the netlink error
+code) ``request`` or ``response`` section can be skipped.
+``request`` and ``response`` sections list the attributes allowed in a message.
+The list contains only the names of attributes from a set referred
+to by the ``attribute-set`` property.
+
+Notifications and events both refer to the asynchronous messages sent by
+the kernel to members of a multicast group. The difference between the
+two is that a notification shares its contents with a GET operation
+(the name of the GET operation is specified in the ``notify`` property).
+This arrangement is commonly used for notifications about
+objects where the notification carries the full object definition.
+
+Events are more focused and carry only a subset of information rather than full
+object state (a made up example would be a link state change event with just
+the interface name and the new link state). Events contain the ``event``
+property. Events are considered less idiomatic for netlink and notifications
+should be preferred.
+
+list
+~~~~
+
+The only property of ``operations`` for ``genetlink``, holds the list of
+operations, notifications etc.
+
+Operation properties
+--------------------
+
+name
+~~~~
+
+Identifies the operation.
+
+value
+~~~~~
+
+Numerical message ID, used in serialized Netlink messages.
+The same enumeration rules are applied as to
+:ref:`attribute values<assign_val>`.
+
+attribute-set
+~~~~~~~~~~~~~
+
+Specifies the attribute set contained within the message.
+
+do
+~~~
+
+Specification for the ``doit`` request. Should contain ``request``, ``reply``
+or both of these properties, each holding a :ref:`attr_list`.
+
+dump
+~~~~
+
+Specification for the ``dumpit`` request. Should contain ``request``, ``reply``
+or both of these properties, each holding a :ref:`attr_list`.
+
+notify
+~~~~~~
+
+Designates the message as a notification. Contains the name of the operation
+(possibly the same as the operation holding this property) which shares
+the contents with the notification (``do``).
+
+event
+~~~~~
+
+Specification of attributes in the event, holds a :ref:`attr_list`.
+``event`` property is mutually exclusive with ``notify``.
+
+mcgrp
+~~~~~
+
+Used with ``event`` and ``notify``, specifies which multicast group
+message belongs to.
+
+.. _attr_list:
+
+Message attribute list
+----------------------
+
+``request``, ``reply`` and ``event`` properties have a single ``attributes``
+property which holds the list of attribute names.
+
+Messages can also define ``pre`` and ``post`` properties which will be rendered
+as ``pre_doit`` and ``post_doit`` calls in the kernel (these properties should
+be ignored by user space).
+
+mcast-groups
+------------
+
+This section lists the multicast groups of the family.
+
+list
+~~~~
+
+The only property of ``mcast-groups`` for ``genetlink``, holds the list
+of groups.
+
+Multicast group properties
+--------------------------
+
+name
+~~~~
+
+Uniquely identifies the multicast group in the family. Similarly to
+Family ID, Multicast Group ID needs to be resolved at runtime, based
+on the name.
+
+.. _attr_types:
+
+Attribute types
+===============
+
+This section describes the attribute types supported by the ``genetlink``
+compatibility level. Refer to documentation of different levels for additional
+attribute types.
+
+Scalar integer types
+--------------------
+
+Fixed-width integer types:
+``u8``, ``u16``, ``u32``, ``u64``, ``s8``, ``s16``, ``s32``, ``s64``.
+
+Note that types smaller than 32 bit should be avoided as using them
+does not save any memory in Netlink messages (due to alignment).
+See :ref:`pad_type` for padding of 64 bit attributes.
+
+The payload of the attribute is the integer in host order unless ``byte-order``
+specifies otherwise.
+
+.. _pad_type:
+
+pad
+---
+
+Special attribute type used for padding attributes which require alignment
+bigger than standard 4B alignment required by netlink (e.g. 64 bit integers).
+There can only be a single attribute of the ``pad`` type in any attribute set
+and it should be automatically used for padding when needed.
+
+flag
+----
+
+Attribute with no payload, its presence is the entire information.
+
+binary
+------
+
+Raw binary data attribute, the contents are opaque to generic code.
+
+string
+------
+
+Character string. Unless ``checks`` has ``unterminated-ok`` set to ``true``
+the string is required to be null terminated.
+``max-len`` in ``checks`` indicates the longest possible string,
+if not present the length of the string is unbounded.
+
+Note that ``max-len`` does not count the terminating character.
+
+nest
+----
+
+Attribute containing other (nested) attributes.
+``nested-attributes`` specifies which attribute set is used inside.
diff --git a/MAINTAINERS b/MAINTAINERS
index f82dd8d43c2b..3842915d863d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14563,8 +14563,10 @@ Q: https://patchwork.kernel.org/project/netdevbpf/list/
B: mailto:netdev@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git
+F: Documentation/core-api/netlink.rst
F: Documentation/networking/
F: Documentation/process/maintainer-netdev.rst
+F: Documentation/userspace-api/netlink/
F: include/linux/in.h
F: include/linux/net.h
F: include/linux/netdevice.h
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v3 2/8] netlink: add schemas for YAML specs
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs) Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-19 14:07 ` Rob Herring
2023-01-19 0:36 ` [PATCH net-next v3 3/8] net: add basic C code generators for Netlink Jakub Kicinski
` (6 subsequent siblings)
8 siblings, 1 reply; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
Add schemas for Netlink spec files. As described in the docs
we have 4 "protocols" or compatibility levels, and each one
comes with its own schema, but the more general / legacy
schemas are superset of more modern ones: genetlink is
the smallest followed by genetlink-c and genetlink-legacy.
There is no schema for raw netlink, yet, I haven't found the time..
I don't know enough jsonschema to do inheritance or something
but the repetition is not too bad. I hope.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
Documentation/netlink/genetlink-c.yaml | 318 ++++++++++++++++++
Documentation/netlink/genetlink-legacy.yaml | 346 ++++++++++++++++++++
Documentation/netlink/genetlink.yaml | 284 ++++++++++++++++
3 files changed, 948 insertions(+)
create mode 100644 Documentation/netlink/genetlink-c.yaml
create mode 100644 Documentation/netlink/genetlink-legacy.yaml
create mode 100644 Documentation/netlink/genetlink.yaml
diff --git a/Documentation/netlink/genetlink-c.yaml b/Documentation/netlink/genetlink-c.yaml
new file mode 100644
index 000000000000..53a7fe94a47e
--- /dev/null
+++ b/Documentation/netlink/genetlink-c.yaml
@@ -0,0 +1,318 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#"
+$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
+
+title: Protocol
+description: Specification of a genetlink protocol
+type: object
+required: [ name, doc, attribute-sets, operations ]
+additionalProperties: False
+properties:
+ name:
+ description: Name of the genetlink family.
+ type: string
+ doc:
+ type: string
+ version:
+ description: Generic Netlink family version. Default is 1.
+ type: integer
+ protocol:
+ description: Schema compatibility level. Default is "genetlink".
+ enum: [ genetlink, genetlink-c]
+ # Start genetlink-c
+ uapi-header:
+ description: Path to the uAPI header, default is linux/${family-name}.h
+ type: string
+ c-family-name:
+ description: Name of the define for the family name.
+ type: string
+ c-version-name:
+ description: Name of the define for the verion of the family.
+ type: string
+ max-by-define:
+ description: Makes the number of attributes and commands be specified by a define, not an enum value.
+ type: boolean
+ # End genetlink-c
+
+ definitions:
+ description: Array of type and constant definitions.
+ type: array
+ items:
+ type: object
+ required: [ type, name ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ header:
+ description: For C-compatible languages, header which already defines this value.
+ type: string
+ type:
+ enum: [ const, enum, flags ]
+ doc:
+ type: string
+ # For const
+ value:
+ description: For const - the value.
+ type: [ string, integer ]
+ # For enum and flags
+ value-start:
+ description: For enum or flags the literal initializer for the first value.
+ type: [ string, integer ]
+ entries:
+ description: For enum or flags array of values
+ type: array
+ items:
+ oneOf:
+ - type: string
+ - type: object
+ required: [ name ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ value:
+ type: integer
+ doc:
+ type: string
+ render-max:
+ description: Render the max members for this enum.
+ type: boolean
+ # Start genetlink-c
+ enum-name:
+ description: Name for enum, if empty no name will be used.
+ type: [ string, "null" ]
+ name-prefix:
+ description: For enum the prefix of the values, optional.
+ type: string
+ # End genetlink-c
+
+ attribute-sets:
+ description: Definition of attribute spaces for this family.
+ type: array
+ items:
+ description: Definition of a single attribute space.
+ type: object
+ required: [ name, attributes ]
+ additionalProperties: False
+ properties:
+ name:
+ description: |
+ Name used when referring to this space in other definitions, not used outside of YAML.
+ type: string
+ # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
+ name-prefix:
+ description: |
+ Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
+ type: string
+ enum-name:
+ description: Name for the enum type of the attribute.
+ type: string
+ doc:
+ description: Documentation of the space.
+ type: string
+ subset-of:
+ description: |
+ Name of another space which this is a logical part of. Sub-spaces can be used to define
+ a limited group of attributes which are used in a nest.
+ type: string
+ # Start genetlink-c
+ attr-cnt-name:
+ description: The explicit name for constant holding the count of attributes (last attr + 1).
+ type: string
+ attr-max-name:
+ description: The explicit name for last member of attribute enum.
+ type: string
+ # End genetlink-c
+ attributes:
+ description: List of attributes in the space.
+ type: array
+ items:
+ type: object
+ required: [ name, type ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ type: &attr-type
+ enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+ string, nest, array-nest, nest-type-value ]
+ doc:
+ description: Documentation of the attribute.
+ type: string
+ value:
+ description: Value for the enum item representing this attribute in the uAPI.
+ type: integer
+ type-value:
+ description: Name of the value extracted from the type of a nest-type-value attribute.
+ type: array
+ items:
+ type: string
+ byte-order:
+ enum: [little-endian, big-endian]
+ multi-attr:
+ type: boolean
+ nested-attributes:
+ description: Name of the space (sub-space) used inside the attribute.
+ type: string
+ enum:
+ description: Name of the enum type used for the atttribute.
+ type: string
+ enum-as-flags:
+ description: |
+ Treat the enum as flags. In most cases enum is either used as flags or as values.
+ Sometimes, however, both forms are necessary, in which case header contains the enum
+ form while specific attributes may request to convert the values into a bitfield.
+ type: boolean
+ checks:
+ description: Kernel input validation.
+ type: object
+ additionalProperties: False
+ properties:
+ flags-mask:
+ description: Name of the flags constant on which to base mask (unsigned scalar types only).
+ type: string
+ min:
+ description: Min value for an integer attribute.
+ type: integer
+ min-len:
+ description: Min length for a binary attribute.
+ oneOf:
+ - type: string
+ pattern: ^[0-9A-Za-z_-]*( - 1)?$
+ - type: integer
+ max-len:
+ description: Max length for a string or a binary attribute.
+ oneOf:
+ - type: string
+ pattern: ^[0-9A-Za-z_-]*( - 1)?$
+ - type: integer
+ sub-type: *attr-type
+ operations:
+ description: Operations supported by the protocol.
+ type: object
+ required: [ list ]
+ additionalProperties: False
+ properties:
+ enum-model:
+ description: |
+ The model of assigning values to the operations.
+ "unified" is the recommended model where all message types belong
+ to a single enum.
+ "directional" has the messages sent to the kernel and from the kernel
+ enumerated separately.
+ "notify-split" has the notifications and request-response types in
+ different enums.
+ enum: [ unified, directional, notify-split ]
+ name-prefix:
+ description: |
+ Prefix for the C enum name of the command. The name is formed by concatenating
+ the prefix with the upper case name of the command, with dashes replaced by underscores.
+ type: string
+ enum-name:
+ description: Name for the enum type with commands.
+ type: string
+ async-prefix:
+ description: Same as name-prefix but used to render notifications and events to separate enum.
+ type: string
+ async-enum:
+ description: Name for the enum type with notifications/events.
+ type: string
+ list:
+ description: List of commands
+ type: array
+ items:
+ type: object
+ additionalProperties: False
+ required: [ name, doc ]
+ properties:
+ name:
+ description: Name of the operation, also defining its C enum value in uAPI.
+ type: string
+ doc:
+ description: Documentation for the command.
+ type: string
+ value:
+ description: Value for the enum in the uAPI.
+ type: integer
+ attribute-set:
+ description: |
+ Attribute space from which attributes directly in the requests and replies
+ to this command are defined.
+ type: string
+ flags: &cmd_flags
+ description: Command flags.
+ type: array
+ items:
+ enum: [ admin-perm ]
+ dont-validate:
+ description: Kernel attribute validation flags.
+ type: array
+ items:
+ enum: [ strict, dump ]
+ do: &subop-type
+ description: Main command handler.
+ type: object
+ additionalProperties: False
+ properties:
+ request: &subop-attr-list
+ description: Definition of the request message for a given command.
+ type: object
+ additionalProperties: False
+ properties:
+ attributes:
+ description: |
+ Names of attributes from the attribute-set (not full attribute
+ definitions, just names).
+ type: array
+ items:
+ type: string
+ reply: *subop-attr-list
+ pre:
+ description: Hook for a function to run before the main callback (pre_doit or start).
+ type: string
+ post:
+ description: Hook for a function to run after the main callback (post_doit or done).
+ type: string
+ dump: *subop-type
+ notify:
+ description: Name of the command sharing the reply type with this notification.
+ type: string
+ event:
+ type: object
+ additionalProperties: False
+ properties:
+ attributes:
+ description: Explicit list of the attributes for the notification.
+ type: array
+ items:
+ type: string
+ mcgrp:
+ description: Name of the multicast group generating given notification.
+ type: string
+ mcast-groups:
+ description: List of multicast groups.
+ type: object
+ required: [ list ]
+ additionalProperties: False
+ properties:
+ list:
+ description: List of groups.
+ type: array
+ items:
+ type: object
+ required: [ name ]
+ additionalProperties: False
+ properties:
+ name:
+ description: |
+ The name for the group, used to form the define and the value of the define.
+ type: string
+ # Start genetlink-c
+ c-define-name:
+ description: Override for the name of the define in C uAPI.
+ type: string
+ # End genetlink-c
+ flags: *cmd_flags
diff --git a/Documentation/netlink/genetlink-legacy.yaml b/Documentation/netlink/genetlink-legacy.yaml
new file mode 100644
index 000000000000..5e57453eac1a
--- /dev/null
+++ b/Documentation/netlink/genetlink-legacy.yaml
@@ -0,0 +1,346 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: "http://kernel.org/schemas/netlink/genetlink-legacy.yaml#"
+$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
+
+title: Protocol
+description: Specification of a genetlink protocol
+type: object
+required: [ name, doc, attribute-sets, operations ]
+additionalProperties: False
+properties:
+ name:
+ description: Name of the genetlink family.
+ type: string
+ doc:
+ type: string
+ version:
+ description: Generic Netlink family version. Default is 1.
+ type: integer
+ protocol:
+ description: Schema compatibility level. Default is "genetlink".
+ enum: [ genetlink, genetlink-c, genetlink-legacy, netlink-raw ] # Trim
+ # Start genetlink-c
+ uapi-header:
+ description: Path to the uAPI header, default is linux/${family-name}.h
+ type: string
+ c-family-name:
+ description: Name of the define for the family name.
+ type: string
+ c-version-name:
+ description: Name of the define for the verion of the family.
+ type: string
+ max-by-define:
+ description: Makes the number of attributes and commands be specified by a define, not an enum value.
+ type: boolean
+ # End genetlink-c
+ # Start genetlink-legacy
+ kernel-policy:
+ description: |
+ Defines if the input policy in the kernel is global, per-operation, or split per operation type.
+ Default is split.
+ enum: [split, per-op, global]
+ # End genetlink-legacy
+
+ definitions:
+ description: Array of type and constant definitions.
+ type: array
+ items:
+ type: object
+ required: [ type, name ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ header:
+ description: For C-compatible languages, header which already defines this value.
+ type: string
+ type:
+ enum: [ const, enum, flags, struct ] # Trim
+ doc:
+ type: string
+ # For const
+ value:
+ description: For const - the value.
+ type: [ string, integer ]
+ # For enum and flags
+ value-start:
+ description: For enum or flags the literal initializer for the first value.
+ type: [ string, integer ]
+ entries:
+ description: For enum or flags array of values
+ type: array
+ items:
+ oneOf:
+ - type: string
+ - type: object
+ required: [ name ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ value:
+ type: integer
+ doc:
+ type: string
+ render-max:
+ description: Render the max members for this enum.
+ type: boolean
+ # Start genetlink-c
+ enum-name:
+ description: Name for enum, if empty no name will be used.
+ type: [ string, "null" ]
+ name-prefix:
+ description: For enum the prefix of the values, optional.
+ type: string
+ # End genetlink-c
+ # Start genetlink-legacy
+ members:
+ description: List of struct members. Only scalars and strings members allowed.
+ type: array
+ items:
+ type: object
+ required: [ name, type ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ type:
+ enum: [ u8, u16, u32, u64, s8, s16, s32, s64, string ]
+ len:
+ oneOf:
+ -
+ type: string
+ pattern: ^[0-9A-Za-z_-]*( - 1)?$
+ -
+ type: integer
+ # End genetlink-legacy
+
+ attribute-sets:
+ description: Definition of attribute spaces for this family.
+ type: array
+ items:
+ description: Definition of a single attribute space.
+ type: object
+ required: [ name, attributes ]
+ additionalProperties: False
+ properties:
+ name:
+ description: |
+ Name used when referring to this space in other definitions, not used outside of YAML.
+ type: string
+ # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
+ name-prefix:
+ description: |
+ Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
+ type: string
+ enum-name:
+ description: Name for the enum type of the attribute.
+ type: string
+ doc:
+ description: Documentation of the space.
+ type: string
+ subset-of:
+ description: |
+ Name of another space which this is a logical part of. Sub-spaces can be used to define
+ a limited group of attributes which are used in a nest.
+ type: string
+ # Start genetlink-c
+ attr-cnt-name:
+ description: The explicit name for constant holding the count of attributes (last attr + 1).
+ type: string
+ attr-max-name:
+ description: The explicit name for last member of attribute enum.
+ type: string
+ # End genetlink-c
+ attributes:
+ description: List of attributes in the space.
+ type: array
+ items:
+ type: object
+ required: [ name, type ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ type: &attr-type
+ enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+ string, nest, array-nest, nest-type-value ]
+ doc:
+ description: Documentation of the attribute.
+ type: string
+ value:
+ description: Value for the enum item representing this attribute in the uAPI.
+ type: integer
+ type-value:
+ description: Name of the value extracted from the type of a nest-type-value attribute.
+ type: array
+ items:
+ type: string
+ byte-order:
+ enum: [little-endian, big-endian]
+ multi-attr:
+ type: boolean
+ nested-attributes:
+ description: Name of the space (sub-space) used inside the attribute.
+ type: string
+ enum:
+ description: Name of the enum type used for the atttribute.
+ type: string
+ enum-as-flags:
+ description: |
+ Treat the enum as flags. In most cases enum is either used as flags or as values.
+ Sometimes, however, both forms are necessary, in which case header contains the enum
+ form while specific attributes may request to convert the values into a bitfield.
+ type: boolean
+ checks:
+ description: Kernel input validation.
+ type: object
+ additionalProperties: False
+ properties:
+ flags-mask:
+ description: Name of the flags constant on which to base mask (unsigned scalar types only).
+ type: string
+ min:
+ description: Min value for an integer attribute.
+ type: integer
+ min-len:
+ description: Min length for a binary attribute.
+ oneOf:
+ - type: string
+ pattern: ^[0-9A-Za-z_-]*( - 1)?$
+ - type: integer
+ max-len:
+ description: Max length for a string or a binary attribute.
+ oneOf:
+ - type: string
+ pattern: ^[0-9A-Za-z_-]*( - 1)?$
+ - type: integer
+ sub-type: *attr-type
+ operations:
+ description: Operations supported by the protocol.
+ type: object
+ required: [ list ]
+ additionalProperties: False
+ properties:
+ enum-model:
+ description: |
+ The model of assigning values to the operations.
+ "unified" is the recommended model where all message types belong
+ to a single enum.
+ "directional" has the messages sent to the kernel and from the kernel
+ enumerated separately.
+ "notify-split" has the notifications and request-response types in
+ different enums.
+ enum: [ unified, directional, notify-split ]
+ name-prefix:
+ description: |
+ Prefix for the C enum name of the command. The name is formed by concatenating
+ the prefix with the upper case name of the command, with dashes replaced by underscores.
+ type: string
+ enum-name:
+ description: Name for the enum type with commands.
+ type: string
+ async-prefix:
+ description: Same as name-prefix but used to render notifications and events to separate enum.
+ type: string
+ async-enum:
+ description: Name for the enum type with notifications/events.
+ type: string
+ list:
+ description: List of commands
+ type: array
+ items:
+ type: object
+ additionalProperties: False
+ required: [ name, doc ]
+ properties:
+ name:
+ description: Name of the operation, also defining its C enum value in uAPI.
+ type: string
+ doc:
+ description: Documentation for the command.
+ type: string
+ value:
+ description: Value for the enum in the uAPI.
+ type: integer
+ attribute-set:
+ description: |
+ Attribute space from which attributes directly in the requests and replies
+ to this command are defined.
+ type: string
+ flags: &cmd_flags
+ description: Command flags.
+ type: array
+ items:
+ enum: [ admin-perm ]
+ dont-validate:
+ description: Kernel attribute validation flags.
+ type: array
+ items:
+ enum: [ strict, dump ]
+ do: &subop-type
+ description: Main command handler.
+ type: object
+ additionalProperties: False
+ properties:
+ request: &subop-attr-list
+ description: Definition of the request message for a given command.
+ type: object
+ additionalProperties: False
+ properties:
+ attributes:
+ description: |
+ Names of attributes from the attribute-set (not full attribute
+ definitions, just names).
+ type: array
+ items:
+ type: string
+ reply: *subop-attr-list
+ pre:
+ description: Hook for a function to run before the main callback (pre_doit or start).
+ type: string
+ post:
+ description: Hook for a function to run after the main callback (post_doit or done).
+ type: string
+ dump: *subop-type
+ notify:
+ description: Name of the command sharing the reply type with this notification.
+ type: string
+ event:
+ type: object
+ additionalProperties: False
+ properties:
+ attributes:
+ description: Explicit list of the attributes for the notification.
+ type: array
+ items:
+ type: string
+ mcgrp:
+ description: Name of the multicast group generating given notification.
+ type: string
+ mcast-groups:
+ description: List of multicast groups.
+ type: object
+ required: [ list ]
+ additionalProperties: False
+ properties:
+ list:
+ description: List of groups.
+ type: array
+ items:
+ type: object
+ required: [ name ]
+ additionalProperties: False
+ properties:
+ name:
+ description: |
+ The name for the group, used to form the define and the value of the define.
+ type: string
+ # Start genetlink-c
+ c-define-name:
+ description: Override for the name of the define in C uAPI.
+ type: string
+ # End genetlink-c
+ flags: *cmd_flags
diff --git a/Documentation/netlink/genetlink.yaml b/Documentation/netlink/genetlink.yaml
new file mode 100644
index 000000000000..aafaaaabcfbe
--- /dev/null
+++ b/Documentation/netlink/genetlink.yaml
@@ -0,0 +1,284 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: "http://kernel.org/schemas/netlink/genetlink.yaml#"
+$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
+
+title: Protocol
+description: Specification of a genetlink protocol
+type: object
+required: [ name, doc, attribute-sets, operations ]
+additionalProperties: False
+properties:
+ name:
+ description: Name of the genetlink family.
+ type: string
+ doc:
+ type: string
+ version:
+ description: Generic Netlink family version. Default is 1.
+ type: integer
+ protocol:
+ description: Schema compatibility level. Default is "genetlink".
+ enum: [ genetlink ]
+
+ definitions:
+ description: Array of type and constant definitions.
+ type: array
+ items:
+ type: object
+ required: [ type, name ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ header:
+ description: For C-compatible languages, header which already defines this value.
+ type: string
+ type:
+ enum: [ const, enum, flags ]
+ doc:
+ type: string
+ # For const
+ value:
+ description: For const - the value.
+ type: [ string, integer ]
+ # For enum and flags
+ value-start:
+ description: For enum or flags the literal initializer for the first value.
+ type: [ string, integer ]
+ entries:
+ description: For enum or flags array of values
+ type: array
+ items:
+ oneOf:
+ - type: string
+ - type: object
+ required: [ name ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ value:
+ type: integer
+ doc:
+ type: string
+
+ render-max:
+ description: Render the max members for this enum.
+ type: boolean
+
+ attribute-sets:
+ description: Definition of attribute spaces for this family.
+ type: array
+ items:
+ description: Definition of a single attribute space.
+ type: object
+ required: [ name, attributes ]
+ additionalProperties: False
+ properties:
+ name:
+ description: |
+ Name used when referring to this space in other definitions, not used outside of YAML.
+ type: string
+ # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
+ name-prefix:
+ description: |
+ Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
+ type: string
+ enum-name:
+ description: Name for the enum type of the attribute.
+ type: string
+ doc:
+ description: Documentation of the space.
+ type: string
+ subset-of:
+ description: |
+ Name of another space which this is a logical part of. Sub-spaces can be used to define
+ a limited group of attributes which are used in a nest.
+ type: string
+ attributes:
+ description: List of attributes in the space.
+ type: array
+ items:
+ type: object
+ required: [ name, type ]
+ additionalProperties: False
+ properties:
+ name:
+ type: string
+ type: &attr-type
+ enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+ string, nest, array-nest, nest-type-value ]
+ doc:
+ description: Documentation of the attribute.
+ type: string
+ value:
+ description: Value for the enum item representing this attribute in the uAPI.
+ type: integer
+ type-value:
+ description: Name of the value extracted from the type of a nest-type-value attribute.
+ type: array
+ items:
+ type: string
+ byte-order:
+ enum: [little-endian, big-endian]
+ multi-attr:
+ type: boolean
+ nested-attributes:
+ description: Name of the space (sub-space) used inside the attribute.
+ type: string
+ enum:
+ description: Name of the enum type used for the atttribute.
+ type: string
+ enum-as-flags:
+ description: |
+ Treat the enum as flags. In most cases enum is either used as flags or as values.
+ Sometimes, however, both forms are necessary, in which case header contains the enum
+ form while specific attributes may request to convert the values into a bitfield.
+ type: boolean
+ checks:
+ description: Kernel input validation.
+ type: object
+ additionalProperties: False
+ properties:
+ flags-mask:
+ description: Name of the flags constant on which to base mask (unsigned scalar types only).
+ type: string
+ min:
+ description: Min value for an integer attribute.
+ type: integer
+ min-len:
+ description: Min length for a binary attribute.
+ oneOf:
+ - type: string
+ pattern: ^[0-9A-Za-z_-]*( - 1)?$
+ - type: integer
+ max-len:
+ description: Max length for a string or a binary attribute.
+ oneOf:
+ - type: string
+ pattern: ^[0-9A-Za-z_-]*( - 1)?$
+ - type: integer
+ sub-type: *attr-type
+ operations:
+ description: Operations supported by the protocol.
+ type: object
+ required: [ list ]
+ additionalProperties: False
+ properties:
+ enum-model:
+ description: |
+ The model of assigning values to the operations.
+ "unified" is the recommended model where all message types belong
+ to a single enum.
+ "directional" has the messages sent to the kernel and from the kernel
+ enumerated separately.
+ "notify-split" has the notifications and request-response types in
+ different enums.
+ enum: [ unified, directional, notify-split ]
+ name-prefix:
+ description: |
+ Prefix for the C enum name of the command. The name is formed by concatenating
+ the prefix with the upper case name of the command, with dashes replaced by underscores.
+ type: string
+ enum-name:
+ description: Name for the enum type with commands.
+ type: string
+ async-prefix:
+ description: Same as name-prefix but used to render notifications and events to separate enum.
+ type: string
+ async-enum:
+ description: Name for the enum type with notifications/events.
+ type: string
+ list:
+ description: List of commands
+ type: array
+ items:
+ type: object
+ additionalProperties: False
+ required: [ name, doc ]
+ properties:
+ name:
+ description: Name of the operation, also defining its C enum value in uAPI.
+ type: string
+ doc:
+ description: Documentation for the command.
+ type: string
+ value:
+ description: Value for the enum in the uAPI.
+ type: integer
+ attribute-set:
+ description: |
+ Attribute space from which attributes directly in the requests and replies
+ to this command are defined.
+ type: string
+ flags: &cmd_flags
+ description: Command flags.
+ type: array
+ items:
+ enum: [ admin-perm ]
+ dont-validate:
+ description: Kernel attribute validation flags.
+ type: array
+ items:
+ enum: [ strict, dump ]
+ do: &subop-type
+ description: Main command handler.
+ type: object
+ additionalProperties: False
+ properties:
+ request: &subop-attr-list
+ description: Definition of the request message for a given command.
+ type: object
+ additionalProperties: False
+ properties:
+ attributes:
+ description: |
+ Names of attributes from the attribute-set (not full attribute
+ definitions, just names).
+ type: array
+ items:
+ type: string
+ reply: *subop-attr-list
+ pre:
+ description: Hook for a function to run before the main callback (pre_doit or start).
+ type: string
+ post:
+ description: Hook for a function to run after the main callback (post_doit or done).
+ type: string
+ dump: *subop-type
+ notify:
+ description: Name of the command sharing the reply type with this notification.
+ type: string
+ event:
+ type: object
+ additionalProperties: False
+ properties:
+ attributes:
+ description: Explicit list of the attributes for the notification.
+ type: array
+ items:
+ type: string
+ mcgrp:
+ description: Name of the multicast group generating given notification.
+ type: string
+ mcast-groups:
+ description: List of multicast groups.
+ type: object
+ required: [ list ]
+ additionalProperties: False
+ properties:
+ list:
+ description: List of groups.
+ type: array
+ items:
+ type: object
+ required: [ name ]
+ additionalProperties: False
+ properties:
+ name:
+ description: |
+ The name for the group, used to form the define and the value of the define.
+ type: string
+ flags: *cmd_flags
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v3 3/8] net: add basic C code generators for Netlink
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs) Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 2/8] netlink: add schemas for YAML specs Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-19 20:53 ` Johannes Berg
2023-01-19 0:36 ` [PATCH net-next v3 4/8] netlink: add a proto specification for FOU Jakub Kicinski
` (5 subsequent siblings)
8 siblings, 1 reply; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
Code generators to turn Netlink specs into C code.
I'm definitely not proud of it.
The main generator is in Python, there's a bash script
to regen all code-gen'ed files in tree after making
spec changes.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
--
v2: - use /* */ comments instead of //
---
MAINTAINERS | 1 +
tools/net/ynl/ynl-gen-c.py | 2376 ++++++++++++++++++++++++++++++++++++
tools/net/ynl/ynl-regen.sh | 30 +
3 files changed, 2407 insertions(+)
create mode 100755 tools/net/ynl/ynl-gen-c.py
create mode 100755 tools/net/ynl/ynl-regen.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index 3842915d863d..75a1395c8023 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14578,6 +14578,7 @@ F: include/uapi/linux/netdevice.h
F: lib/net_utils.c
F: lib/random32.c
F: net/
+F: tools/net/
F: tools/testing/selftests/net/
NETWORKING [IPSEC]
diff --git a/tools/net/ynl/ynl-gen-c.py b/tools/net/ynl/ynl-gen-c.py
new file mode 100755
index 000000000000..c3ffea6c01d4
--- /dev/null
+++ b/tools/net/ynl/ynl-gen-c.py
@@ -0,0 +1,2376 @@
+#!/usr/bin/env python
+
+import argparse
+import jsonschema
+import os
+import yaml
+
+
+def c_upper(name):
+ return name.upper().replace('-', '_')
+
+
+def c_lower(name):
+ return name.lower().replace('-', '_')
+
+
+class BaseNlLib:
+ def __init__(self):
+ pass
+
+ def get_family_id(self):
+ return 'ys->family_id'
+
+ def parse_cb_run(self, cb, data, is_dump=False, indent=1):
+ ind = '\n\t\t' + '\t' * indent + ' '
+ if is_dump:
+ return f"mnl_cb_run2(ys->rx_buf, len, 0, 0, {cb}, {data},{ind}ynl_cb_array, NLMSG_MIN_TYPE)"
+ else:
+ return f"mnl_cb_run2(ys->rx_buf, len, ys->seq, ys->portid,{ind}{cb}, {data},{ind}" + \
+ "ynl_cb_array, NLMSG_MIN_TYPE)"
+
+
+class Type:
+ def __init__(self, family, attr_set, attr):
+ self.family = family
+ self.attr = attr
+ self.value = attr['value']
+ self.name = attr['name'].replace('-', '_')
+ self.type = attr['type']
+ self.checks = attr.get('checks', {})
+
+ if 'len' in attr:
+ self.len = attr['len']
+ if 'nested-attributes' in attr:
+ self.nested_attrs = attr['nested-attributes']
+ if self.nested_attrs == family.name:
+ self.nested_render_name = f"{family.name}"
+ else:
+ self.nested_render_name = f"{family.name}_{self.nested_attrs.replace('-', '_')}"
+
+ self.enum_name = f"{attr_set.name_prefix}{self.name}"
+ self.enum_name = self.enum_name.upper().replace('-', '_')
+ self.c_name = self.name.replace('-', '_')
+ if self.c_name in c_kw:
+ self.c_name += '_'
+
+ def __getitem__(self, key):
+ return self.attr[key]
+
+ def __contains__(self, key):
+ return key in self.attr
+
+ def is_multi_val(self):
+ return None
+
+ def is_scalar(self):
+ return self.type in {'u8', 'u16', 'u32', 'u64', 's32', 's64'}
+
+ def presence_type(self):
+ return 'bit'
+
+ def presence_member(self, space, type_filter):
+ if self.presence_type() != type_filter:
+ return
+
+ if self.presence_type() == 'bit':
+ pfx = '__' if space == 'user' else ''
+ return f"{pfx}u32 {self.c_name}:1;"
+
+ if self.presence_type() == 'len':
+ pfx = '__' if space == 'user' else ''
+ return f"{pfx}u32 {self.c_name}_len;"
+
+ def _complex_member_type(self, ri):
+ return None
+
+ def free_needs_iter(self):
+ return False
+
+ def free(self, ri, var, ref):
+ if self.is_multi_val() or self.presence_type() == 'len':
+ ri.cw.p(f'free({var}->{ref}{self.c_name});')
+
+ def arg_member(self, ri):
+ member = self._complex_member_type(ri)
+ if member:
+ return [member + ' *' + self.c_name]
+ raise Exception(f"Struct member not implemented for class type {self.type}")
+
+ def struct_member(self, ri):
+ if self.is_multi_val():
+ ri.cw.p(f"unsigned int n_{self.c_name};")
+ member = self._complex_member_type(ri)
+ if member:
+ ptr = ''
+ if self. is_multi_val():
+ ptr = '*'
+ ri.cw.p(f"{member} {ptr}{self.c_name};")
+ return
+ members = self.arg_member(ri)
+ for one in members:
+ ri.cw.p(one + ';')
+
+ def _attr_policy(self, policy):
+ return '{ .type = ' + policy + ', }'
+
+ def attr_policy(self, cw):
+ policy = 'NLA_' + self.attr['type'].upper()
+ policy = policy.replace('-', '_')
+
+ spec = self._attr_policy(policy)
+ cw.p(f"\t[{self.enum_name}] = {spec},")
+
+ def _attr_typol(self):
+ raise Exception(f"Type policy not implemented for class type {self.type}")
+
+ def attr_typol(self, cw):
+ typol = self._attr_typol()
+ cw.p(f'[{self.enum_name}] = {"{"} .name = "{self.name}", {typol}{"}"},')
+
+ def _attr_put_line(self, ri, var, line):
+ if self.presence_type() == 'bit':
+ ri.cw.p(f"if ({var}->_present.{self.c_name})")
+ elif self.presence_type() == 'len':
+ ri.cw.p(f"if ({var}->_present.{self.c_name}_len)")
+ ri.cw.p(f"{line};")
+
+ def _attr_put_simple(self, ri, var, put_type):
+ line = f"mnl_attr_put_{put_type}(nlh, {self.enum_name}, {var}->{self.c_name})"
+ self._attr_put_line(ri, var, line)
+
+ def attr_put(self, ri, var):
+ raise Exception(f"Put not implemented for class type {self.type}")
+
+ def _attr_get(self, ri, var):
+ raise Exception(f"Attr get not implemented for class type {self.type}")
+
+ def attr_get(self, ri, var, first):
+ lines, init_lines, local_vars = self._attr_get(ri, var)
+ if type(lines) is str:
+ lines = [lines]
+ if type(init_lines) is str:
+ init_lines = [init_lines]
+
+ kw = 'if' if first else 'else if'
+ ri.cw.block_start(line=f"{kw} (mnl_attr_get_type(attr) == {self.enum_name})")
+ if local_vars:
+ for local in local_vars:
+ ri.cw.p(local)
+ ri.cw.nl()
+
+ if not self.is_multi_val():
+ ri.cw.p(f"if (ynl_attr_validate(yarg, attr))")
+ ri.cw.p(f"return MNL_CB_ERROR;")
+ if self.presence_type() == 'bit':
+ ri.cw.p(f"{var}->_present.{self.c_name} = 1;")
+
+ if init_lines:
+ ri.cw.nl()
+ for line in init_lines:
+ ri.cw.p(line)
+
+ for line in lines:
+ ri.cw.p(line)
+ ri.cw.block_end()
+
+ def _setter_lines(self, ri, member, presence):
+ raise Exception(f"Setter not implemented for class type {self.type}")
+
+ def setter(self, ri, space, direction, deref=False, ref=None):
+ ref = (ref if ref else []) + [self.c_name]
+ var = "req"
+ member = f"{var}->{'.'.join(ref)}"
+
+ code = []
+ presence = ''
+ for i in range(0, len(ref)):
+ presence = f"{var}->{'.'.join(ref[:i] + [''])}_present.{ref[i]}"
+ if self.presence_type() == 'bit':
+ code.append(presence + ' = 1;')
+ code += self._setter_lines(ri, member, presence)
+
+ ri.cw.write_func('static inline void',
+ f"{op_prefix(ri, direction, deref=deref)}_set_{'_'.join(ref)}",
+ body=code,
+ args=[f'{type_name(ri, direction, deref=deref)} *{var}'] + self.arg_member(ri))
+
+
+class TypeUnused(Type):
+ def presence_type(self):
+ return ''
+
+ def _attr_typol(self):
+ return '.type = YNL_PT_REJECT, '
+
+ def attr_policy(self, cw):
+ pass
+
+
+class TypePad(Type):
+ def presence_type(self):
+ return ''
+
+ def _attr_typol(self):
+ return '.type = YNL_PT_REJECT, '
+
+ def attr_policy(self, cw):
+ pass
+
+
+class TypeScalar(Type):
+ def __init__(self, family, attr_set, attr):
+ super().__init__(family, attr_set, attr)
+
+ self.is_bitfield = False
+ if 'enum' in self.attr:
+ self.is_bitfield = family.consts[self.attr['enum']]['type'] == 'flags'
+ if 'enum-as-flags' in self.attr and self.attr['enum-as-flags']:
+ self.is_bitfield = True
+
+ if 'enum' in self.attr and not self.is_bitfield:
+ self.type_name = f"enum {family.name}_{self.attr['enum'].replace('-', '_')}"
+ else:
+ self.type_name = '__' + self.type
+
+ self.byte_order_comment = ''
+ if 'byte-order' in attr:
+ self.byte_order_comment = f" /* {attr['byte-order']} */"
+
+ def _mnl_type(self):
+ t = self.type
+ # mnl does not have a helper for signed types
+ if t[0] == 's':
+ t = 'u' + t[1:]
+ return t
+
+ def _attr_policy(self, policy):
+ if 'flags-mask' in self.checks or self.is_bitfield:
+ if self.is_bitfield:
+ mask = self.family.consts[self.attr['enum']].get_mask()
+ else:
+ flags = self.family.consts[self.checks['flags-mask']]
+ flag_cnt = len(flags['entries'])
+ mask = (1 << flag_cnt) - 1
+ return f"NLA_POLICY_MASK({policy}, 0x{mask:x})"
+ elif 'min' in self.checks:
+ return f"NLA_POLICY_MIN({policy}, {self.checks['min']})"
+ elif 'enum' in self.attr:
+ enum = self.family.consts[self.attr['enum']]
+ cnt = len(enum['entries'])
+ return f"NLA_POLICY_MAX({policy}, {cnt - 1})"
+ return super()._attr_policy(policy)
+
+ def _attr_typol(self):
+ return f'.type = YNL_PT_U{self.type[1:]}, '
+
+ def arg_member(self, ri):
+ return [f'{self.type_name} {self.c_name}{self.byte_order_comment}']
+
+ def attr_put(self, ri, var):
+ self._attr_put_simple(ri, var, self._mnl_type())
+
+ def _attr_get(self, ri, var):
+ return f"{var}->{self.c_name} = mnl_attr_get_{self._mnl_type()}(attr);", None, None
+
+ def _setter_lines(self, ri, member, presence):
+ return [f"{member} = {self.c_name};"]
+
+
+class TypeFlag(Type):
+ def arg_member(self, ri):
+ return []
+
+ def _attr_typol(self):
+ return '.type = YNL_PT_FLAG, '
+
+ def attr_put(self, ri, var):
+ self._attr_put_line(ri, var, f"mnl_attr_put(nlh, {self.enum_name}, 0, NULL)")
+
+ def _attr_get(self, ri, var):
+ return [], None, None
+
+ def _setter_lines(self, ri, member, presence):
+ return []
+
+
+class TypeString(Type):
+ def arg_member(self, ri):
+ return [f"const char *{self.c_name}"]
+
+ def presence_type(self):
+ return 'len'
+
+ def struct_member(self, ri):
+ ri.cw.p(f"char *{self.c_name};")
+
+ def _attr_typol(self):
+ return f'.type = YNL_PT_NUL_STR, '
+
+ def _attr_policy(self, policy):
+ mem = '{ .type = ' + policy
+ if 'max-len' in self.checks:
+ mem += ', .len = ' + str(self.checks['max-len'])
+ mem += ', }'
+ return mem
+
+ def attr_policy(self, cw):
+ if 'unterminated-ok' in self.checks and self.checks['unterminated-ok']:
+ policy = 'NLA_STRING'
+ else:
+ policy = 'NLA_NUL_STRING'
+
+ spec = self._attr_policy(policy)
+ cw.p(f"\t[{self.enum_name}] = {spec},")
+
+ def attr_put(self, ri, var):
+ self._attr_put_simple(ri, var, 'strz')
+
+ def _attr_get(self, ri, var):
+ len_mem = var + '->_present.' + self.c_name + '_len'
+ return [f"{len_mem} = len;",
+ f"{var}->{self.c_name} = malloc(len + 1);",
+ f"memcpy({var}->{self.c_name}, mnl_attr_get_str(attr), len);",
+ f"{var}->{self.c_name}[len] = 0;"], \
+ ['len = strnlen(mnl_attr_get_str(attr), mnl_attr_get_payload_len(attr));'], \
+ ['unsigned int len;']
+
+ def _setter_lines(self, ri, member, presence):
+ return [f"free({member});",
+ f"{presence}_len = strlen({self.c_name});",
+ f"{member} = malloc({presence}_len + 1);",
+ f'memcpy({member}, {self.c_name}, {presence}_len);',
+ f'{member}[{presence}_len] = 0;']
+
+
+class TypeBinary(Type):
+ def arg_member(self, ri):
+ return [f"const void *{self.c_name}", 'size_t len']
+
+ def presence_type(self):
+ return 'len'
+
+ def struct_member(self, ri):
+ ri.cw.p(f"void *{self.c_name};")
+
+ def _attr_typol(self):
+ return f'.type = YNL_PT_BINARY,'
+
+ def _attr_policy(self, policy):
+ mem = '{ '
+ if len(self.checks) == 1 and 'min-len' in self.checks:
+ mem += '.len = ' + str(self.checks['min-len'])
+ elif len(self.checks) == 0:
+ mem += '.type = NLA_BINARY'
+ else:
+ raise Exception('Binary type not implemented, yet')
+ mem += ', }'
+ return mem
+
+ def attr_put(self, ri, var):
+ self._attr_put_line(ri, var, f"mnl_attr_put(nlh, {self.enum_name}, " +
+ f"{var}->_present.{self.c_name}_len, {var}->{self.c_name})")
+
+ def _attr_get(self, ri, var):
+ len_mem = var + '->_present.' + self.c_name + '_len'
+ return [f"{len_mem} = len;",
+ f"{var}->{self.c_name} = malloc(len);",
+ f"memcpy({var}->{self.c_name}, mnl_attr_get_payload(attr), len);"], \
+ ['len = mnl_attr_get_payload_len(attr);'], \
+ ['unsigned int len;']
+
+ def _setter_lines(self, ri, member, presence):
+ return [f"free({member});",
+ f"{member} = malloc({presence}_len);",
+ f'memcpy({member}, {self.c_name}, {presence}_len);']
+
+
+class TypeNest(Type):
+ def _complex_member_type(self, ri):
+ return f"struct {self.nested_render_name}"
+
+ def free(self, ri, var, ref):
+ ri.cw.p(f'{self.nested_render_name}_free(&{var}->{ref}{self.c_name});')
+
+ def _attr_typol(self):
+ return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, '
+
+ def _attr_policy(self, policy):
+ return 'NLA_POLICY_NESTED(' + self.nested_render_name + '_nl_policy)'
+
+ def attr_put(self, ri, var):
+ self._attr_put_line(ri, var, f"{self.nested_render_name}_put(nlh, " +
+ f"{self.enum_name}, &{var}->{self.c_name})")
+
+ def _attr_get(self, ri, var):
+ get_lines = [f"{self.nested_render_name}_parse(&parg, attr);"]
+ init_lines = [f"parg.rsp_policy = &{self.nested_render_name}_nest;",
+ f"parg.data = &{var}->{self.c_name};"]
+ return get_lines, init_lines, None
+
+ def setter(self, ri, space, direction, deref=False, ref=None):
+ ref = (ref if ref else []) + [self.c_name]
+
+ for _, attr in ri.family.pure_nested_structs[self.nested_attrs].member_list():
+ attr.setter(ri, self.nested_attrs, direction, deref=deref, ref=ref)
+
+
+class TypeMultiAttr(Type):
+ def is_multi_val(self):
+ return True
+
+ def presence_type(self):
+ return 'count'
+
+ def _complex_member_type(self, ri):
+ if 'type' not in self.attr or self.attr['type'] == 'nest':
+ return f"struct {self.nested_render_name}"
+ elif self.attr['type'] in scalars:
+ scalar_pfx = '__' if ri.ku_space == 'user' else ''
+ return scalar_pfx + self.attr['type']
+ else:
+ raise Exception(f"Sub-type {self.attr['type']} not supported yet")
+
+ def free_needs_iter(self):
+ return 'type' not in self.attr or self.attr['type'] == 'nest'
+
+ def free(self, ri, var, ref):
+ if 'type' not in self.attr or self.attr['type'] == 'nest':
+ ri.cw.p(f"for (i = 0; i < {var}->{ref}n_{self.c_name}; i++)")
+ ri.cw.p(f'{self.nested_render_name}_free(&{var}->{ref}{self.c_name}[i]);')
+
+ def _attr_typol(self):
+ if 'type' not in self.attr or self.attr['type'] == 'nest':
+ return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, '
+ elif self.attr['type'] in scalars:
+ return f".type = YNL_PT_U{self.attr['type'][1:]}, "
+ else:
+ raise Exception(f"Sub-type {self.attr['type']} not supported yet")
+
+ def _attr_get(self, ri, var):
+ return f'{var}->n_{self.c_name}++;', None, None
+
+
+class TypeArrayNest(Type):
+ def is_multi_val(self):
+ return True
+
+ def presence_type(self):
+ return 'count'
+
+ def _complex_member_type(self, ri):
+ if 'sub-type' not in self.attr or self.attr['sub-type'] == 'nest':
+ return f"struct {self.nested_render_name}"
+ elif self.attr['sub-type'] in scalars:
+ scalar_pfx = '__' if ri.ku_space == 'user' else ''
+ return scalar_pfx + self.attr['sub-type']
+ else:
+ raise Exception(f"Sub-type {self.attr['sub-type']} not supported yet")
+
+ def _attr_typol(self):
+ return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, '
+
+ def _attr_get(self, ri, var):
+ local_vars = ['const struct nlattr *attr2;']
+ get_lines = [f'attr_{self.c_name} = attr;',
+ 'mnl_attr_for_each_nested(attr2, attr)',
+ f'\t{var}->n_{self.c_name}++;']
+ return get_lines, None, local_vars
+
+
+class TypeNestTypeValue(Type):
+ def _complex_member_type(self, ri):
+ return f"struct {self.nested_render_name}"
+
+ def _attr_typol(self):
+ return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, '
+
+ def _attr_get(self, ri, var):
+ prev = 'attr'
+ tv_args = ''
+ get_lines = []
+ local_vars = []
+ init_lines = [f"parg.rsp_policy = &{self.nested_render_name}_nest;",
+ f"parg.data = &{var}->{self.c_name};"]
+ if 'type-value' in self.attr:
+ tv_names = [x.replace('-', '_') for x in self.attr["type-value"]]
+ local_vars += [f'const struct nlattr *attr_{", *attr_".join(tv_names)};']
+ local_vars += [f'__u32 {", ".join(tv_names)};']
+ for level in self.attr["type-value"]:
+ level = level.replace('-', '_')
+ get_lines += [f'attr_{level} = mnl_attr_get_payload({prev});']
+ get_lines += [f'{level} = mnl_attr_get_type(attr_{level});']
+ prev = 'attr_' + level
+
+ tv_args = f", {', '.join(tv_names)}"
+
+ get_lines += [f"{self.nested_render_name}_parse(&parg, {prev}{tv_args});"]
+ return get_lines, init_lines, local_vars
+
+
+class Struct:
+ def __init__(self, family, space_name, type_list=None, inherited=None):
+ self.family = family
+ self.space_name = space_name
+ self.attr_set = family.attr_sets[space_name]
+ # Use list to catch comparisons with empty sets
+ self._inherited = inherited if inherited is not None else []
+ self.inherited = []
+
+ self.nested = type_list is None
+ if family.name == space_name.replace('-', '_'):
+ self.render_name = f"{family.name}"
+ else:
+ self.render_name = f"{family.name}_{space_name.replace('-', '_')}"
+ self.struct_name = 'struct ' + self.render_name
+ self.ptr_name = self.struct_name + ' *'
+
+ self.request = False
+ self.reply = False
+
+ self.attr_list = []
+ self.attrs = dict()
+ if type_list:
+ for t in type_list:
+ self.attr_list.append((t, self.attr_set[t]),)
+ else:
+ for t in self.attr_set:
+ self.attr_list.append((t, self.attr_set[t]),)
+
+ max_val = 0
+ self.attr_max_val = None
+ for name, attr in self.attr_list:
+ if attr.value > max_val:
+ max_val = attr.value
+ self.attr_max_val = attr
+ self.attrs[name] = attr
+
+ def __iter__(self):
+ yield from self.attrs
+
+ def __getitem__(self, key):
+ return self.attrs[key]
+
+ def member_list(self):
+ return self.attr_list
+
+ def set_inherited(self, new_inherited):
+ if self._inherited != new_inherited:
+ raise Exception("Inheriting different members not supported")
+ self.inherited = [x.replace('-', '_') for x in sorted(self._inherited)]
+
+
+class EnumEntry:
+ def __init__(self, enum_set, yaml, prev, value_start):
+ if isinstance(yaml, str):
+ self.name = yaml
+ yaml = {}
+ self.doc = ''
+ else:
+ self.name = yaml['name']
+ self.doc = yaml.get('doc', '')
+
+ self.yaml = yaml
+ self.c_name = c_upper(enum_set.value_pfx + self.name)
+
+ if 'value' in yaml:
+ self.value = yaml['value']
+ if prev:
+ self.value_change = (self.value != prev.value + 1)
+ elif prev:
+ self.value = prev.value + 1
+ else:
+ self.value = value_start
+ self.value_change = (self.value != 0)
+
+ def __getitem__(self, key):
+ return self.yaml[key]
+
+ def __contains__(self, key):
+ return key in self.yaml
+
+ def has_doc(self):
+ return bool(self.doc)
+
+
+class EnumSet:
+ def __init__(self, family, yaml):
+ self.yaml = yaml
+ self.family = family
+
+ self.render_name = c_lower(family.name + '-' + yaml['name'])
+ self.enum_name = 'enum ' + self.render_name
+
+ self.value_pfx = yaml.get('name-prefix', f"{family.name}-{yaml['name']}-")
+
+ self.type = yaml['type']
+
+ prev_entry = None
+ value_start = self.yaml.get('value-start', 0)
+ self.entries = {}
+ self.entry_list = []
+ for entry in self.yaml['entries']:
+ e = EnumEntry(self, entry, prev_entry, value_start)
+ self.entries[e.name] = e
+ self.entry_list.append(e)
+ prev_entry = e
+
+ def __getitem__(self, key):
+ return self.yaml[key]
+
+ def __contains__(self, key):
+ return key in self.yaml
+
+ def has_doc(self):
+ if 'doc' in self.yaml:
+ return True
+ for entry in self.entry_list:
+ if entry.has_doc():
+ return True
+ return False
+
+ def get_mask(self):
+ mask = 0
+ idx = self.yaml.get('value-start', 0)
+ for _ in self.entry_list:
+ mask |= 1 << idx
+ idx += 1
+ return mask
+
+
+class AttrSet:
+ def __init__(self, family, yaml):
+ self.yaml = yaml
+
+ self.attrs = dict()
+ self.name = self.yaml['name']
+ if 'subset-of' not in yaml:
+ self.subset_of = None
+ if 'name-prefix' in yaml:
+ pfx = yaml['name-prefix']
+ elif self.name == family.name:
+ pfx = family.name + '-a-'
+ else:
+ pfx = f"{family.name}-a-{self.name}-"
+ self.name_prefix = c_upper(pfx)
+ self.max_name = c_upper(self.yaml.get('attr-max-name', f"{self.name_prefix}max"))
+ else:
+ self.subset_of = self.yaml['subset-of']
+ self.name_prefix = family.attr_sets[self.subset_of].name_prefix
+ self.max_name = family.attr_sets[self.subset_of].max_name
+
+ self.c_name = self.name.replace('-', '_')
+ if self.c_name in c_kw:
+ self.c_name += '_'
+ if self.c_name == family.c_name:
+ self.c_name = ''
+
+ val = 0
+ for elem in self.yaml['attributes']:
+ if 'value' in elem:
+ val = elem['value']
+ else:
+ elem['value'] = val
+ val += 1
+
+ if 'multi-attr' in elem and elem['multi-attr']:
+ attr = TypeMultiAttr(family, self, elem)
+ elif elem['type'] in scalars:
+ attr = TypeScalar(family, self, elem)
+ elif elem['type'] == 'unused':
+ attr = TypeUnused(family, self, elem)
+ elif elem['type'] == 'pad':
+ attr = TypePad(family, self, elem)
+ elif elem['type'] == 'flag':
+ attr = TypeFlag(family, self, elem)
+ elif elem['type'] == 'string':
+ attr = TypeString(family, self, elem)
+ elif elem['type'] == 'binary':
+ attr = TypeBinary(family, self, elem)
+ elif elem['type'] == 'nest':
+ attr = TypeNest(family, self, elem)
+ elif elem['type'] == 'array-nest':
+ attr = TypeArrayNest(family, self, elem)
+ elif elem['type'] == 'nest-type-value':
+ attr = TypeNestTypeValue(family, self, elem)
+ else:
+ raise Exception(f"No typed class for type {elem['type']}")
+
+ self.attrs[elem['name']] = attr
+
+ def __getitem__(self, key):
+ return self.attrs[key]
+
+ def __contains__(self, key):
+ return key in self.yaml
+
+ def __iter__(self):
+ yield from self.attrs
+
+ def items(self):
+ return self.attrs.items()
+
+
+class Operation:
+ def __init__(self, family, yaml, value):
+ self.yaml = yaml
+ self.value = value
+
+ self.name = self.yaml['name']
+ self.render_name = family.name + '_' + self.name.replace('-', '_')
+ self.is_async = 'notify' in yaml or 'event' in yaml
+ if not self.is_async:
+ self.enum_name = family.op_prefix + self.name.upper().replace('-', '_')
+ else:
+ self.enum_name = family.async_op_prefix + self.name.upper().replace('-', '_')
+
+ self.dual_policy = ('do' in yaml and 'request' in yaml['do']) and \
+ ('dump' in yaml and 'request' in yaml['dump'])
+
+ def __getitem__(self, key):
+ return self.yaml[key]
+
+ def __contains__(self, key):
+ return key in self.yaml
+
+ def add_notification(self, op):
+ if 'notify' not in self.yaml:
+ self.yaml['notify'] = dict()
+ self.yaml['notify']['reply'] = self.yaml['do']['reply']
+ self.yaml['notify']['cmds'] = []
+ self.yaml['notify']['cmds'].append(op)
+
+
+class Family:
+ def __init__(self, file_name):
+ with open(file_name, "r") as stream:
+ self.yaml = yaml.safe_load(stream)
+
+ self.proto = self.yaml.get('protocol', 'genetlink')
+
+ with open(os.path.dirname(os.path.dirname(file_name)) +
+ f'/{self.proto}.yaml', "r") as stream:
+ schema = yaml.safe_load(stream)
+
+ jsonschema.validate(self.yaml, schema)
+
+ if self.yaml.get('protocol', 'genetlink') not in {'genetlink', 'genetlink-c', 'genetlink-legacy'}:
+ raise Exception("Codegen only supported for genetlink")
+
+ self.fam_key = c_upper(self.yaml.get('c-family-name', self.yaml["name"] + '_FAMILY_NAME'))
+ self.ver_key = c_upper(self.yaml.get('c-version-name', self.yaml["name"] + '_FAMILY_VERSION'))
+
+ if 'definitions' not in self.yaml:
+ self.yaml['definitions'] = []
+
+ self.name = self.yaml['name']
+ self.c_name = self.name.replace('-', '_')
+ if 'uapi-header' in self.yaml:
+ self.uapi_header = self.yaml['uapi-header']
+ else:
+ self.uapi_header = f"linux/{self.name}.h"
+ if 'name-prefix' in self.yaml['operations']:
+ self.op_prefix = self.yaml['operations']['name-prefix'].upper().replace('-', '_')
+ else:
+ self.op_prefix = (self.yaml['name'] + '-cmd-').upper().replace('-', '_')
+ if 'async-prefix' in self.yaml['operations']:
+ self.async_op_prefix = self.yaml['operations']['async-prefix'].upper().replace('-', '_')
+ else:
+ self.async_op_prefix = self.op_prefix
+
+ self.mcgrps = self.yaml.get('mcast-groups', {'list': []})
+
+ self.consts = dict()
+ self.ops = dict()
+ self.ops_list = []
+ self.attr_sets = dict()
+ self.attr_sets_list = []
+
+ self.hooks = dict()
+ for when in ['pre', 'post']:
+ self.hooks[when] = dict()
+ for op_mode in ['do', 'dump']:
+ self.hooks[when][op_mode] = dict()
+ self.hooks[when][op_mode]['set'] = set()
+ self.hooks[when][op_mode]['list'] = []
+
+ # dict space-name -> 'request': set(attrs), 'reply': set(attrs)
+ self.root_sets = dict()
+ # dict space-name -> set('request', 'reply')
+ self.pure_nested_structs = dict()
+ self.all_notify = dict()
+
+ self._mock_up_events()
+
+ self._dictify()
+ self._load_root_sets()
+ self._load_nested_sets()
+ self._load_all_notify()
+ self._load_hooks()
+
+ self.kernel_policy = self.yaml.get('kernel-policy', 'split')
+ if self.kernel_policy == 'global':
+ self._load_global_policy()
+
+ def __getitem__(self, key):
+ return self.yaml[key]
+
+ def get(self, key, default=None):
+ return self.yaml.get(key, default)
+
+ # Fake a 'do' equivalent of all events, so that we can render their response parsing
+ def _mock_up_events(self):
+ for op in self.yaml['operations']['list']:
+ if 'event' in op:
+ op['do'] = {
+ 'reply': {
+ 'attributes': op['event']['attributes']
+ }
+ }
+
+ def _dictify(self):
+ for elem in self.yaml['definitions']:
+ if elem['type'] == 'enum':
+ self.consts[elem['name']] = EnumSet(self, elem)
+ else:
+ self.consts[elem['name']] = elem
+
+ for elem in self.yaml['attribute-sets']:
+ attr_set = AttrSet(self, elem)
+ self.attr_sets[elem['name']] = attr_set
+ self.attr_sets_list.append((elem['name'], attr_set), )
+
+ ntf = []
+ val = 0
+ for elem in self.yaml['operations']['list']:
+ if 'value' in elem:
+ val = elem['value']
+
+ op = Operation(self, elem, val)
+ val += 1
+
+ self.ops_list.append((elem['name'], op),)
+ if 'notify' in elem:
+ ntf.append(op)
+ continue
+ if 'attribute-set' not in elem:
+ continue
+ self.ops[elem['name']] = op
+ for n in ntf:
+ self.ops[n['notify']].add_notification(n)
+
+ def _load_root_sets(self):
+ for op_name, op in self.ops.items():
+ if 'attribute-set' not in op:
+ continue
+
+ req_attrs = set()
+ rsp_attrs = set()
+ for op_mode in ['do', 'dump']:
+ if op_mode in op and 'request' in op[op_mode]:
+ req_attrs.update(set(op[op_mode]['request']['attributes']))
+ if op_mode in op and 'reply' in op[op_mode]:
+ rsp_attrs.update(set(op[op_mode]['reply']['attributes']))
+
+ if op['attribute-set'] not in self.root_sets:
+ self.root_sets[op['attribute-set']] = {'request': req_attrs, 'reply': rsp_attrs}
+ else:
+ self.root_sets[op['attribute-set']]['request'].update(req_attrs)
+ self.root_sets[op['attribute-set']]['reply'].update(rsp_attrs)
+
+ def _load_nested_sets(self):
+ for root_set, rs_members in self.root_sets.items():
+ for attr, spec in self.attr_sets[root_set].items():
+ if 'nested-attributes' in spec:
+ inherit = set()
+ nested = spec['nested-attributes']
+ if nested not in self.root_sets:
+ self.pure_nested_structs[nested] = Struct(self, nested, inherited=inherit)
+ if attr in rs_members['request']:
+ self.pure_nested_structs[nested].request = True
+ if attr in rs_members['reply']:
+ self.pure_nested_structs[nested].reply = True
+
+ if 'type-value' in spec:
+ if nested in self.root_sets:
+ raise Exception("Inheriting members to a space used as root not supported")
+ inherit.update(set(spec['type-value']))
+ elif spec['type'] == 'array-nest':
+ inherit.add('idx')
+ self.pure_nested_structs[nested].set_inherited(inherit)
+
+ def _load_all_notify(self):
+ for op_name, op in self.ops.items():
+ if not op:
+ continue
+
+ if 'notify' in op:
+ self.all_notify[op_name] = op['notify']['cmds']
+
+ def _load_global_policy(self):
+ global_set = set()
+ attr_set_name = None
+ for op_name, op in self.ops.items():
+ if not op:
+ continue
+ if 'attribute-set' not in op:
+ continue
+
+ if attr_set_name is None:
+ attr_set_name = op['attribute-set']
+ if attr_set_name != op['attribute-set']:
+ raise Exception('For a global policy all ops must use the same set')
+
+ for op_mode in {'do', 'dump'}:
+ if op_mode in op:
+ global_set.update(op[op_mode].get('request', []))
+
+ self.global_policy = []
+ self.global_policy_set = attr_set_name
+ for attr in self.attr_sets[attr_set_name]:
+ if attr in global_set:
+ self.global_policy.append(attr)
+
+ def _load_hooks(self):
+ for op in self.ops.values():
+ for op_mode in ['do', 'dump']:
+ if op_mode not in op:
+ continue
+ for when in ['pre', 'post']:
+ if when not in op[op_mode]:
+ continue
+ name = op[op_mode][when]
+ if name in self.hooks[when][op_mode]['set']:
+ continue
+ self.hooks[when][op_mode]['set'].add(name)
+ self.hooks[when][op_mode]['list'].append(name)
+
+
+class RenderInfo:
+ def __init__(self, cw, family, ku_space, op, op_name, op_mode, attr_set=None):
+ self.family = family
+ self.nl = cw.nlib
+ self.ku_space = ku_space
+ self.op = op
+ self.op_name = op_name
+ self.op_mode = op_mode
+
+ # 'do' and 'dump' response parsing is identical
+ if op_mode != 'do' and 'dump' in op and 'do' in op and 'reply' in op['do'] and \
+ op["do"]["reply"] == op["dump"]["reply"]:
+ self.type_consistent = True
+ else:
+ self.type_consistent = op_mode == 'event'
+
+ self.attr_set = attr_set
+ if not self.attr_set:
+ self.attr_set = op['attribute-set']
+
+ if op:
+ self.type_name = op_name.replace('-', '_')
+ else:
+ self.type_name = attr_set.replace('-', '_')
+
+ self.cw = cw
+
+ self.struct = dict()
+ for op_dir in ['request', 'reply']:
+ if op and op_dir in op[op_mode]:
+ self.struct[op_dir] = Struct(family, self.attr_set,
+ type_list=op[op_mode][op_dir]['attributes'])
+ if op_mode == 'event':
+ self.struct['reply'] = Struct(family, self.attr_set, type_list=op['event']['attributes'])
+
+
+class CodeWriter:
+ def __init__(self, nlib, out_file):
+ self.nlib = nlib
+
+ self._nl = False
+ self._silent_block = False
+ self._ind = 0
+ self._out = out_file
+
+ @classmethod
+ def _is_cond(cls, line):
+ return line.startswith('if') or line.startswith('while') or line.startswith('for')
+
+ def p(self, line, add_ind=0):
+ if self._nl:
+ self._out.write('\n')
+ self._nl = False
+ ind = self._ind
+ if line[-1] == ':':
+ ind -= 1
+ if self._silent_block:
+ ind += 1
+ self._silent_block = line.endswith(')') and CodeWriter._is_cond(line)
+ if add_ind:
+ ind += add_ind
+ self._out.write('\t' * ind + line + '\n')
+
+ def nl(self):
+ self._nl = True
+
+ def block_start(self, line=''):
+ if line:
+ line = line + ' '
+ self.p(line + '{')
+ self._ind += 1
+
+ def block_end(self, line=''):
+ if line and line[0] not in {';', ','}:
+ line = ' ' + line
+ self._ind -= 1
+ self.p('}' + line)
+
+ def write_doc_line(self, doc, indent=True):
+ words = doc.split()
+ line = ' *'
+ for word in words:
+ if len(line) + len(word) >= 79:
+ self.p(line)
+ line = ' *'
+ if indent:
+ line += ' '
+ line += ' ' + word
+ self.p(line)
+
+ def write_func_prot(self, qual_ret, name, args=None, doc=None, suffix=''):
+ if not args:
+ args = ['void']
+
+ if doc:
+ self.p('/*')
+ self.p(' * ' + doc)
+ self.p(' */')
+
+ oneline = qual_ret
+ if qual_ret[-1] != '*':
+ oneline += ' '
+ oneline += f"{name}({', '.join(args)}){suffix}"
+
+ if len(oneline) < 80:
+ self.p(oneline)
+ return
+
+ v = qual_ret
+ if len(v) > 3:
+ self.p(v)
+ v = ''
+ elif qual_ret[-1] != '*':
+ v += ' '
+ v += name + '('
+ ind = '\t' * (len(v) // 8) + ' ' * (len(v) % 8)
+ delta_ind = len(v) - len(ind)
+ v += args[0]
+ i = 1
+ while i < len(args):
+ next_len = len(v) + len(args[i])
+ if v[0] == '\t':
+ next_len += delta_ind
+ if next_len > 76:
+ self.p(v + ',')
+ v = ind
+ else:
+ v += ', '
+ v += args[i]
+ i += 1
+ self.p(v + ')' + suffix)
+
+ def write_func_lvar(self, local_vars):
+ if not local_vars:
+ return
+
+ if type(local_vars) is str:
+ local_vars = [local_vars]
+
+ local_vars.sort(key=len, reverse=True)
+ for var in local_vars:
+ self.p(var)
+ self.nl()
+
+ def write_func(self, qual_ret, name, body, args=None, local_vars=None):
+ self.write_func_prot(qual_ret=qual_ret, name=name, args=args)
+ self.write_func_lvar(local_vars=local_vars)
+
+ self.block_start()
+ for line in body:
+ self.p(line)
+ self.block_end()
+
+ def writes_defines(self, defines):
+ longest = 0
+ for define in defines:
+ if len(define[0]) > longest:
+ longest = len(define[0])
+ longest = ((longest + 8) // 8) * 8
+ for define in defines:
+ line = '#define ' + define[0]
+ line += '\t' * ((longest - len(define[0]) + 7) // 8)
+ if type(define[1]) is int:
+ line += str(define[1])
+ elif type(define[1]) is str:
+ line += '"' + define[1] + '"'
+ self.p(line)
+
+ def write_struct_init(self, members):
+ longest = max([len(x[0]) for x in members])
+ longest += 1 # because we prepend a .
+ longest = ((longest + 8) // 8) * 8
+ for one in members:
+ line = '.' + one[0]
+ line += '\t' * ((longest - len(one[0]) - 1 + 7) // 8)
+ line += '= ' + one[1] + ','
+ self.p(line)
+
+
+scalars = {'u8', 'u16', 'u32', 'u64', 's32', 's64'}
+
+direction_to_suffix = {
+ 'reply': '_rsp',
+ 'request': '_req',
+ '': ''
+}
+
+op_mode_to_wrapper = {
+ 'do': '',
+ 'dump': '_list',
+ 'notify': '_ntf',
+ 'event': '',
+}
+
+c_kw = {
+ 'do'
+}
+
+
+def rdir(direction):
+ if direction == 'reply':
+ return 'request'
+ if direction == 'request':
+ return 'reply'
+ return direction
+
+
+def op_prefix(ri, direction, deref=False):
+ suffix = f"_{ri.type_name}"
+
+ if not ri.op_mode or ri.op_mode == 'do':
+ suffix += f"{direction_to_suffix[direction]}"
+ else:
+ if direction == 'request':
+ suffix += '_req_dump'
+ else:
+ if ri.type_consistent:
+ if deref:
+ suffix += f"{direction_to_suffix[direction]}"
+ else:
+ suffix += op_mode_to_wrapper[ri.op_mode]
+ else:
+ suffix += '_rsp'
+ suffix += '_dump' if deref else '_list'
+
+ return f"{ri.family['name']}{suffix}"
+
+
+def type_name(ri, direction, deref=False):
+ return f"struct {op_prefix(ri, direction, deref=deref)}"
+
+
+def print_prototype(ri, direction, terminate=True, doc=None):
+ suffix = ';' if terminate else ''
+
+ fname = ri.op.render_name
+ if ri.op_mode == 'dump':
+ fname += '_dump'
+
+ args = ['struct ynl_sock *ys']
+ if 'request' in ri.op[ri.op_mode]:
+ args.append(f"{type_name(ri, direction)} *" + f"{direction_to_suffix[direction][1:]}")
+
+ ret = 'int'
+ if 'reply' in ri.op[ri.op_mode]:
+ ret = f"{type_name(ri, rdir(direction))} *"
+
+ ri.cw.write_func_prot(ret, fname, args, doc=doc, suffix=suffix)
+
+
+def print_req_prototype(ri):
+ print_prototype(ri, "request", doc=ri.op['doc'])
+
+
+def print_dump_prototype(ri):
+ print_prototype(ri, "request")
+
+
+def put_typol_fwd(cw, struct):
+ cw.p(f'extern struct ynl_policy_nest {struct.render_name}_nest;')
+
+
+def put_typol(cw, struct):
+ type_max = struct.attr_set.max_name
+ cw.block_start(line=f'struct ynl_policy_attr {struct.render_name}_policy[{type_max} + 1] =')
+
+ for _, arg in struct.member_list():
+ arg.attr_typol(cw)
+
+ cw.block_end(line=';')
+ cw.nl()
+
+ cw.block_start(line=f'struct ynl_policy_nest {struct.render_name}_nest =')
+ cw.p(f'.max_attr = {type_max},')
+ cw.p(f'.table = {struct.render_name}_policy,')
+ cw.block_end(line=';')
+ cw.nl()
+
+
+def put_req_nested(ri, struct):
+ func_args = ['struct nlmsghdr *nlh',
+ 'unsigned int attr_type',
+ f'{struct.ptr_name}obj']
+
+ ri.cw.write_func_prot('int', f'{struct.render_name}_put', func_args)
+ ri.cw.block_start()
+ ri.cw.write_func_lvar('struct nlattr *nest;')
+
+ ri.cw.p("nest = mnl_attr_nest_start(nlh, attr_type);")
+
+ for _, arg in struct.member_list():
+ arg.attr_put(ri, "obj")
+
+ ri.cw.p("mnl_attr_nest_end(nlh, nest);")
+
+ ri.cw.nl()
+ ri.cw.p('return 0;')
+ ri.cw.block_end()
+ ri.cw.nl()
+
+
+def _multi_parse(ri, struct, init_lines, local_vars):
+ if struct.nested:
+ iter_line = "mnl_attr_for_each_nested(attr, nested)"
+ else:
+ iter_line = "mnl_attr_for_each(attr, nlh, sizeof(struct genlmsghdr))"
+
+ array_nests = set()
+ multi_attrs = set()
+ needs_parg = False
+ for arg, aspec in struct.member_list():
+ if aspec['type'] == 'array-nest':
+ local_vars.append(f'const struct nlattr *attr_{aspec.c_name};')
+ array_nests.add(arg)
+ if 'multi-attr' in aspec:
+ multi_attrs.add(arg)
+ needs_parg |= 'nested-attributes' in aspec
+ if array_nests or multi_attrs:
+ local_vars.append('int i;')
+ if needs_parg:
+ local_vars.append('struct ynl_parse_arg parg;')
+ init_lines.append('parg.ys = yarg->ys;')
+
+ ri.cw.block_start()
+ ri.cw.write_func_lvar(local_vars)
+
+ for line in init_lines:
+ ri.cw.p(line)
+ ri.cw.nl()
+
+ for arg in struct.inherited:
+ ri.cw.p(f'dst->{arg} = {arg};')
+
+ ri.cw.nl()
+ ri.cw.block_start(line=iter_line)
+
+ first = True
+ for _, arg in struct.member_list():
+ arg.attr_get(ri, 'dst', first=first)
+ first = False
+
+ ri.cw.block_end()
+ ri.cw.nl()
+
+ for anest in sorted(array_nests):
+ aspec = struct[anest]
+
+ ri.cw.block_start(line=f"if (dst->n_{aspec.c_name})")
+ ri.cw.p(f"dst->{aspec.c_name} = calloc(dst->n_{aspec.c_name}, sizeof(*dst->{aspec.c_name}));")
+ ri.cw.p('i = 0;')
+ ri.cw.p(f"parg.rsp_policy = &{aspec.nested_render_name}_nest;")
+ ri.cw.block_start(line=f"mnl_attr_for_each_nested(attr, attr_{aspec.c_name})")
+ ri.cw.p(f"parg.data = &dst->{aspec.c_name}[i];")
+ ri.cw.p(f"if ({aspec.nested_render_name}_parse(&parg, attr, mnl_attr_get_type(attr)))")
+ ri.cw.p('return MNL_CB_ERROR;')
+ ri.cw.p('i++;')
+ ri.cw.block_end()
+ ri.cw.block_end()
+ ri.cw.nl()
+
+ for anest in sorted(multi_attrs):
+ aspec = struct[anest]
+ ri.cw.block_start(line=f"if (dst->n_{aspec.c_name})")
+ ri.cw.p(f"dst->{aspec.c_name} = calloc(dst->n_{aspec.c_name}, sizeof(*dst->{aspec.c_name}));")
+ ri.cw.p('i = 0;')
+ if 'nested-attributes' in aspec:
+ ri.cw.p(f"parg.rsp_policy = &{aspec.nested_render_name}_nest;")
+ ri.cw.block_start(line=iter_line)
+ ri.cw.block_start(line=f"if (mnl_attr_get_type(attr) == {aspec.enum_name})")
+ if 'nested-attributes' in aspec:
+ ri.cw.p(f"parg.data = &dst->{aspec.c_name}[i];")
+ ri.cw.p(f"if ({aspec.nested_render_name}_parse(&parg, attr))")
+ ri.cw.p('return MNL_CB_ERROR;')
+ elif aspec['type'] in scalars:
+ t = aspec['type']
+ if t[0] == 's':
+ t = 'u' + t[1:]
+ ri.cw.p(f"dst->{aspec.c_name}[i] = mnl_attr_get_{t}(attr);")
+ else:
+ raise Exception('Nest parsing type not supported yet')
+ ri.cw.p('i++;')
+ ri.cw.block_end()
+ ri.cw.block_end()
+ ri.cw.block_end()
+ ri.cw.nl()
+
+ if struct.nested:
+ ri.cw.p('return 0;')
+ else:
+ ri.cw.p('return MNL_CB_OK;')
+ ri.cw.block_end()
+ ri.cw.nl()
+
+
+def parse_rsp_nested(ri, struct):
+ func_args = ['struct ynl_parse_arg *yarg',
+ 'const struct nlattr *nested']
+ for arg in struct.inherited:
+ func_args.append('__u32 ' + arg)
+
+ local_vars = ['const struct nlattr *attr;',
+ f'{struct.ptr_name}dst = yarg->data;']
+ init_lines = []
+
+ ri.cw.write_func_prot('int', f'{struct.render_name}_parse', func_args)
+
+ _multi_parse(ri, struct, init_lines, local_vars)
+
+
+def parse_rsp_msg(ri, deref=False):
+ if 'reply' not in ri.op[ri.op_mode] and ri.op_mode != 'event':
+ return
+
+ func_args = ['const struct nlmsghdr *nlh',
+ 'void *data']
+
+ local_vars = [f'{type_name(ri, "reply", deref=deref)} *dst;',
+ 'struct ynl_parse_arg *yarg = data;',
+ 'const struct nlattr *attr;']
+ init_lines = ['dst = yarg->data;']
+
+ ri.cw.write_func_prot('int', f'{op_prefix(ri, "reply", deref=deref)}_parse', func_args)
+
+ _multi_parse(ri, ri.struct["reply"], init_lines, local_vars)
+
+
+def print_req(ri):
+ ret_ok = '0'
+ ret_err = '-1'
+ direction = "request"
+ local_vars = ['struct nlmsghdr *nlh;',
+ 'int len, err;']
+
+ if 'reply' in ri.op[ri.op_mode]:
+ ret_ok = 'rsp'
+ ret_err = 'NULL'
+ local_vars += [f'{type_name(ri, rdir(direction))} *rsp;',
+ 'struct ynl_parse_arg yarg = { .ys = ys, };']
+
+ print_prototype(ri, direction, terminate=False)
+ ri.cw.block_start()
+ ri.cw.write_func_lvar(local_vars)
+
+ ri.cw.p(f"nlh = ynl_gemsg_start_req(ys, {ri.nl.get_family_id()}, {ri.op.enum_name}, 1);")
+
+ ri.cw.p(f"ys->req_policy = &{ri.struct['request'].render_name}_nest;")
+ if 'reply' in ri.op[ri.op_mode]:
+ ri.cw.p(f"yarg.rsp_policy = &{ri.struct['reply'].render_name}_nest;")
+ ri.cw.nl()
+ for _, attr in ri.struct["request"].member_list():
+ attr.attr_put(ri, "req")
+ ri.cw.nl()
+
+ ri.cw.p('err = mnl_socket_sendto(ys->sock, nlh, nlh->nlmsg_len);')
+ ri.cw.p('if (err < 0)')
+ ri.cw.p(f"return {ret_err};")
+ ri.cw.nl()
+ ri.cw.p('len = mnl_socket_recvfrom(ys->sock, ys->rx_buf, MNL_SOCKET_BUFFER_SIZE);')
+ ri.cw.p('if (len < 0)')
+ ri.cw.p(f"return {ret_err};")
+ ri.cw.nl()
+
+ if 'reply' in ri.op[ri.op_mode]:
+ ri.cw.p('rsp = calloc(1, sizeof(*rsp));')
+ ri.cw.p('yarg.data = rsp;')
+ ri.cw.nl()
+ ri.cw.p(f"err = {ri.nl.parse_cb_run(op_prefix(ri, 'reply') + '_parse', '&yarg', False)};")
+ ri.cw.p('if (err < 0)')
+ ri.cw.p('goto err_free;')
+ ri.cw.nl()
+
+ ri.cw.p('err = ynl_recv_ack(ys, err);')
+ ri.cw.p('if (err)')
+ ri.cw.p('goto err_free;')
+ ri.cw.nl()
+ ri.cw.p(f"return {ret_ok};")
+ ri.cw.nl()
+ ri.cw.p('err_free:')
+
+ if 'reply' in ri.op[ri.op_mode]:
+ ri.cw.p(f"{call_free(ri, rdir(direction), 'rsp')}")
+ ri.cw.p(f"return {ret_err};")
+ ri.cw.block_end()
+
+
+def print_dump(ri):
+ direction = "request"
+ print_prototype(ri, direction, terminate=False)
+ ri.cw.block_start()
+ local_vars = ['struct ynl_dump_state yds = {};',
+ 'struct nlmsghdr *nlh;',
+ 'int len, err;']
+
+ for var in local_vars:
+ ri.cw.p(f'{var}')
+ ri.cw.nl()
+
+ ri.cw.p('yds.ys = ys;')
+ ri.cw.p(f"yds.alloc_sz = sizeof({type_name(ri, rdir(direction))});")
+ ri.cw.p(f"yds.cb = {op_prefix(ri, 'reply', deref=True)}_parse;")
+ ri.cw.p(f"yds.rsp_policy = &{ri.struct['reply'].render_name}_nest;")
+ ri.cw.nl()
+ ri.cw.p(f"nlh = ynl_gemsg_start_dump(ys, {ri.nl.get_family_id()}, {ri.op.enum_name}, 1);")
+
+ if "request" in ri.op[ri.op_mode]:
+ ri.cw.p(f"ys->req_policy = &{ri.struct['request'].render_name}_nest;")
+ ri.cw.nl()
+ for _, attr in ri.struct["request"].member_list():
+ attr.attr_put(ri, "req")
+ ri.cw.nl()
+
+ ri.cw.p('err = mnl_socket_sendto(ys->sock, nlh, nlh->nlmsg_len);')
+ ri.cw.p('if (err < 0)')
+ ri.cw.p('return NULL;')
+ ri.cw.nl()
+
+ ri.cw.block_start(line='do')
+ ri.cw.p('len = mnl_socket_recvfrom(ys->sock, ys->rx_buf, MNL_SOCKET_BUFFER_SIZE);')
+ ri.cw.p('if (len < 0)')
+ ri.cw.p('goto free_list;')
+ ri.cw.nl()
+ ri.cw.p(f"err = {ri.nl.parse_cb_run('ynl_dump_trampoline', '&yds', False, indent=2)};")
+ ri.cw.p('if (err < 0)')
+ ri.cw.p('goto free_list;')
+ ri.cw.block_end(line='while (err > 0);')
+ ri.cw.nl()
+
+ ri.cw.p('return yds.first;')
+ ri.cw.nl()
+ ri.cw.p('free_list:')
+ ri.cw.p(call_free(ri, rdir(direction), 'yds.first'))
+ ri.cw.p('return NULL;')
+ ri.cw.block_end()
+
+
+def call_free(ri, direction, var):
+ return f"{op_prefix(ri, direction)}_free({var});"
+
+
+def free_arg_name(direction):
+ if direction:
+ return direction_to_suffix[direction][1:]
+ return 'obj'
+
+
+def print_free_prototype(ri, direction, suffix=';'):
+ name = op_prefix(ri, direction)
+ arg = free_arg_name(direction)
+ ri.cw.write_func_prot('void', f"{name}_free", [f"struct {name} *{arg}"], suffix=suffix)
+
+
+def _print_type(ri, direction, struct):
+ suffix = f'_{ri.type_name}{direction_to_suffix[direction]}'
+
+ if ri.op_mode == 'dump':
+ suffix += '_dump'
+
+ ri.cw.block_start(line=f"struct {ri.family['name']}{suffix}")
+
+ meta_started = False
+ for _, attr in struct.member_list():
+ for type_filter in ['len', 'bit']:
+ line = attr.presence_member(ri.ku_space, type_filter)
+ if line:
+ if not meta_started:
+ ri.cw.block_start(line=f"struct")
+ meta_started = True
+ ri.cw.p(line)
+ if meta_started:
+ ri.cw.block_end(line='_present;')
+ ri.cw.nl()
+
+ for arg in struct.inherited:
+ ri.cw.p(f"__u32 {arg};")
+
+ for _, attr in struct.member_list():
+ attr.struct_member(ri)
+
+ ri.cw.block_end(line=';')
+ ri.cw.nl()
+
+
+def print_type(ri, direction):
+ _print_type(ri, direction, ri.struct[direction])
+
+
+def print_type_full(ri, struct):
+ _print_type(ri, "", struct)
+
+
+def print_type_helpers(ri, direction, deref=False):
+ print_free_prototype(ri, direction)
+
+ if ri.ku_space == 'user' and direction == 'request':
+ for _, attr in ri.struct[direction].member_list():
+ attr.setter(ri, ri.attr_set, direction, deref=deref)
+ ri.cw.nl()
+
+
+def print_req_type_helpers(ri):
+ print_type_helpers(ri, "request")
+
+
+def print_rsp_type_helpers(ri):
+ if 'reply' not in ri.op[ri.op_mode]:
+ return
+ print_type_helpers(ri, "reply")
+
+
+def print_parse_prototype(ri, direction, terminate=True):
+ suffix = "_rsp" if direction == "reply" else "_req"
+ term = ';' if terminate else ''
+
+ ri.cw.write_func_prot('void', f"{ri.op.render_name}{suffix}_parse",
+ ['const struct nlattr **tb',
+ f"struct {ri.op.render_name}{suffix} *req"],
+ suffix=term)
+
+
+def print_req_type(ri):
+ print_type(ri, "request")
+
+
+def print_rsp_type(ri):
+ if (ri.op_mode == 'do' or ri.op_mode == 'dump') and 'reply' in ri.op[ri.op_mode]:
+ direction = 'reply'
+ elif ri.op_mode == 'event':
+ direction = 'reply'
+ else:
+ return
+ print_type(ri, direction)
+
+
+def print_wrapped_type(ri):
+ ri.cw.block_start(line=f"{type_name(ri, 'reply')}")
+ if ri.op_mode == 'dump':
+ ri.cw.p(f"{type_name(ri, 'reply')} *next;")
+ elif ri.op_mode == 'notify' or ri.op_mode == 'event':
+ ri.cw.p('__u16 family;')
+ ri.cw.p('__u8 cmd;')
+ ri.cw.p(f"void (*free)({type_name(ri, 'reply')} *ntf);")
+ ri.cw.p(f"{type_name(ri, 'reply', deref=True)} obj __attribute__ ((aligned (8)));")
+ ri.cw.block_end(line=';')
+ ri.cw.nl()
+ print_free_prototype(ri, 'reply')
+ ri.cw.nl()
+
+
+def _free_type_members_iter(ri, struct):
+ for _, attr in struct.member_list():
+ if attr.free_needs_iter():
+ ri.cw.p('unsigned int i;')
+ ri.cw.nl()
+ break
+
+
+def _free_type_members(ri, var, struct, ref=''):
+ for _, attr in struct.member_list():
+ attr.free(ri, var, ref)
+
+
+def _free_type(ri, direction, struct):
+ var = free_arg_name(direction)
+
+ print_free_prototype(ri, direction, suffix='')
+ ri.cw.block_start()
+ _free_type_members_iter(ri, struct)
+ _free_type_members(ri, var, struct)
+ if direction:
+ ri.cw.p(f'free({var});')
+ ri.cw.block_end()
+ ri.cw.nl()
+
+
+def free_rsp_nested(ri, struct):
+ _free_type(ri, "", struct)
+
+
+def print_rsp_free(ri):
+ if 'reply' not in ri.op[ri.op_mode]:
+ return
+ _free_type(ri, 'reply', ri.struct['reply'])
+
+
+def print_dump_type_free(ri):
+ sub_type = type_name(ri, 'reply')
+
+ print_free_prototype(ri, 'reply', suffix='')
+ ri.cw.block_start()
+ ri.cw.p(f"{sub_type} *next = rsp;")
+ ri.cw.nl()
+ ri.cw.block_start(line='while (next)')
+ _free_type_members_iter(ri, ri.struct['reply'])
+ ri.cw.p('rsp = next;')
+ ri.cw.p('next = rsp->next;')
+ ri.cw.nl()
+
+ _free_type_members(ri, 'rsp', ri.struct['reply'], ref='obj.')
+ ri.cw.p(f'free(rsp);')
+ ri.cw.block_end()
+ ri.cw.block_end()
+ ri.cw.nl()
+
+
+def print_ntf_type_free(ri):
+ print_free_prototype(ri, 'reply', suffix='')
+ ri.cw.block_start()
+ _free_type_members_iter(ri, ri.struct['reply'])
+ _free_type_members(ri, 'rsp', ri.struct['reply'], ref='obj.')
+ ri.cw.p(f'free(rsp);')
+ ri.cw.block_end()
+ ri.cw.nl()
+
+
+def print_ntf_parse_prototype(family, cw, suffix=';'):
+ cw.write_func_prot('struct ynl_ntf_base_type *', f"{family['name']}_ntf_parse",
+ ['struct ynl_sock *ys'], suffix=suffix)
+
+
+def print_ntf_type_parse(family, cw, ku_mode):
+ print_ntf_parse_prototype(family, cw, suffix='')
+ cw.block_start()
+ cw.write_func_lvar(['struct genlmsghdr *genlh;',
+ 'struct nlmsghdr *nlh;',
+ 'struct ynl_parse_arg yarg = { .ys = ys, };',
+ 'struct ynl_ntf_base_type *rsp;',
+ 'int len, err;',
+ 'mnl_cb_t parse;'])
+ cw.p('len = mnl_socket_recvfrom(ys->sock, ys->rx_buf, MNL_SOCKET_BUFFER_SIZE);')
+ cw.p('if (len < (ssize_t)(sizeof(*nlh) + sizeof(*genlh)))')
+ cw.p('return NULL;')
+ cw.nl()
+ cw.p('nlh = (struct nlmsghdr *)ys->rx_buf;')
+ cw.p('genlh = mnl_nlmsg_get_payload(nlh);')
+ cw.nl()
+ cw.block_start(line='switch (genlh->cmd)')
+ for ntf_op in sorted(family.all_notify.keys()):
+ op = family.ops[ntf_op]
+ ri = RenderInfo(cw, family, ku_mode, op, ntf_op, "notify")
+ for ntf in op['notify']['cmds']:
+ cw.p(f"case {ntf.enum_name}:")
+ cw.p(f"rsp = calloc(1, sizeof({type_name(ri, 'notify')}));")
+ cw.p(f"parse = {op_prefix(ri, 'reply', deref=True)}_parse;")
+ cw.p(f"yarg.rsp_policy = &{ri.struct['reply'].render_name}_nest;")
+ cw.p(f"rsp->free = (void *){op_prefix(ri, 'notify')}_free;")
+ cw.p('break;')
+ for op_name, op in family.ops.items():
+ if 'event' not in op:
+ continue
+ ri = RenderInfo(cw, family, ku_mode, op, op_name, "event")
+ cw.p(f"case {op.enum_name}:")
+ cw.p(f"rsp = calloc(1, sizeof({type_name(ri, 'event')}));")
+ cw.p(f"parse = {op_prefix(ri, 'reply', deref=True)}_parse;")
+ cw.p(f"yarg.rsp_policy = &{ri.struct['reply'].render_name}_nest;")
+ cw.p(f"rsp->free = (void *){op_prefix(ri, 'notify')}_free;")
+ cw.p('break;')
+ cw.p('default:')
+ cw.p('ynl_error_unknown_notification(ys, genlh->cmd);')
+ cw.p('return NULL;')
+ cw.block_end()
+ cw.nl()
+ cw.p('yarg.data = rsp->data;')
+ cw.nl()
+ cw.p(f"err = {cw.nlib.parse_cb_run('parse', '&yarg', True)};")
+ cw.p('if (err < 0)')
+ cw.p('goto err_free;')
+ cw.nl()
+ cw.p('rsp->family = nlh->nlmsg_type;')
+ cw.p('rsp->cmd = genlh->cmd;')
+ cw.p('return rsp;')
+ cw.nl()
+ cw.p('err_free:')
+ cw.p('free(rsp);')
+ cw.p('return NULL;')
+ cw.block_end()
+ cw.nl()
+
+
+def print_req_policy_fwd(cw, struct, ri=None, terminate=True):
+ if terminate and ri and kernel_can_gen_family_struct(struct.family):
+ return
+
+ if terminate:
+ prefix = 'extern '
+ else:
+ if kernel_can_gen_family_struct(struct.family) and ri:
+ prefix = 'static '
+ else:
+ prefix = ''
+
+ suffix = ';' if terminate else ' = {'
+
+ max_attr = struct.attr_max_val
+ if ri:
+ name = ri.op.render_name
+ if ri.op.dual_policy:
+ name += '_' + ri.op_mode
+ else:
+ name = struct.render_name
+ cw.p(f"{prefix}const struct nla_policy {name}_nl_policy[{max_attr.enum_name} + 1]{suffix}")
+
+
+def print_req_policy(cw, struct, ri=None):
+ print_req_policy_fwd(cw, struct, ri=ri, terminate=False)
+ for _, arg in struct.member_list():
+ arg.attr_policy(cw)
+ cw.p("};")
+
+
+def kernel_can_gen_family_struct(family):
+ return family.proto == 'genetlink'
+
+
+def print_kernel_op_table_fwd(family, cw, terminate):
+ exported = not kernel_can_gen_family_struct(family)
+
+ if not terminate or exported:
+ cw.p(f"// Ops table for {family.name}")
+
+ pol_to_struct = {'global': 'genl_small_ops',
+ 'per-op': 'genl_ops',
+ 'split': 'genl_split_ops'}
+ struct_type = pol_to_struct[family.kernel_policy]
+
+ if family.kernel_policy == 'split':
+ cnt = 0
+ for op in family.ops.values():
+ if 'do' in op:
+ cnt += 1
+ if 'dump' in op:
+ cnt += 1
+ else:
+ cnt = len(family.ops)
+
+ qual = 'static const' if not exported else 'const'
+ line = f"{qual} struct {struct_type} {family.name}_nl_ops[{cnt}]"
+ if terminate:
+ cw.p(f"extern {line};")
+ else:
+ cw.block_start(line=line + ' =')
+
+ if not terminate:
+ return
+
+ cw.nl()
+ for name in family.hooks['pre']['do']['list']:
+ cw.write_func_prot('int', c_lower(name),
+ ['const struct genl_split_ops *ops',
+ 'struct sk_buff *skb', 'struct genl_info *info'], suffix=';')
+ for name in family.hooks['post']['do']['list']:
+ cw.write_func_prot('void', c_lower(name),
+ ['const struct genl_split_ops *ops',
+ 'struct sk_buff *skb', 'struct genl_info *info'], suffix=';')
+ for name in family.hooks['pre']['dump']['list']:
+ cw.write_func_prot('int', c_lower(name),
+ ['struct netlink_callback *cb'], suffix=';')
+ for name in family.hooks['post']['dump']['list']:
+ cw.write_func_prot('int', c_lower(name),
+ ['struct netlink_callback *cb'], suffix=';')
+
+ cw.nl()
+
+ for op_name, op in family.ops.items():
+ if op.is_async:
+ continue
+
+ if 'do' in op:
+ name = c_lower(f"{family.name}-nl-{op_name}-doit")
+ cw.write_func_prot('int', name,
+ ['struct sk_buff *skb', 'struct genl_info *info'], suffix=';')
+
+ if 'dump' in op:
+ name = c_lower(f"{family.name}-nl-{op_name}-dumpit")
+ cw.write_func_prot('int', name,
+ ['struct sk_buff *skb', 'struct netlink_callback *cb'], suffix=';')
+ cw.nl()
+
+
+def print_kernel_op_table_hdr(family, cw):
+ print_kernel_op_table_fwd(family, cw, terminate=True)
+
+
+def print_kernel_op_table(family, cw):
+ print_kernel_op_table_fwd(family, cw, terminate=False)
+ if family.kernel_policy == 'global' or family.kernel_policy == 'per-op':
+ for op_name, op in family.ops.items():
+ if op.is_async:
+ continue
+
+ cw.block_start()
+ members = [('cmd', op.enum_name)]
+ if 'dont-validate' in op:
+ members.append(('validate',
+ ' | '.join([c_upper('genl-dont-validate-' + x)
+ for x in op['dont-validate']])), )
+ for op_mode in ['do', 'dump']:
+ if op_mode in op:
+ name = c_lower(f"{family.name}-nl-{op_name}-{op_mode}it")
+ members.append((op_mode + 'it', name))
+ if family.kernel_policy == 'per-op':
+ struct = Struct(family, op['attribute-set'],
+ type_list=op['do']['request']['attributes'])
+
+ name = c_lower(f"{family.name}-{op_name}-nl-policy")
+ members.append(('policy', name))
+ members.append(('maxattr', struct.attr_max_val.enum_name))
+ if 'flags' in op:
+ members.append(('flags', ' | '.join([c_upper('genl-' + x) for x in op['flags']])))
+ cw.write_struct_init(members)
+ cw.block_end(line=',')
+ elif family.kernel_policy == 'split':
+ cb_names = {'do': {'pre': 'pre_doit', 'post': 'post_doit'},
+ 'dump': {'pre': 'start', 'post': 'done'}}
+
+ for op_name, op in family.ops.items():
+ for op_mode in ['do', 'dump']:
+ if op.is_async or op_mode not in op:
+ continue
+
+ cw.block_start()
+ members = [('cmd', op.enum_name)]
+ if 'dont-validate' in op:
+ members.append(('validate',
+ ' | '.join([c_upper('genl-dont-validate-' + x)
+ for x in op['dont-validate']])), )
+ name = c_lower(f"{family.name}-nl-{op_name}-{op_mode}it")
+ if 'pre' in op[op_mode]:
+ members.append((cb_names[op_mode]['pre'], c_lower(op[op_mode]['pre'])))
+ members.append((op_mode + 'it', name))
+ if 'post' in op[op_mode]:
+ members.append((cb_names[op_mode]['post'], c_lower(op[op_mode]['post'])))
+ if 'request' in op[op_mode]:
+ struct = Struct(family, op['attribute-set'],
+ type_list=op[op_mode]['request']['attributes'])
+
+ if op.dual_policy:
+ name = c_lower(f"{family.name}-{op_name}-{op_mode}-nl-policy")
+ else:
+ name = c_lower(f"{family.name}-{op_name}-nl-policy")
+ members.append(('policy', name))
+ members.append(('maxattr', struct.attr_max_val.enum_name))
+ flags = (op['flags'] if 'flags' in op else []) + ['cmd-cap-' + op_mode]
+ members.append(('flags', ' | '.join([c_upper('genl-' + x) for x in flags])))
+ cw.write_struct_init(members)
+ cw.block_end(line=',')
+
+ cw.block_end(line=';')
+ cw.nl()
+
+
+def print_kernel_mcgrp_hdr(family, cw):
+ if not family.mcgrps['list']:
+ return
+
+ cw.block_start('enum')
+ for grp in family.mcgrps['list']:
+ grp_id = f"{family.name}-nlgrp-{grp['name']},".upper().replace('-', '_')
+ cw.p(grp_id)
+ cw.block_end(';')
+ cw.nl()
+
+
+def print_kernel_mcgrp_src(family, cw):
+ if not family.mcgrps['list']:
+ return
+
+ cw.block_start('static const struct genl_multicast_group ' + family.name + '_nl_mcgrps[] =')
+ for grp in family.mcgrps['list']:
+ name = grp['name']
+ grp_id = f"{family.name}-nlgrp-{name}".upper().replace('-', '_')
+ cw.p('[' + grp_id + '] = { "' + name + '", },')
+ cw.block_end(';')
+ cw.nl()
+
+
+def print_kernel_family_struct_hdr(family, cw):
+ if not kernel_can_gen_family_struct(family):
+ return
+
+ cw.p(f"extern struct genl_family {family.name}_nl_family;")
+ cw.nl()
+
+
+def print_kernel_family_struct_src(family, cw):
+ if not kernel_can_gen_family_struct(family):
+ return
+
+ cw.block_start(f"struct genl_family {family.name}_nl_family __ro_after_init =")
+ cw.p('.name\t\t= ' + family.fam_key + ',')
+ cw.p('.version\t= ' + family.ver_key + ',')
+ cw.p('.netnsok\t= true,')
+ cw.p('.parallel_ops\t= true,')
+ cw.p('.module\t\t= THIS_MODULE,')
+ if family.kernel_policy == 'per-op':
+ cw.p(f'.ops\t\t= {family.name}_nl_ops,')
+ cw.p(f'.n_ops\t\t= ARRAY_SIZE({family.name}_nl_ops),')
+ elif family.kernel_policy == 'split':
+ cw.p(f'.split_ops\t= {family.name}_nl_ops,')
+ cw.p(f'.n_split_ops\t= ARRAY_SIZE({family.name}_nl_ops),')
+ if family.mcgrps['list']:
+ cw.p(f'.mcgrps\t\t= {family.name}_nl_mcgrps,')
+ cw.p(f'.n_mcgrps\t= ARRAY_SIZE({family.name}_nl_mcgrps),')
+ cw.block_end(';')
+
+
+def uapi_enum_start(family, cw, obj, ckey='', enum_name='enum-name'):
+ start_line = 'enum'
+ if enum_name in obj:
+ if obj[enum_name]:
+ start_line = 'enum ' + obj[enum_name].replace('-', '_')
+ elif ckey and ckey in obj:
+ start_line = 'enum ' + family.name + '_' + obj[ckey].replace('-', '_')
+ cw.block_start(line=start_line)
+
+
+def render_uapi(family, cw):
+ hdr_prot = f"_UAPI_LINUX_{family.name.upper()}_H"
+ cw.p('#ifndef ' + hdr_prot)
+ cw.p('#define ' + hdr_prot)
+ cw.nl()
+
+ defines = [(family.fam_key, family["name"]),
+ (family.ver_key, family.get('version', 1))]
+ cw.writes_defines(defines)
+ cw.nl()
+
+ defines = []
+ for const in family['definitions']:
+ if const['type'] != 'const':
+ cw.writes_defines(defines)
+ defines = []
+ cw.nl()
+
+ if const['type'] == 'enum':
+ enum = family.consts[const['name']]
+
+ if enum.has_doc():
+ cw.p('/**')
+ doc = ''
+ if 'doc' in enum:
+ doc = ' - ' + enum['doc']
+ cw.write_doc_line(enum.enum_name + doc)
+ for entry in enum.entry_list:
+ if entry.has_doc():
+ doc = '@' + entry.c_name + ': ' + entry['doc']
+ cw.write_doc_line(doc)
+ cw.p(' */')
+
+ uapi_enum_start(family, cw, const, 'name')
+ first = True
+ name_pfx = const.get('name-prefix', f"{family.name}-{const['name']}-")
+ for entry in enum.entry_list:
+ suffix = ','
+ if first and 'value-start' in const:
+ suffix = f" = {const['value-start']}" + suffix
+ first = False
+ cw.p(entry.c_name + suffix)
+
+ if const.get('render-max', False):
+ cw.nl()
+ max_name = c_upper(name_pfx + 'max')
+ cw.p('__' + max_name + ',')
+ cw.p(max_name + ' = (__' + max_name + ' - 1)')
+ cw.block_end(line=';')
+ cw.nl()
+ elif const['type'] == 'flags':
+ uapi_enum_start(family, cw, const, 'name')
+ i = const.get('value-start', 0)
+ for item in const['entries']:
+ item_name = item
+ if 'name-prefix' in const:
+ item_name = c_upper(const['name-prefix'] + item)
+ cw.p(f'{item_name} = {1 << i},')
+ i += 1
+ cw.block_end(line=';')
+ cw.nl()
+ elif const['type'] == 'const':
+ defines.append([c_upper(family.get('c-define-name',
+ f"{family.name}-{const['name']}")),
+ const['value']])
+
+ if defines:
+ cw.writes_defines(defines)
+ cw.nl()
+
+ max_by_define = family.get('max-by-define', False)
+
+ for _, attr_set in family.attr_sets_list:
+ if attr_set.subset_of:
+ continue
+
+ cnt_name = c_upper(family.get('attr-cnt-name', f"__{attr_set.name_prefix}MAX"))
+ max_value = f"({cnt_name} - 1)"
+
+ val = 0
+ uapi_enum_start(family, cw, attr_set.yaml, 'enum-name')
+ for _, attr in attr_set.items():
+ suffix = ','
+ if attr['value'] != val:
+ suffix = f" = {attr['value']},"
+ val = attr['value']
+ val += 1
+ cw.p(attr.enum_name + suffix)
+ cw.nl()
+ cw.p(cnt_name + ('' if max_by_define else ','))
+ if not max_by_define:
+ cw.p(f"{attr_set.max_name} = {max_value}")
+ cw.block_end(line=';')
+ if max_by_define:
+ cw.p(f"#define {attr_set.max_name} {max_value}")
+ cw.nl()
+
+ # Commands
+ separate_ntf = 'async-prefix' in family['operations']
+
+ max_name = c_upper(family.get('cmd-max-name', f"{family.op_prefix}MAX"))
+ cnt_name = c_upper(family.get('cmd-cnt-name', f"__{family.op_prefix}MAX"))
+ max_value = f"({cnt_name} - 1)"
+
+ uapi_enum_start(family, cw, family['operations'], 'enum-name')
+ for _, op in family.ops_list:
+ if separate_ntf and ('notify' in op or 'event' in op):
+ continue
+
+ suffix = ','
+ if 'value' in op:
+ suffix = f" = {op['value']},"
+ cw.p(op.enum_name + suffix)
+ cw.nl()
+ cw.p(cnt_name + ('' if max_by_define else ','))
+ if not max_by_define:
+ cw.p(f"{max_name} = {max_value}")
+ cw.block_end(line=';')
+ if max_by_define:
+ cw.p(f"#define {max_name} {max_value}")
+ cw.nl()
+
+ if separate_ntf:
+ uapi_enum_start(family, cw, family['operations'], enum_name='async-enum')
+ for _, op in family.ops_list:
+ if separate_ntf and not ('notify' in op or 'event' in op):
+ continue
+
+ suffix = ','
+ if 'value' in op:
+ suffix = f" = {op['value']},"
+ cw.p(op.enum_name + suffix)
+ cw.block_end(line=';')
+ cw.nl()
+
+ # Multicast
+ defines = []
+ for grp in family.mcgrps['list']:
+ name = grp['name']
+ defines.append([c_upper(grp.get('c-define-name', f"{family.name}-mcgrp-{name}")),
+ f'{name}'])
+ cw.nl()
+ if defines:
+ cw.writes_defines(defines)
+ cw.nl()
+
+ cw.p(f'#endif /* {hdr_prot} */')
+
+
+def find_kernel_root(full_path):
+ sub_path = ''
+ while True:
+ sub_path = os.path.join(os.path.basename(full_path), sub_path)
+ full_path = os.path.dirname(full_path)
+ maintainers = os.path.join(full_path, "MAINTAINERS")
+ if os.path.exists(maintainers):
+ return full_path, sub_path[:-1]
+
+
+def main():
+ parser = argparse.ArgumentParser(description='Netlink simple parsing generator')
+ parser.add_argument('--mode', dest='mode', type=str, required=True)
+ parser.add_argument('--spec', dest='spec', type=str, required=True)
+ parser.add_argument('--header', dest='header', action='store_true', default=None)
+ parser.add_argument('--source', dest='header', action='store_false')
+ parser.add_argument('--user-header', nargs='+', default=[])
+ parser.add_argument('-o', dest='out_file', type=str)
+ args = parser.parse_args()
+
+ out_file = open(args.out_file, 'w+') if args.out_file else os.sys.stdout
+
+ if args.header is None:
+ parser.error("--header or --source is required")
+
+ try:
+ parsed = Family(args.spec)
+ except yaml.YAMLError as exc:
+ print(exc)
+ os.sys.exit(1)
+ return
+
+ cw = CodeWriter(BaseNlLib(), out_file)
+
+ _, spec_kernel = find_kernel_root(args.spec)
+ if args.mode == 'uapi':
+ cw.p('/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */')
+ else:
+ cw.p('// SPDX-License-Identifier: BSD-3-Clause')
+ cw.p("/* Do not edit directly, auto-generated from: */")
+ cw.p(f"/*\t{spec_kernel} */")
+ cw.p(f"/* YNL-GEN {args.mode} {'header' if args.header else 'source'} */")
+ cw.nl()
+
+ if args.mode == 'uapi':
+ render_uapi(parsed, cw)
+ return
+
+ hdr_prot = f"_LINUX_{parsed.name.upper()}_GEN_H"
+ if args.header:
+ cw.p('#ifndef ' + hdr_prot)
+ cw.p('#define ' + hdr_prot)
+ cw.nl()
+
+ if args.mode == 'kernel':
+ cw.p('#include <net/netlink.h>')
+ cw.p('#include <net/genetlink.h>')
+ cw.nl()
+ if not args.header:
+ if args.out_file:
+ cw.p(f'#include "{os.path.basename(args.out_file[:-2])}.h"')
+ cw.nl()
+ headers = [parsed.uapi_header]
+ for definition in parsed['definitions']:
+ if 'header' in definition:
+ headers.append(definition['header'])
+ for one in headers:
+ cw.p(f"#include <{one}>")
+ cw.nl()
+
+ if args.mode == "user":
+ if not args.header:
+ cw.p("#include <stdlib.h>")
+ cw.p("#include <stdio.h>")
+ cw.p("#include <string.h>")
+ cw.p("#include <libmnl/libmnl.h>")
+ cw.p("#include <linux/genetlink.h>")
+ cw.nl()
+ for one in args.user_header:
+ cw.p(f'#include "{one}"')
+ else:
+ cw.p('struct ynl_sock;')
+ cw.nl()
+
+ if args.mode == "kernel":
+ if args.header:
+ for _, struct in sorted(parsed.pure_nested_structs.items()):
+ if struct.request:
+ cw.p('/* Common nested types */')
+ break
+ for attr_set, struct in sorted(parsed.pure_nested_structs.items()):
+ if struct.request:
+ print_req_policy_fwd(cw, struct)
+ cw.nl()
+
+ if parsed.kernel_policy == 'global':
+ cw.p(f"// Global operation policy for {parsed.name}")
+
+ struct = Struct(parsed, parsed.global_policy_set, type_list=parsed.global_policy)
+ print_req_policy_fwd(cw, struct)
+ cw.nl()
+
+ if parsed.kernel_policy in {'per-op', 'split'}:
+ for op_name, op in parsed.ops.items():
+ if 'do' in op and 'event' not in op:
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, "do")
+ print_req_policy_fwd(cw, ri.struct['request'], ri=ri)
+ cw.nl()
+
+ print_kernel_op_table_hdr(parsed, cw)
+ print_kernel_mcgrp_hdr(parsed, cw)
+ print_kernel_family_struct_hdr(parsed, cw)
+ else:
+ for _, struct in sorted(parsed.pure_nested_structs.items()):
+ if struct.request:
+ cw.p('/* Common nested types */')
+ break
+ for attr_set, struct in sorted(parsed.pure_nested_structs.items()):
+ if struct.request:
+ print_req_policy(cw, struct)
+ cw.nl()
+
+ if parsed.kernel_policy == 'global':
+ cw.p(f"// Global operation policy for {parsed.name}")
+
+ struct = Struct(parsed, parsed.global_policy_set, type_list=parsed.global_policy)
+ print_req_policy(cw, struct)
+ cw.nl()
+
+ for op_name, op in parsed.ops.items():
+ if parsed.kernel_policy in {'per-op', 'split'}:
+ for op_mode in {'do', 'dump'}:
+ if op_mode in op and 'request' in op[op_mode]:
+ cw.p(f"// {op.enum_name} - {op_mode}")
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, op_mode)
+ print_req_policy(cw, ri.struct['request'], ri=ri)
+ cw.nl()
+
+ print_kernel_op_table(parsed, cw)
+ print_kernel_mcgrp_src(parsed, cw)
+ print_kernel_family_struct_src(parsed, cw)
+
+ if args.mode == "user":
+ has_ntf = False
+ if args.header:
+ cw.p('// Common nested types')
+ for attr_set, struct in sorted(parsed.pure_nested_structs.items()):
+ ri = RenderInfo(cw, parsed, args.mode, "", "", "", attr_set)
+ print_type_full(ri, struct)
+
+ for op_name, op in parsed.ops.items():
+ cw.p(f"/* ============== {op.enum_name} ============== */")
+
+ if 'do' in op and 'event' not in op:
+ cw.p(f"// {op.enum_name} - do")
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, "do")
+ print_req_type(ri)
+ print_req_type_helpers(ri)
+ cw.nl()
+ print_rsp_type(ri)
+ print_rsp_type_helpers(ri)
+ cw.nl()
+ print_req_prototype(ri)
+ cw.nl()
+
+ if 'dump' in op:
+ cw.p(f"// {op.enum_name} - dump")
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, 'dump')
+ if 'request' in op['dump']:
+ print_req_type(ri)
+ print_req_type_helpers(ri)
+ if not ri.type_consistent:
+ print_rsp_type(ri)
+ print_wrapped_type(ri)
+ print_dump_prototype(ri)
+ cw.nl()
+
+ if 'notify' in op:
+ cw.p(f"// {op.enum_name} - notify")
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, 'notify')
+ has_ntf = True
+ if not ri.type_consistent:
+ raise Exception('Only notifications with consistent types supported')
+ print_wrapped_type(ri)
+
+ if 'event' in op:
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, 'event')
+ cw.p(f"// {op.enum_name} - event")
+ print_rsp_type(ri)
+ cw.nl()
+ print_wrapped_type(ri)
+
+ if has_ntf:
+ cw.p('// --------------- Common notification parsing --------------- //')
+ print_ntf_parse_prototype(parsed, cw)
+ cw.nl()
+ else:
+ cw.p('// Policies')
+ for name, _ in parsed.attr_sets.items():
+ struct = Struct(parsed, name)
+ put_typol_fwd(cw, struct)
+ cw.nl()
+
+ for name, _ in parsed.attr_sets.items():
+ struct = Struct(parsed, name)
+ put_typol(cw, struct)
+
+ cw.p('// Common nested types')
+ for attr_set, struct in sorted(parsed.pure_nested_structs.items()):
+ ri = RenderInfo(cw, parsed, args.mode, "", "", "", attr_set)
+
+ free_rsp_nested(ri, struct)
+ if struct.request:
+ put_req_nested(ri, struct)
+ if struct.reply:
+ parse_rsp_nested(ri, struct)
+
+ for op_name, op in parsed.ops.items():
+ cw.p(f"/* ============== {op.enum_name} ============== */")
+ if 'do' in op and 'event' not in op:
+ cw.p(f"// {op.enum_name} - do")
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, "do")
+ print_rsp_free(ri)
+ parse_rsp_msg(ri)
+ print_req(ri)
+ cw.nl()
+
+ if 'dump' in op:
+ cw.p(f"// {op.enum_name} - dump")
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, "dump")
+ if not ri.type_consistent:
+ parse_rsp_msg(ri, deref=True)
+ print_dump_type_free(ri)
+ print_dump(ri)
+ cw.nl()
+
+ if 'notify' in op:
+ cw.p(f"// {op.enum_name} - notify")
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, 'notify')
+ has_ntf = True
+ if not ri.type_consistent:
+ raise Exception('Only notifications with consistent types supported')
+ print_ntf_type_free(ri)
+
+ if 'event' in op:
+ cw.p(f"// {op.enum_name} - event")
+ has_ntf = True
+
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, "do")
+ parse_rsp_msg(ri)
+
+ ri = RenderInfo(cw, parsed, args.mode, op, op_name, "event")
+ print_ntf_type_free(ri)
+
+ if has_ntf:
+ cw.p('// --------------- Common notification parsing --------------- //')
+ print_ntf_type_parse(parsed, cw, args.mode)
+
+ if args.header:
+ cw.p(f'#endif /* {hdr_prot} */')
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/net/ynl/ynl-regen.sh b/tools/net/ynl/ynl-regen.sh
new file mode 100755
index 000000000000..43989ae48ed0
--- /dev/null
+++ b/tools/net/ynl/ynl-regen.sh
@@ -0,0 +1,30 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+
+TOOL=$(dirname $(realpath $0))/ynl-gen-c.py
+
+force=
+
+while [ ! -z "$1" ]; do
+ case "$1" in
+ -f ) force=yes; shift ;;
+ * ) echo "Unrecognized option '$1'"; exit 1 ;;
+ esac
+done
+
+KDIR=$(dirname $(dirname $(dirname $(dirname $(realpath $0)))))
+
+files=$(git grep --files-with-matches '^/\* YNL-GEN \(kernel\|uapi\)')
+for f in $files; do
+ # params: 0 1 2 3
+ # $YAML YNL-GEN kernel $mode
+ params=( $(git grep -B1 -h '/\* YNL-GEN' $f | sed 's@/\*\(.*\)\*/@\1@') )
+
+ if [ $f -nt ${params[0]} -a -z "$force" ]; then
+ echo -e "\tSKIP $f"
+ continue
+ fi
+
+ echo -e "\tGEN ${params[2]}\t$f"
+ $TOOL --mode ${params[2]} --${params[3]} --spec $KDIR/${params[0]} -o $f
+done
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v3 4/8] netlink: add a proto specification for FOU
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
` (2 preceding siblings ...)
2023-01-19 0:36 ` [PATCH net-next v3 3/8] net: add basic C code generators for Netlink Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 5/8] net: fou: regenerate the uAPI from the spec Jakub Kicinski
` (4 subsequent siblings)
8 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
FOU has a reasonably modern Genetlink family. Add a spec.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
Documentation/netlink/specs/fou.yaml | 128 +++++++++++++++++++++++++++
1 file changed, 128 insertions(+)
create mode 100644 Documentation/netlink/specs/fou.yaml
diff --git a/Documentation/netlink/specs/fou.yaml b/Documentation/netlink/specs/fou.yaml
new file mode 100644
index 000000000000..266c386eedf3
--- /dev/null
+++ b/Documentation/netlink/specs/fou.yaml
@@ -0,0 +1,128 @@
+name: fou
+
+protocol: genetlink-legacy
+
+doc: |
+ Foo-over-UDP.
+
+c-family-name: fou-genl-name
+c-version-name: fou-genl-version
+max-by-define: true
+kernel-policy: global
+
+definitions:
+ -
+ type: enum
+ name: encap_type
+ name-prefix: fou-encap-
+ enum-name:
+ entries: [ unspec, direct, gue ]
+
+attribute-sets:
+ -
+ name: fou
+ name-prefix: fou-attr-
+ attributes:
+ -
+ name: unspec
+ type: unused
+ -
+ name: port
+ type: u16
+ byte-order: big-endian
+ -
+ name: af
+ type: u8
+ -
+ name: ipproto
+ type: u8
+ -
+ name: type
+ type: u8
+ -
+ name: remcsum_nopartial
+ type: flag
+ -
+ name: local_v4
+ type: u32
+ -
+ name: local_v6
+ type: binary
+ checks:
+ min-len: 16
+ -
+ name: peer_v4
+ type: u32
+ -
+ name: peer_v6
+ type: binary
+ checks:
+ min-len: 16
+ -
+ name: peer_port
+ type: u16
+ byte-order: big-endian
+ -
+ name: ifindex
+ type: s32
+
+operations:
+ list:
+ -
+ name: unspec
+ doc: unused
+
+ -
+ name: add
+ doc: Add port.
+ attribute-set: fou
+
+ dont-validate: [ strict, dump ]
+ flags: [ admin-perm ]
+
+ do:
+ request: &all_attrs
+ attributes:
+ - port
+ - ipproto
+ - type
+ - remcsum_nopartial
+ - local_v4
+ - peer_v4
+ - local_v6
+ - peer_v6
+ - peer_port
+ - ifindex
+
+ -
+ name: del
+ doc: Delete port.
+ attribute-set: fou
+
+ dont-validate: [ strict, dump ]
+ flags: [ admin-perm ]
+
+ do:
+ request: &select_attrs
+ attributes:
+ - af
+ - ifindex
+ - port
+ - peer_port
+ - local_v4
+ - peer_v4
+ - local_v6
+ - peer_v6
+
+ -
+ name: get
+ doc: Get tunnel info.
+ attribute-set: fou
+ dont-validate: [ strict, dump ]
+
+ do:
+ request: *select_attrs
+ reply: *all_attrs
+
+ dump:
+ reply: *all_attrs
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v3 5/8] net: fou: regenerate the uAPI from the spec
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
` (3 preceding siblings ...)
2023-01-19 0:36 ` [PATCH net-next v3 4/8] netlink: add a proto specification for FOU Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 6/8] net: fou: rename the source for linking Jakub Kicinski
` (3 subsequent siblings)
8 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
Regenerate the FOU uAPI header from the YAML spec.
The flags now come before attributes which use them,
and the comments for type disappear (coders should look
at the spec instead).
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
include/uapi/linux/fou.h | 54 +++++++++++++++++++---------------------
1 file changed, 26 insertions(+), 28 deletions(-)
diff --git a/include/uapi/linux/fou.h b/include/uapi/linux/fou.h
index 87c2c9f08803..19ebbef41a63 100644
--- a/include/uapi/linux/fou.h
+++ b/include/uapi/linux/fou.h
@@ -1,32 +1,37 @@
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-/* fou.h - FOU Interface */
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/fou.yaml */
+/* YNL-GEN uapi header */
#ifndef _UAPI_LINUX_FOU_H
#define _UAPI_LINUX_FOU_H
-/* NETLINK_GENERIC related info
- */
#define FOU_GENL_NAME "fou"
-#define FOU_GENL_VERSION 0x1
+#define FOU_GENL_VERSION 1
enum {
- FOU_ATTR_UNSPEC,
- FOU_ATTR_PORT, /* u16 */
- FOU_ATTR_AF, /* u8 */
- FOU_ATTR_IPPROTO, /* u8 */
- FOU_ATTR_TYPE, /* u8 */
- FOU_ATTR_REMCSUM_NOPARTIAL, /* flag */
- FOU_ATTR_LOCAL_V4, /* u32 */
- FOU_ATTR_LOCAL_V6, /* in6_addr */
- FOU_ATTR_PEER_V4, /* u32 */
- FOU_ATTR_PEER_V6, /* in6_addr */
- FOU_ATTR_PEER_PORT, /* u16 */
- FOU_ATTR_IFINDEX, /* s32 */
-
- __FOU_ATTR_MAX,
+ FOU_ENCAP_UNSPEC,
+ FOU_ENCAP_DIRECT,
+ FOU_ENCAP_GUE,
};
-#define FOU_ATTR_MAX (__FOU_ATTR_MAX - 1)
+enum {
+ FOU_ATTR_UNSPEC,
+ FOU_ATTR_PORT,
+ FOU_ATTR_AF,
+ FOU_ATTR_IPPROTO,
+ FOU_ATTR_TYPE,
+ FOU_ATTR_REMCSUM_NOPARTIAL,
+ FOU_ATTR_LOCAL_V4,
+ FOU_ATTR_LOCAL_V6,
+ FOU_ATTR_PEER_V4,
+ FOU_ATTR_PEER_V6,
+ FOU_ATTR_PEER_PORT,
+ FOU_ATTR_IFINDEX,
+
+ __FOU_ATTR_MAX
+};
+#define FOU_ATTR_MAX (__FOU_ATTR_MAX - 1)
enum {
FOU_CMD_UNSPEC,
@@ -34,15 +39,8 @@ enum {
FOU_CMD_DEL,
FOU_CMD_GET,
- __FOU_CMD_MAX,
+ __FOU_CMD_MAX
};
-
-enum {
- FOU_ENCAP_UNSPEC,
- FOU_ENCAP_DIRECT,
- FOU_ENCAP_GUE,
-};
-
-#define FOU_CMD_MAX (__FOU_CMD_MAX - 1)
+#define FOU_CMD_MAX (__FOU_CMD_MAX - 1)
#endif /* _UAPI_LINUX_FOU_H */
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v3 6/8] net: fou: rename the source for linking
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
` (4 preceding siblings ...)
2023-01-19 0:36 ` [PATCH net-next v3 5/8] net: fou: regenerate the uAPI from the spec Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec Jakub Kicinski
` (2 subsequent siblings)
8 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
We'll need to link two objects together to form the fou module.
This means the source can't be called fou, the build system expects
fou.o to be the combined object.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
net/ipv4/Makefile | 1 +
net/ipv4/{fou.c => fou_core.c} | 0
2 files changed, 1 insertion(+)
rename net/ipv4/{fou.c => fou_core.c} (100%)
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index af7d2cf490fb..fabbe46897ce 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -26,6 +26,7 @@ obj-$(CONFIG_IP_MROUTE) += ipmr.o
obj-$(CONFIG_IP_MROUTE_COMMON) += ipmr_base.o
obj-$(CONFIG_NET_IPIP) += ipip.o
gre-y := gre_demux.o
+fou-y := fou_core.o
obj-$(CONFIG_NET_FOU) += fou.o
obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o
obj-$(CONFIG_NET_IPGRE) += ip_gre.o
diff --git a/net/ipv4/fou.c b/net/ipv4/fou_core.c
similarity index 100%
rename from net/ipv4/fou.c
rename to net/ipv4/fou_core.c
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
` (5 preceding siblings ...)
2023-01-19 0:36 ` [PATCH net-next v3 6/8] net: fou: rename the source for linking Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-19 20:56 ` Johannes Berg
2023-01-19 0:36 ` [PATCH net-next v3 8/8] tools: ynl: add a completely generic client Jakub Kicinski
2023-01-19 17:07 ` [PATCH net-next v3 0/8] Netlink protocol specs Stanislav Fomichev
8 siblings, 1 reply; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
Generate and plug in the spec-based tables.
A little bit of renaming is needed in the FOU code.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
net/ipv4/Makefile | 2 +-
net/ipv4/fou-nl.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
net/ipv4/fou-nl.h | 25 +++++++++++++++++++++++
net/ipv4/fou_core.c | 47 +++++++-------------------------------------
4 files changed, 81 insertions(+), 41 deletions(-)
create mode 100644 net/ipv4/fou-nl.c
create mode 100644 net/ipv4/fou-nl.h
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index fabbe46897ce..984da5159afb 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -26,7 +26,7 @@ obj-$(CONFIG_IP_MROUTE) += ipmr.o
obj-$(CONFIG_IP_MROUTE_COMMON) += ipmr_base.o
obj-$(CONFIG_NET_IPIP) += ipip.o
gre-y := gre_demux.o
-fou-y := fou_core.o
+fou-y := fou_core.o fou-nl.o
obj-$(CONFIG_NET_FOU) += fou.o
obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o
obj-$(CONFIG_NET_IPGRE) += ip_gre.o
diff --git a/net/ipv4/fou-nl.c b/net/ipv4/fou-nl.c
new file mode 100644
index 000000000000..2ce8151301b4
--- /dev/null
+++ b/net/ipv4/fou-nl.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/fou.yaml */
+/* YNL-GEN kernel source */
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include "fou-nl.h"
+
+#include <linux/fou.h>
+
+// Global operation policy for fou
+const struct nla_policy fou_nl_policy[FOU_ATTR_IFINDEX + 1] = {
+ [FOU_ATTR_PORT] = { .type = NLA_U16, },
+ [FOU_ATTR_AF] = { .type = NLA_U8, },
+ [FOU_ATTR_IPPROTO] = { .type = NLA_U8, },
+ [FOU_ATTR_TYPE] = { .type = NLA_U8, },
+ [FOU_ATTR_REMCSUM_NOPARTIAL] = { .type = NLA_FLAG, },
+ [FOU_ATTR_LOCAL_V4] = { .type = NLA_U32, },
+ [FOU_ATTR_LOCAL_V6] = { .len = 16, },
+ [FOU_ATTR_PEER_V4] = { .type = NLA_U32, },
+ [FOU_ATTR_PEER_V6] = { .len = 16, },
+ [FOU_ATTR_PEER_PORT] = { .type = NLA_U16, },
+ [FOU_ATTR_IFINDEX] = { .type = NLA_S32, },
+};
+
+// Ops table for fou
+const struct genl_small_ops fou_nl_ops[3] = {
+ {
+ .cmd = FOU_CMD_ADD,
+ .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .doit = fou_nl_add_doit,
+ .flags = GENL_ADMIN_PERM,
+ },
+ {
+ .cmd = FOU_CMD_DEL,
+ .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .doit = fou_nl_del_doit,
+ .flags = GENL_ADMIN_PERM,
+ },
+ {
+ .cmd = FOU_CMD_GET,
+ .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .doit = fou_nl_get_doit,
+ .dumpit = fou_nl_get_dumpit,
+ },
+};
diff --git a/net/ipv4/fou-nl.h b/net/ipv4/fou-nl.h
new file mode 100644
index 000000000000..fc1d1c671c4d
--- /dev/null
+++ b/net/ipv4/fou-nl.h
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/fou.yaml */
+/* YNL-GEN kernel header */
+
+#ifndef _LINUX_FOU_GEN_H
+#define _LINUX_FOU_GEN_H
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include <linux/fou.h>
+
+// Global operation policy for fou
+extern const struct nla_policy fou_nl_policy[FOU_ATTR_IFINDEX + 1];
+
+// Ops table for fou
+extern const struct genl_small_ops fou_nl_ops[3];
+
+int fou_nl_add_doit(struct sk_buff *skb, struct genl_info *info);
+int fou_nl_del_doit(struct sk_buff *skb, struct genl_info *info);
+int fou_nl_get_doit(struct sk_buff *skb, struct genl_info *info);
+int fou_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
+
+#endif /* _LINUX_FOU_GEN_H */
diff --git a/net/ipv4/fou_core.c b/net/ipv4/fou_core.c
index 0c3c6d0cee29..606d2a4556aa 100644
--- a/net/ipv4/fou_core.c
+++ b/net/ipv4/fou_core.c
@@ -19,6 +19,8 @@
#include <uapi/linux/fou.h>
#include <uapi/linux/genetlink.h>
+#include "fou-nl.h"
+
struct fou {
struct socket *sock;
u8 protocol;
@@ -640,20 +642,6 @@ static int fou_destroy(struct net *net, struct fou_cfg *cfg)
static struct genl_family fou_nl_family;
-static const struct nla_policy fou_nl_policy[FOU_ATTR_MAX + 1] = {
- [FOU_ATTR_PORT] = { .type = NLA_U16, },
- [FOU_ATTR_AF] = { .type = NLA_U8, },
- [FOU_ATTR_IPPROTO] = { .type = NLA_U8, },
- [FOU_ATTR_TYPE] = { .type = NLA_U8, },
- [FOU_ATTR_REMCSUM_NOPARTIAL] = { .type = NLA_FLAG, },
- [FOU_ATTR_LOCAL_V4] = { .type = NLA_U32, },
- [FOU_ATTR_PEER_V4] = { .type = NLA_U32, },
- [FOU_ATTR_LOCAL_V6] = { .len = sizeof(struct in6_addr), },
- [FOU_ATTR_PEER_V6] = { .len = sizeof(struct in6_addr), },
- [FOU_ATTR_PEER_PORT] = { .type = NLA_U16, },
- [FOU_ATTR_IFINDEX] = { .type = NLA_S32, },
-};
-
static int parse_nl_config(struct genl_info *info,
struct fou_cfg *cfg)
{
@@ -745,7 +733,7 @@ static int parse_nl_config(struct genl_info *info,
return 0;
}
-static int fou_nl_cmd_add_port(struct sk_buff *skb, struct genl_info *info)
+int fou_nl_add_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
struct fou_cfg cfg;
@@ -758,7 +746,7 @@ static int fou_nl_cmd_add_port(struct sk_buff *skb, struct genl_info *info)
return fou_create(net, &cfg, NULL);
}
-static int fou_nl_cmd_rm_port(struct sk_buff *skb, struct genl_info *info)
+int fou_nl_del_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
struct fou_cfg cfg;
@@ -827,7 +815,7 @@ static int fou_dump_info(struct fou *fou, u32 portid, u32 seq,
return -EMSGSIZE;
}
-static int fou_nl_cmd_get_port(struct sk_buff *skb, struct genl_info *info)
+int fou_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
struct fou_net *fn = net_generic(net, fou_net_id);
@@ -874,7 +862,7 @@ static int fou_nl_cmd_get_port(struct sk_buff *skb, struct genl_info *info)
return ret;
}
-static int fou_nl_dump(struct sk_buff *skb, struct netlink_callback *cb)
+int fou_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb)
{
struct net *net = sock_net(skb->sk);
struct fou_net *fn = net_generic(net, fou_net_id);
@@ -897,33 +885,12 @@ static int fou_nl_dump(struct sk_buff *skb, struct netlink_callback *cb)
return skb->len;
}
-static const struct genl_small_ops fou_nl_ops[] = {
- {
- .cmd = FOU_CMD_ADD,
- .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .doit = fou_nl_cmd_add_port,
- .flags = GENL_ADMIN_PERM,
- },
- {
- .cmd = FOU_CMD_DEL,
- .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .doit = fou_nl_cmd_rm_port,
- .flags = GENL_ADMIN_PERM,
- },
- {
- .cmd = FOU_CMD_GET,
- .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .doit = fou_nl_cmd_get_port,
- .dumpit = fou_nl_dump,
- },
-};
-
static struct genl_family fou_nl_family __ro_after_init = {
.hdrsize = 0,
.name = FOU_GENL_NAME,
.version = FOU_GENL_VERSION,
.maxattr = FOU_ATTR_MAX,
- .policy = fou_nl_policy,
+ .policy = fou_nl_policy,
.netnsok = true,
.module = THIS_MODULE,
.small_ops = fou_nl_ops,
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v3 8/8] tools: ynl: add a completely generic client
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
` (6 preceding siblings ...)
2023-01-19 0:36 ` [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec Jakub Kicinski
@ 2023-01-19 0:36 ` Jakub Kicinski
2023-01-20 0:50 ` Jacob Keller
2023-01-19 17:07 ` [PATCH net-next v3 0/8] Netlink protocol specs Stanislav Fomichev
8 siblings, 1 reply; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 0:36 UTC (permalink / raw
To: davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel,
Jakub Kicinski
Add a CLI sample which can take in arbitrary request
in JSON format, convert it to Netlink and do the inverse
for output.
It's meant as a development tool primarily and perhaps
for selftests which need to tickle netlink in a special way.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
tools/net/ynl/samples/cli.py | 47 +++
tools/net/ynl/samples/ynl.py | 534 +++++++++++++++++++++++++++++++++++
2 files changed, 581 insertions(+)
create mode 100755 tools/net/ynl/samples/cli.py
create mode 100644 tools/net/ynl/samples/ynl.py
diff --git a/tools/net/ynl/samples/cli.py b/tools/net/ynl/samples/cli.py
new file mode 100755
index 000000000000..b27159c70710
--- /dev/null
+++ b/tools/net/ynl/samples/cli.py
@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+
+import argparse
+import json
+import pprint
+import time
+
+from ynl import YnlFamily
+
+
+def main():
+ parser = argparse.ArgumentParser(description='YNL CLI sample')
+ parser.add_argument('--spec', dest='spec', type=str, required=True)
+ parser.add_argument('--schema', dest='schema', type=str)
+ parser.add_argument('--json', dest='json_text', type=str)
+ parser.add_argument('--do', dest='do', type=str)
+ parser.add_argument('--dump', dest='dump', type=str)
+ parser.add_argument('--sleep', dest='sleep', type=int)
+ parser.add_argument('--subscribe', dest='ntf', type=str)
+ args = parser.parse_args()
+
+ attrs = {}
+ if args.json_text:
+ attrs = json.loads(args.json_text)
+
+ ynl = YnlFamily(args.spec, args.schema)
+
+ if args.ntf:
+ ynl.ntf_subscribe(args.ntf)
+
+ if args.sleep:
+ time.sleep(args.sleep)
+
+ if args.do or args.dump:
+ method = getattr(ynl, args.do if args.do else args.dump)
+
+ reply = method(attrs, dump=bool(args.dump))
+ pprint.PrettyPrinter().pprint(reply)
+
+ if args.ntf:
+ ynl.check_ntf()
+ pprint.PrettyPrinter().pprint(ynl.async_msg_queue)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/net/ynl/samples/ynl.py b/tools/net/ynl/samples/ynl.py
new file mode 100644
index 000000000000..b71523d71d46
--- /dev/null
+++ b/tools/net/ynl/samples/ynl.py
@@ -0,0 +1,534 @@
+# SPDX-License-Identifier: BSD-3-Clause
+
+import functools
+import jsonschema
+import os
+import random
+import socket
+import struct
+import yaml
+
+#
+# Generic Netlink code which should really be in some library, but I can't quickly find one.
+#
+
+
+class Netlink:
+ # Netlink socket
+ SOL_NETLINK = 270
+
+ NETLINK_ADD_MEMBERSHIP = 1
+ NETLINK_CAP_ACK = 10
+ NETLINK_EXT_ACK = 11
+
+ # Netlink message
+ NLMSG_ERROR = 2
+ NLMSG_DONE = 3
+
+ NLM_F_REQUEST = 1
+ NLM_F_ACK = 4
+ NLM_F_ROOT = 0x100
+ NLM_F_MATCH = 0x200
+ NLM_F_APPEND = 0x800
+
+ NLM_F_CAPPED = 0x100
+ NLM_F_ACK_TLVS = 0x200
+
+ NLM_F_DUMP = NLM_F_ROOT | NLM_F_MATCH
+
+ NLA_F_NESTED = 0x8000
+ NLA_F_NET_BYTEORDER = 0x4000
+
+ NLA_TYPE_MASK = NLA_F_NESTED | NLA_F_NET_BYTEORDER
+
+ # Genetlink defines
+ NETLINK_GENERIC = 16
+
+ GENL_ID_CTRL = 0x10
+
+ # nlctrl
+ CTRL_CMD_GETFAMILY = 3
+
+ CTRL_ATTR_FAMILY_ID = 1
+ CTRL_ATTR_FAMILY_NAME = 2
+ CTRL_ATTR_MAXATTR = 5
+ CTRL_ATTR_MCAST_GROUPS = 7
+
+ CTRL_ATTR_MCAST_GRP_NAME = 1
+ CTRL_ATTR_MCAST_GRP_ID = 2
+
+ # Extack types
+ NLMSGERR_ATTR_MSG = 1
+ NLMSGERR_ATTR_OFFS = 2
+ NLMSGERR_ATTR_COOKIE = 3
+ NLMSGERR_ATTR_POLICY = 4
+ NLMSGERR_ATTR_MISS_TYPE = 5
+ NLMSGERR_ATTR_MISS_NEST = 6
+
+
+class NlAttr:
+ def __init__(self, raw, offset):
+ self._len, self._type = struct.unpack("HH", raw[offset:offset + 4])
+ self.type = self._type & ~Netlink.NLA_TYPE_MASK
+ self.payload_len = self._len
+ self.full_len = (self.payload_len + 3) & ~3
+ self.raw = raw[offset + 4:offset + self.payload_len]
+
+ def as_u16(self):
+ return struct.unpack("H", self.raw)[0]
+
+ def as_u32(self):
+ return struct.unpack("I", self.raw)[0]
+
+ def as_u64(self):
+ return struct.unpack("Q", self.raw)[0]
+
+ def as_strz(self):
+ return self.raw.decode('ascii')[:-1]
+
+ def as_bin(self):
+ return self.raw
+
+ def __repr__(self):
+ return f"[type:{self.type} len:{self._len}] {self.raw}"
+
+
+class NlAttrs:
+ def __init__(self, msg):
+ self.attrs = []
+
+ offset = 0
+ while offset < len(msg):
+ attr = NlAttr(msg, offset)
+ offset += attr.full_len
+ self.attrs.append(attr)
+
+ def __iter__(self):
+ yield from self.attrs
+
+ def __repr__(self):
+ msg = ''
+ for a in self.attrs:
+ if msg:
+ msg += '\n'
+ msg += repr(a)
+ return msg
+
+
+class NlMsg:
+ def __init__(self, msg, offset, attr_space=None):
+ self.hdr = msg[offset:offset + 16]
+
+ self.nl_len, self.nl_type, self.nl_flags, self.nl_seq, self.nl_portid = \
+ struct.unpack("IHHII", self.hdr)
+
+ self.raw = msg[offset + 16:offset + self.nl_len]
+
+ self.error = 0
+ self.done = 0
+
+ extack_off = None
+ if self.nl_type == Netlink.NLMSG_ERROR:
+ self.error = struct.unpack("i", self.raw[0:4])[0]
+ self.done = 1
+ extack_off = 20
+ elif self.nl_type == Netlink.NLMSG_DONE:
+ self.done = 1
+ extack_off = 4
+
+ self.extack = None
+ if self.nl_flags & Netlink.NLM_F_ACK_TLVS and extack_off:
+ self.extack = dict()
+ extack_attrs = NlAttrs(self.raw[extack_off:])
+ for extack in extack_attrs:
+ if extack.type == Netlink.NLMSGERR_ATTR_MSG:
+ self.extack['msg'] = extack.as_strz()
+ elif extack.type == Netlink.NLMSGERR_ATTR_MISS_TYPE:
+ self.extack['miss-type'] = extack.as_u32()
+ elif extack.type == Netlink.NLMSGERR_ATTR_MISS_NEST:
+ self.extack['miss-nest'] = extack.as_u32()
+ elif extack.type == Netlink.NLMSGERR_ATTR_OFFS:
+ self.extack['bad-attr-offs'] = extack.as_u32()
+ else:
+ if 'unknown' not in self.extack:
+ self.extack['unknown'] = []
+ self.extack['unknown'].append(extack)
+
+ if attr_space:
+ # We don't have the ability to parse nests yet, so only do global
+ if 'miss-type' in self.extack and 'miss-nest' not in self.extack:
+ miss_type = self.extack['miss-type']
+ if len(attr_space.attr_list) > miss_type:
+ spec = attr_space.attr_list[miss_type]
+ desc = spec['name']
+ if 'doc' in spec:
+ desc += f" ({spec['doc']})"
+ self.extack['miss-type'] = desc
+
+ def __repr__(self):
+ msg = f"nl_len = {self.nl_len} ({len(self.raw)}) nl_flags = 0x{self.nl_flags:x} nl_type = {self.nl_type}\n"
+ if self.error:
+ msg += '\terror: ' + str(self.error)
+ if self.extack:
+ msg += '\textack: ' + repr(self.extack)
+ return msg
+
+
+class NlMsgs:
+ def __init__(self, data, attr_space=None):
+ self.msgs = []
+
+ offset = 0
+ while offset < len(data):
+ msg = NlMsg(data, offset, attr_space=attr_space)
+ offset += msg.nl_len
+ self.msgs.append(msg)
+
+ def __iter__(self):
+ yield from self.msgs
+
+
+genl_family_name_to_id = None
+
+
+def _genl_msg(nl_type, nl_flags, genl_cmd, genl_version, seq=None):
+ # we prepend length in _genl_msg_finalize()
+ if seq is None:
+ seq = random.randint(1, 1024)
+ nlmsg = struct.pack("HHII", nl_type, nl_flags, seq, 0)
+ genlmsg = struct.pack("bbH", genl_cmd, genl_version, 0)
+ return nlmsg + genlmsg
+
+
+def _genl_msg_finalize(msg):
+ return struct.pack("I", len(msg) + 4) + msg
+
+
+def _genl_load_families():
+ with socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, Netlink.NETLINK_GENERIC) as sock:
+ sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_CAP_ACK, 1)
+
+ msg = _genl_msg(Netlink.GENL_ID_CTRL,
+ Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK | Netlink.NLM_F_DUMP,
+ Netlink.CTRL_CMD_GETFAMILY, 1)
+ msg = _genl_msg_finalize(msg)
+
+ sock.send(msg, 0)
+
+ global genl_family_name_to_id
+ genl_family_name_to_id = dict()
+
+ while True:
+ reply = sock.recv(128 * 1024)
+ nms = NlMsgs(reply)
+ for nl_msg in nms:
+ if nl_msg.error:
+ print("Netlink error:", nl_msg.error)
+ return
+ if nl_msg.done:
+ return
+
+ gm = GenlMsg(nl_msg)
+ fam = dict()
+ for attr in gm.raw_attrs:
+ if attr.type == Netlink.CTRL_ATTR_FAMILY_ID:
+ fam['id'] = attr.as_u16()
+ elif attr.type == Netlink.CTRL_ATTR_FAMILY_NAME:
+ fam['name'] = attr.as_strz()
+ elif attr.type == Netlink.CTRL_ATTR_MAXATTR:
+ fam['maxattr'] = attr.as_u32()
+ elif attr.type == Netlink.CTRL_ATTR_MCAST_GROUPS:
+ fam['mcast'] = dict()
+ for entry in NlAttrs(attr.raw):
+ mcast_name = None
+ mcast_id = None
+ for entry_attr in NlAttrs(entry.raw):
+ if entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_NAME:
+ mcast_name = entry_attr.as_strz()
+ elif entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_ID:
+ mcast_id = entry_attr.as_u32()
+ if mcast_name and mcast_id is not None:
+ fam['mcast'][mcast_name] = mcast_id
+ if 'name' in fam and 'id' in fam:
+ genl_family_name_to_id[fam['name']] = fam
+
+
+class GenlMsg:
+ def __init__(self, nl_msg):
+ self.nl = nl_msg
+
+ self.hdr = nl_msg.raw[0:4]
+ self.raw = nl_msg.raw[4:]
+
+ self.genl_cmd, self.genl_version, _ = struct.unpack("bbH", self.hdr)
+
+ self.raw_attrs = NlAttrs(self.raw)
+
+ def __repr__(self):
+ msg = repr(self.nl)
+ msg += f"\tgenl_cmd = {self.genl_cmd} genl_ver = {self.genl_version}\n"
+ for a in self.raw_attrs:
+ msg += '\t\t' + repr(a) + '\n'
+ return msg
+
+
+class GenlFamily:
+ def __init__(self, family_name):
+ self.family_name = family_name
+
+ global genl_family_name_to_id
+ if genl_family_name_to_id is None:
+ _genl_load_families()
+
+ self.genl_family = genl_family_name_to_id[family_name]
+ self.family_id = genl_family_name_to_id[family_name]['id']
+
+
+#
+# YNL implementation details.
+#
+
+
+class YnlAttrSpace:
+ def __init__(self, family, yaml):
+ self.yaml = yaml
+
+ self.attrs = dict()
+ self.name = self.yaml['name']
+ self.subspace_of = self.yaml['subset-of'] if 'subspace-of' in self.yaml else None
+
+ val = 0
+ max_val = 0
+ for elem in self.yaml['attributes']:
+ if 'value' in elem:
+ val = elem['value']
+ else:
+ elem['value'] = val
+ if val > max_val:
+ max_val = val
+ val += 1
+
+ self.attrs[elem['name']] = elem
+
+ self.attr_list = [None] * (max_val + 1)
+ for elem in self.yaml['attributes']:
+ self.attr_list[elem['value']] = elem
+
+ def __getitem__(self, key):
+ return self.attrs[key]
+
+ def __contains__(self, key):
+ return key in self.yaml
+
+ def __iter__(self):
+ yield from self.attrs
+
+ def items(self):
+ return self.attrs.items()
+
+
+class YnlFamily:
+ def __init__(self, def_path, schema=None):
+ self.include_raw = False
+
+ with open(def_path, "r") as stream:
+ self.yaml = yaml.safe_load(stream)
+
+ if schema:
+ with open(schema, "r") as stream:
+ schema = yaml.safe_load(stream)
+
+ jsonschema.validate(self.yaml, schema)
+
+ self.sock = socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, Netlink.NETLINK_GENERIC)
+ self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_CAP_ACK, 1)
+ self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_EXT_ACK, 1)
+
+ self._ops = dict()
+ self._spaces = dict()
+ self._types = dict()
+
+ for elem in self.yaml['attribute-sets']:
+ self._spaces[elem['name']] = YnlAttrSpace(self, elem)
+
+ for elem in self.yaml['definitions']:
+ self._types[elem['name']] = elem
+
+ async_separation = 'async-prefix' in self.yaml['operations']
+ self.async_msg_ids = set()
+ self.async_msg_queue = []
+ val = 0
+ max_val = 0
+ for elem in self.yaml['operations']['list']:
+ if not (async_separation and ('notify' in elem or 'event' in elem)):
+ if 'value' in elem:
+ val = elem['value']
+ else:
+ elem['value'] = val
+ val += 1
+ max_val = max(val, max_val)
+
+ if 'notify' in elem or 'event' in elem:
+ self.async_msg_ids.add(elem['value'])
+
+ self._ops[elem['name']] = elem
+
+ op_name = elem['name'].replace('-', '_')
+
+ bound_f = functools.partial(self._op, elem['name'])
+ setattr(self, op_name, bound_f)
+
+ self._op_array = [None] * max_val
+ for _, op in self._ops.items():
+ self._op_array[op['value']] = op
+ if 'notify' in op:
+ op['attribute-set'] = self._ops[op['notify']]['attribute-set']
+
+ self.family = GenlFamily(self.yaml['name'])
+
+ def ntf_subscribe(self, mcast_name):
+ if mcast_name not in self.family.genl_family['mcast']:
+ raise Exception(f'Multicast group "{mcast_name}" not present in the family')
+
+ self.sock.bind((0, 0))
+ self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_ADD_MEMBERSHIP,
+ self.family.genl_family['mcast'][mcast_name])
+
+ def _add_attr(self, space, name, value):
+ attr = self._spaces[space][name]
+ nl_type = attr['value']
+ if attr["type"] == 'nest':
+ nl_type |= Netlink.NLA_F_NESTED
+ attr_payload = b''
+ for subname, subvalue in value.items():
+ attr_payload += self._add_attr(attr['nested-attributes'], subname, subvalue)
+ elif attr["type"] == 'u32':
+ attr_payload = struct.pack("I", int(value))
+ elif attr["type"] == 'string':
+ attr_payload = str(value).encode('ascii') + b'\x00'
+ elif attr["type"] == 'binary':
+ attr_payload = value
+ else:
+ raise Exception(f'Unknown type at {space} {name} {value} {attr["type"]}')
+
+ pad = b'\x00' * ((4 - len(attr_payload) % 4) % 4)
+ return struct.pack('HH', len(attr_payload) + 4, nl_type) + attr_payload + pad
+
+ def _decode_enum(self, rsp, attr_spec):
+ raw = rsp[attr_spec['name']]
+ enum = self._types[attr_spec['enum']]
+ i = attr_spec.get('value-start', 0)
+ if 'enum-as-flags' in attr_spec and attr_spec['enum-as-flags']:
+ value = set()
+ while raw:
+ if raw & 1:
+ value.add(enum['entries'][i])
+ raw >>= 1
+ i += 1
+ else:
+ value = enum['entries'][raw - i]
+ rsp[attr_spec['name']] = value
+
+ def _decode(self, attrs, space):
+ attr_space = self._spaces[space]
+ rsp = dict()
+ for attr in attrs:
+ attr_spec = attr_space.attr_list[attr.type]
+ if attr_spec["type"] == 'nest':
+ subdict = self._decode(NlAttrs(attr.raw), attr_spec['nested-attributes'])
+ rsp[attr_spec['name']] = subdict
+ elif attr_spec['type'] == 'u32':
+ rsp[attr_spec['name']] = attr.as_u32()
+ elif attr_spec['type'] == 'u64':
+ rsp[attr_spec['name']] = attr.as_u64()
+ elif attr_spec["type"] == 'string':
+ rsp[attr_spec['name']] = attr.as_strz()
+ elif attr_spec["type"] == 'binary':
+ rsp[attr_spec['name']] = attr.as_bin()
+ else:
+ raise Exception(f'Unknown {attr.type} {attr_spec["name"]} {attr_spec["type"]}')
+
+ if 'enum' in attr_spec:
+ self._decode_enum(rsp, attr_spec)
+ return rsp
+
+ def handle_ntf(self, nl_msg, genl_msg):
+ msg = dict()
+ if self.include_raw:
+ msg['nlmsg'] = nl_msg
+ msg['genlmsg'] = genl_msg
+ op = self._op_array[genl_msg.genl_cmd]
+ msg['name'] = op['name']
+ msg['msg'] = self._decode(genl_msg.raw_attrs, op['attribute-set'])
+ self.async_msg_queue.append(msg)
+
+ def check_ntf(self):
+ while True:
+ try:
+ reply = self.sock.recv(128 * 1024, socket.MSG_DONTWAIT)
+ except BlockingIOError:
+ return
+
+ nms = NlMsgs(reply)
+ for nl_msg in nms:
+ if nl_msg.error:
+ print("Netlink error in ntf!?", os.strerror(-nl_msg.error))
+ print(nl_msg)
+ continue
+ if nl_msg.done:
+ print("Netlink done while checking for ntf!?")
+ continue
+
+ gm = GenlMsg(nl_msg)
+ if gm.genl_cmd not in self.async_msg_ids:
+ print("Unexpected msg id done while checking for ntf", gm)
+ continue
+
+ self.handle_ntf(nl_msg, gm)
+
+ def _op(self, method, vals, dump=False):
+ op = self._ops[method]
+
+ nl_flags = Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK
+ if dump:
+ nl_flags |= Netlink.NLM_F_DUMP
+
+ req_seq = random.randint(1024, 65535)
+ msg = _genl_msg(self.family.family_id, nl_flags, op['value'], 1, req_seq)
+ for name, value in vals.items():
+ msg += self._add_attr(op['attribute-set'], name, value)
+ msg = _genl_msg_finalize(msg)
+
+ self.sock.send(msg, 0)
+
+ done = False
+ rsp = []
+ while not done:
+ reply = self.sock.recv(128 * 1024)
+ nms = NlMsgs(reply, attr_space=self._spaces[op['attribute-set']])
+ for nl_msg in nms:
+ if nl_msg.error:
+ print("Netlink error:", os.strerror(-nl_msg.error))
+ print(nl_msg)
+ return
+ if nl_msg.done:
+ done = True
+ break
+
+ gm = GenlMsg(nl_msg)
+ # Check if this is a reply to our request
+ if nl_msg.nl_seq != req_seq or gm.genl_cmd != op['value']:
+ if gm.genl_cmd in self.async_msg_ids:
+ self.handle_ntf(nl_msg, gm)
+ continue
+ else:
+ print('Unexpected message: ' + repr(gm))
+ continue
+
+ rsp.append(self._decode(gm.raw_attrs, op['attribute-set']))
+
+ if not rsp:
+ return None
+ if not dump and len(rsp) == 1:
+ return rsp[0]
+ return rsp
--
2.39.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs
2023-01-19 0:36 ` [PATCH net-next v3 2/8] netlink: add schemas for YAML specs Jakub Kicinski
@ 2023-01-19 14:07 ` Rob Herring
2023-01-19 21:49 ` Jakub Kicinski
2023-01-20 0:08 ` Jakub Kicinski
0 siblings, 2 replies; 31+ messages in thread
From: Rob Herring @ 2023-01-19 14:07 UTC (permalink / raw
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Wed, Jan 18, 2023 at 6:36 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> Add schemas for Netlink spec files. As described in the docs
> we have 4 "protocols" or compatibility levels, and each one
> comes with its own schema, but the more general / legacy
> schemas are superset of more modern ones: genetlink is
> the smallest followed by genetlink-c and genetlink-legacy.
> There is no schema for raw netlink, yet, I haven't found the time..
>
> I don't know enough jsonschema to do inheritance or something
> but the repetition is not too bad. I hope.
Generally you put common schemas under '$defs' and the then reference
them with '$ref'.
$defs:
some-prop-type:
type: integer
minimum: 0
properties:
foo:
$ref: '#/$defs/some-prop-type'
bar:
$ref: '#/$defs/some-prop-type'
If you have objects with common sets of properties, you can do the
same thing, but then you need 'unevaluatedProperties' if you want to
define a base set of properties and add to them. We do that frequently
in DT schemas. Unlike typical inheritance, you can't override the
'base' schema. It's an AND operation.
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> Documentation/netlink/genetlink-c.yaml | 318 ++++++++++++++++++
> Documentation/netlink/genetlink-legacy.yaml | 346 ++++++++++++++++++++
> Documentation/netlink/genetlink.yaml | 284 ++++++++++++++++
> 3 files changed, 948 insertions(+)
> create mode 100644 Documentation/netlink/genetlink-c.yaml
> create mode 100644 Documentation/netlink/genetlink-legacy.yaml
> create mode 100644 Documentation/netlink/genetlink.yaml
>
> diff --git a/Documentation/netlink/genetlink-c.yaml b/Documentation/netlink/genetlink-c.yaml
> new file mode 100644
> index 000000000000..53a7fe94a47e
> --- /dev/null
> +++ b/Documentation/netlink/genetlink-c.yaml
> @@ -0,0 +1,318 @@
> +# SPDX-License-Identifier: GPL-2.0
> +%YAML 1.2
> +---
> +$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#"
> +$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
There's no core.yaml. If you don't have a custom meta-schema, then
just set this to the schema for the json-schema version you are using.
Then the tools can validate the schemas without your own validator
class.
Also, you can drop quotes on these.
> +
> +title: Protocol
> +description: Specification of a genetlink protocol
> +type: object
> +required: [ name, doc, attribute-sets, operations ]
> +additionalProperties: False
> +properties:
> + name:
> + description: Name of the genetlink family.
> + type: string
> + doc:
> + type: string
> + version:
> + description: Generic Netlink family version. Default is 1.
> + type: integer
Constraints?
minimum: 0
Same on all other integers.
> + protocol:
> + description: Schema compatibility level. Default is "genetlink".
> + enum: [ genetlink, genetlink-c]
> + # Start genetlink-c
> + uapi-header:
> + description: Path to the uAPI header, default is linux/${family-name}.h
> + type: string
> + c-family-name:
> + description: Name of the define for the family name.
> + type: string
> + c-version-name:
> + description: Name of the define for the verion of the family.
typo
> + type: string
> + max-by-define:
> + description: Makes the number of attributes and commands be specified by a define, not an enum value.
> + type: boolean
> + # End genetlink-c
> +
> + definitions:
> + description: Array of type and constant definitions.
> + type: array
> + items:
> + type: object
> + required: [ type, name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + header:
> + description: For C-compatible languages, header which already defines this value.
> + type: string
> + type:
> + enum: [ const, enum, flags ]
> + doc:
> + type: string
> + # For const
> + value:
> + description: For const - the value.
> + type: [ string, integer ]
> + # For enum and flags
> + value-start:
> + description: For enum or flags the literal initializer for the first value.
> + type: [ string, integer ]
> + entries:
> + description: For enum or flags array of values
> + type: array
> + items:
> + oneOf:
> + - type: string
> + - type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + value:
> + type: integer
> + doc:
> + type: string
> + render-max:
> + description: Render the max members for this enum.
> + type: boolean
> + # Start genetlink-c
> + enum-name:
> + description: Name for enum, if empty no name will be used.
> + type: [ string, "null" ]
> + name-prefix:
> + description: For enum the prefix of the values, optional.
> + type: string
> + # End genetlink-c
> +
> + attribute-sets:
> + description: Definition of attribute spaces for this family.
> + type: array
> + items:
> + description: Definition of a single attribute space.
> + type: object
> + required: [ name, attributes ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + Name used when referring to this space in other definitions, not used outside of YAML.
> + type: string
> + # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
If one is required:
oneOf:
- required: [ name-prefix ]
- required: [ subset-of ]
Or if both are optional:
dependencies:
name-prefix:
not:
required: [ subset-of ]
subset-of:
not:
required: [ name-prefix ]
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
> + type: string
> + enum-name:
> + description: Name for the enum type of the attribute.
> + type: string
> + doc:
> + description: Documentation of the space.
> + type: string
> + subset-of:
> + description: |
> + Name of another space which this is a logical part of. Sub-spaces can be used to define
> + a limited group of attributes which are used in a nest.
> + type: string
> + # Start genetlink-c
> + attr-cnt-name:
> + description: The explicit name for constant holding the count of attributes (last attr + 1).
> + type: string
> + attr-max-name:
> + description: The explicit name for last member of attribute enum.
> + type: string
> + # End genetlink-c
> + attributes:
> + description: List of attributes in the space.
> + type: array
> + items:
> + type: object
> + required: [ name, type ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + type: &attr-type
> + enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
> + string, nest, array-nest, nest-type-value ]
> + doc:
> + description: Documentation of the attribute.
> + type: string
> + value:
> + description: Value for the enum item representing this attribute in the uAPI.
> + type: integer
> + type-value:
> + description: Name of the value extracted from the type of a nest-type-value attribute.
> + type: array
> + items:
> + type: string
> + byte-order:
> + enum: [little-endian, big-endian]
> + multi-attr:
> + type: boolean
> + nested-attributes:
> + description: Name of the space (sub-space) used inside the attribute.
> + type: string
> + enum:
> + description: Name of the enum type used for the atttribute.
typo
> + type: string
> + enum-as-flags:
> + description: |
> + Treat the enum as flags. In most cases enum is either used as flags or as values.
> + Sometimes, however, both forms are necessary, in which case header contains the enum
> + form while specific attributes may request to convert the values into a bitfield.
> + type: boolean
> + checks:
> + description: Kernel input validation.
> + type: object
> + additionalProperties: False
> + properties:
> + flags-mask:
> + description: Name of the flags constant on which to base mask (unsigned scalar types only).
> + type: string
> + min:
> + description: Min value for an integer attribute.
> + type: integer
> + min-len:
> + description: Min length for a binary attribute.
> + oneOf:
> + - type: string
> + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> + - type: integer
How can a length be a string?
Anyways, this is something you could pull out into a $defs entry and
reference. It will also work without the oneOf because 'pattern' will
just be ignored for an integer. That's one gotcha with json-schema. If
a keyword doesn't apply to the instance, it is silently ignored. (That
includes unknown keywords such as ones with typos. Fun!). 'oneOf' will
give you pretty crappy error messages, so it's good to avoid when
possible.
> + max-len:
> + description: Max length for a string or a binary attribute.
> + oneOf:
> + - type: string
> + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> + - type: integer
> + sub-type: *attr-type
> + operations:
> + description: Operations supported by the protocol.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + enum-model:
> + description: |
> + The model of assigning values to the operations.
> + "unified" is the recommended model where all message types belong
> + to a single enum.
> + "directional" has the messages sent to the kernel and from the kernel
> + enumerated separately.
> + "notify-split" has the notifications and request-response types in
> + different enums.
> + enum: [ unified, directional, notify-split ]
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the command. The name is formed by concatenating
> + the prefix with the upper case name of the command, with dashes replaced by underscores.
> + type: string
> + enum-name:
> + description: Name for the enum type with commands.
> + type: string
> + async-prefix:
> + description: Same as name-prefix but used to render notifications and events to separate enum.
> + type: string
> + async-enum:
> + description: Name for the enum type with notifications/events.
> + type: string
> + list:
> + description: List of commands
> + type: array
> + items:
> + type: object
> + additionalProperties: False
> + required: [ name, doc ]
> + properties:
> + name:
> + description: Name of the operation, also defining its C enum value in uAPI.
> + type: string
> + doc:
> + description: Documentation for the command.
> + type: string
> + value:
> + description: Value for the enum in the uAPI.
> + type: integer
> + attribute-set:
> + description: |
> + Attribute space from which attributes directly in the requests and replies
> + to this command are defined.
> + type: string
> + flags: &cmd_flags
> + description: Command flags.
> + type: array
> + items:
> + enum: [ admin-perm ]
> + dont-validate:
> + description: Kernel attribute validation flags.
> + type: array
> + items:
> + enum: [ strict, dump ]
> + do: &subop-type
> + description: Main command handler.
> + type: object
> + additionalProperties: False
> + properties:
> + request: &subop-attr-list
> + description: Definition of the request message for a given command.
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: |
> + Names of attributes from the attribute-set (not full attribute
> + definitions, just names).
> + type: array
> + items:
> + type: string
> + reply: *subop-attr-list
> + pre:
> + description: Hook for a function to run before the main callback (pre_doit or start).
> + type: string
> + post:
> + description: Hook for a function to run after the main callback (post_doit or done).
> + type: string
> + dump: *subop-type
> + notify:
> + description: Name of the command sharing the reply type with this notification.
> + type: string
> + event:
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: Explicit list of the attributes for the notification.
> + type: array
> + items:
> + type: string
> + mcgrp:
> + description: Name of the multicast group generating given notification.
> + type: string
> + mcast-groups:
> + description: List of multicast groups.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + list:
> + description: List of groups.
> + type: array
> + items:
> + type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + The name for the group, used to form the define and the value of the define.
> + type: string
> + # Start genetlink-c
> + c-define-name:
> + description: Override for the name of the define in C uAPI.
> + type: string
> + # End genetlink-c
> + flags: *cmd_flags
> diff --git a/Documentation/netlink/genetlink-legacy.yaml b/Documentation/netlink/genetlink-legacy.yaml
> new file mode 100644
> index 000000000000..5e57453eac1a
> --- /dev/null
> +++ b/Documentation/netlink/genetlink-legacy.yaml
> @@ -0,0 +1,346 @@
> +# SPDX-License-Identifier: GPL-2.0
> +%YAML 1.2
> +---
> +$id: "http://kernel.org/schemas/netlink/genetlink-legacy.yaml#"
> +$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
> +
> +title: Protocol
> +description: Specification of a genetlink protocol
> +type: object
> +required: [ name, doc, attribute-sets, operations ]
> +additionalProperties: False
> +properties:
> + name:
> + description: Name of the genetlink family.
> + type: string
> + doc:
> + type: string
> + version:
> + description: Generic Netlink family version. Default is 1.
> + type: integer
> + protocol:
> + description: Schema compatibility level. Default is "genetlink".
> + enum: [ genetlink, genetlink-c, genetlink-legacy, netlink-raw ] # Trim
> + # Start genetlink-c
> + uapi-header:
> + description: Path to the uAPI header, default is linux/${family-name}.h
> + type: string
> + c-family-name:
> + description: Name of the define for the family name.
> + type: string
> + c-version-name:
> + description: Name of the define for the verion of the family.
typo
> + type: string
> + max-by-define:
> + description: Makes the number of attributes and commands be specified by a define, not an enum value.
> + type: boolean
> + # End genetlink-c
> + # Start genetlink-legacy
> + kernel-policy:
> + description: |
> + Defines if the input policy in the kernel is global, per-operation, or split per operation type.
> + Default is split.
> + enum: [split, per-op, global]
> + # End genetlink-legacy
> +
> + definitions:
> + description: Array of type and constant definitions.
> + type: array
> + items:
> + type: object
> + required: [ type, name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + header:
> + description: For C-compatible languages, header which already defines this value.
> + type: string
> + type:
> + enum: [ const, enum, flags, struct ] # Trim
> + doc:
> + type: string
> + # For const
> + value:
> + description: For const - the value.
> + type: [ string, integer ]
> + # For enum and flags
> + value-start:
> + description: For enum or flags the literal initializer for the first value.
> + type: [ string, integer ]
> + entries:
> + description: For enum or flags array of values
> + type: array
> + items:
> + oneOf:
> + - type: string
> + - type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + value:
> + type: integer
> + doc:
> + type: string
> + render-max:
> + description: Render the max members for this enum.
> + type: boolean
> + # Start genetlink-c
> + enum-name:
> + description: Name for enum, if empty no name will be used.
> + type: [ string, "null" ]
> + name-prefix:
> + description: For enum the prefix of the values, optional.
> + type: string
> + # End genetlink-c
> + # Start genetlink-legacy
> + members:
> + description: List of struct members. Only scalars and strings members allowed.
> + type: array
> + items:
> + type: object
> + required: [ name, type ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + type:
> + enum: [ u8, u16, u32, u64, s8, s16, s32, s64, string ]
> + len:
> + oneOf:
> + -
> + type: string
> + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> + -
> + type: integer
Inconsistent style, but this is the subschema you can factor out.
> + # End genetlink-legacy
> +
> + attribute-sets:
> + description: Definition of attribute spaces for this family.
> + type: array
> + items:
> + description: Definition of a single attribute space.
> + type: object
> + required: [ name, attributes ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + Name used when referring to this space in other definitions, not used outside of YAML.
> + type: string
> + # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
> + type: string
> + enum-name:
> + description: Name for the enum type of the attribute.
> + type: string
> + doc:
> + description: Documentation of the space.
> + type: string
> + subset-of:
> + description: |
> + Name of another space which this is a logical part of. Sub-spaces can be used to define
> + a limited group of attributes which are used in a nest.
> + type: string
> + # Start genetlink-c
> + attr-cnt-name:
> + description: The explicit name for constant holding the count of attributes (last attr + 1).
> + type: string
> + attr-max-name:
> + description: The explicit name for last member of attribute enum.
> + type: string
> + # End genetlink-c
> + attributes:
> + description: List of attributes in the space.
> + type: array
> + items:
> + type: object
> + required: [ name, type ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + type: &attr-type
> + enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
> + string, nest, array-nest, nest-type-value ]
> + doc:
> + description: Documentation of the attribute.
> + type: string
> + value:
> + description: Value for the enum item representing this attribute in the uAPI.
> + type: integer
> + type-value:
> + description: Name of the value extracted from the type of a nest-type-value attribute.
> + type: array
> + items:
> + type: string
> + byte-order:
> + enum: [little-endian, big-endian]
> + multi-attr:
> + type: boolean
> + nested-attributes:
> + description: Name of the space (sub-space) used inside the attribute.
> + type: string
> + enum:
> + description: Name of the enum type used for the atttribute.
> + type: string
> + enum-as-flags:
> + description: |
> + Treat the enum as flags. In most cases enum is either used as flags or as values.
> + Sometimes, however, both forms are necessary, in which case header contains the enum
> + form while specific attributes may request to convert the values into a bitfield.
> + type: boolean
> + checks:
> + description: Kernel input validation.
> + type: object
> + additionalProperties: False
> + properties:
> + flags-mask:
> + description: Name of the flags constant on which to base mask (unsigned scalar types only).
> + type: string
> + min:
> + description: Min value for an integer attribute.
> + type: integer
> + min-len:
> + description: Min length for a binary attribute.
> + oneOf:
> + - type: string
> + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> + - type: integer
> + max-len:
> + description: Max length for a string or a binary attribute.
> + oneOf:
> + - type: string
> + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> + - type: integer
> + sub-type: *attr-type
> + operations:
> + description: Operations supported by the protocol.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + enum-model:
> + description: |
> + The model of assigning values to the operations.
> + "unified" is the recommended model where all message types belong
> + to a single enum.
> + "directional" has the messages sent to the kernel and from the kernel
> + enumerated separately.
> + "notify-split" has the notifications and request-response types in
> + different enums.
> + enum: [ unified, directional, notify-split ]
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the command. The name is formed by concatenating
> + the prefix with the upper case name of the command, with dashes replaced by underscores.
> + type: string
> + enum-name:
> + description: Name for the enum type with commands.
> + type: string
> + async-prefix:
> + description: Same as name-prefix but used to render notifications and events to separate enum.
> + type: string
> + async-enum:
> + description: Name for the enum type with notifications/events.
> + type: string
> + list:
> + description: List of commands
> + type: array
> + items:
> + type: object
> + additionalProperties: False
> + required: [ name, doc ]
> + properties:
> + name:
> + description: Name of the operation, also defining its C enum value in uAPI.
> + type: string
> + doc:
> + description: Documentation for the command.
> + type: string
> + value:
> + description: Value for the enum in the uAPI.
> + type: integer
> + attribute-set:
> + description: |
> + Attribute space from which attributes directly in the requests and replies
> + to this command are defined.
> + type: string
> + flags: &cmd_flags
> + description: Command flags.
> + type: array
> + items:
> + enum: [ admin-perm ]
> + dont-validate:
> + description: Kernel attribute validation flags.
> + type: array
> + items:
> + enum: [ strict, dump ]
> + do: &subop-type
> + description: Main command handler.
> + type: object
> + additionalProperties: False
> + properties:
> + request: &subop-attr-list
> + description: Definition of the request message for a given command.
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: |
> + Names of attributes from the attribute-set (not full attribute
> + definitions, just names).
> + type: array
> + items:
> + type: string
> + reply: *subop-attr-list
> + pre:
> + description: Hook for a function to run before the main callback (pre_doit or start).
> + type: string
> + post:
> + description: Hook for a function to run after the main callback (post_doit or done).
> + type: string
> + dump: *subop-type
> + notify:
> + description: Name of the command sharing the reply type with this notification.
> + type: string
> + event:
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: Explicit list of the attributes for the notification.
> + type: array
> + items:
> + type: string
> + mcgrp:
> + description: Name of the multicast group generating given notification.
> + type: string
> + mcast-groups:
> + description: List of multicast groups.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + list:
> + description: List of groups.
> + type: array
> + items:
> + type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + The name for the group, used to form the define and the value of the define.
> + type: string
> + # Start genetlink-c
> + c-define-name:
> + description: Override for the name of the define in C uAPI.
> + type: string
> + # End genetlink-c
> + flags: *cmd_flags
> diff --git a/Documentation/netlink/genetlink.yaml b/Documentation/netlink/genetlink.yaml
> new file mode 100644
> index 000000000000..aafaaaabcfbe
> --- /dev/null
> +++ b/Documentation/netlink/genetlink.yaml
> @@ -0,0 +1,284 @@
> +# SPDX-License-Identifier: GPL-2.0
> +%YAML 1.2
> +---
> +$id: "http://kernel.org/schemas/netlink/genetlink.yaml#"
> +$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
> +
> +title: Protocol
> +description: Specification of a genetlink protocol
> +type: object
> +required: [ name, doc, attribute-sets, operations ]
> +additionalProperties: False
> +properties:
> + name:
> + description: Name of the genetlink family.
> + type: string
> + doc:
> + type: string
> + version:
> + description: Generic Netlink family version. Default is 1.
> + type: integer
> + protocol:
> + description: Schema compatibility level. Default is "genetlink".
> + enum: [ genetlink ]
> +
> + definitions:
> + description: Array of type and constant definitions.
> + type: array
> + items:
> + type: object
> + required: [ type, name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + header:
> + description: For C-compatible languages, header which already defines this value.
> + type: string
> + type:
> + enum: [ const, enum, flags ]
> + doc:
> + type: string
> + # For const
> + value:
> + description: For const - the value.
> + type: [ string, integer ]
> + # For enum and flags
> + value-start:
> + description: For enum or flags the literal initializer for the first value.
> + type: [ string, integer ]
> + entries:
> + description: For enum or flags array of values
> + type: array
> + items:
> + oneOf:
> + - type: string
> + - type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + value:
> + type: integer
> + doc:
> + type: string
> +
> + render-max:
> + description: Render the max members for this enum.
> + type: boolean
> +
> + attribute-sets:
> + description: Definition of attribute spaces for this family.
> + type: array
> + items:
> + description: Definition of a single attribute space.
> + type: object
> + required: [ name, attributes ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + Name used when referring to this space in other definitions, not used outside of YAML.
> + type: string
> + # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
> + type: string
> + enum-name:
> + description: Name for the enum type of the attribute.
> + type: string
> + doc:
> + description: Documentation of the space.
> + type: string
> + subset-of:
> + description: |
> + Name of another space which this is a logical part of. Sub-spaces can be used to define
> + a limited group of attributes which are used in a nest.
> + type: string
> + attributes:
> + description: List of attributes in the space.
> + type: array
> + items:
> + type: object
> + required: [ name, type ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + type: &attr-type
> + enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
> + string, nest, array-nest, nest-type-value ]
> + doc:
> + description: Documentation of the attribute.
> + type: string
> + value:
> + description: Value for the enum item representing this attribute in the uAPI.
> + type: integer
> + type-value:
> + description: Name of the value extracted from the type of a nest-type-value attribute.
> + type: array
> + items:
> + type: string
> + byte-order:
> + enum: [little-endian, big-endian]
> + multi-attr:
> + type: boolean
> + nested-attributes:
> + description: Name of the space (sub-space) used inside the attribute.
> + type: string
> + enum:
> + description: Name of the enum type used for the atttribute.
> + type: string
> + enum-as-flags:
> + description: |
> + Treat the enum as flags. In most cases enum is either used as flags or as values.
> + Sometimes, however, both forms are necessary, in which case header contains the enum
> + form while specific attributes may request to convert the values into a bitfield.
> + type: boolean
> + checks:
> + description: Kernel input validation.
> + type: object
> + additionalProperties: False
> + properties:
> + flags-mask:
> + description: Name of the flags constant on which to base mask (unsigned scalar types only).
> + type: string
> + min:
> + description: Min value for an integer attribute.
> + type: integer
> + min-len:
> + description: Min length for a binary attribute.
> + oneOf:
> + - type: string
> + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> + - type: integer
> + max-len:
> + description: Max length for a string or a binary attribute.
> + oneOf:
> + - type: string
> + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> + - type: integer
> + sub-type: *attr-type
> + operations:
> + description: Operations supported by the protocol.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + enum-model:
> + description: |
> + The model of assigning values to the operations.
> + "unified" is the recommended model where all message types belong
> + to a single enum.
> + "directional" has the messages sent to the kernel and from the kernel
> + enumerated separately.
> + "notify-split" has the notifications and request-response types in
> + different enums.
> + enum: [ unified, directional, notify-split ]
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the command. The name is formed by concatenating
> + the prefix with the upper case name of the command, with dashes replaced by underscores.
> + type: string
> + enum-name:
> + description: Name for the enum type with commands.
> + type: string
> + async-prefix:
> + description: Same as name-prefix but used to render notifications and events to separate enum.
> + type: string
> + async-enum:
> + description: Name for the enum type with notifications/events.
> + type: string
> + list:
> + description: List of commands
> + type: array
> + items:
> + type: object
> + additionalProperties: False
> + required: [ name, doc ]
> + properties:
> + name:
> + description: Name of the operation, also defining its C enum value in uAPI.
> + type: string
> + doc:
> + description: Documentation for the command.
> + type: string
> + value:
> + description: Value for the enum in the uAPI.
> + type: integer
> + attribute-set:
> + description: |
> + Attribute space from which attributes directly in the requests and replies
> + to this command are defined.
> + type: string
> + flags: &cmd_flags
> + description: Command flags.
> + type: array
> + items:
> + enum: [ admin-perm ]
> + dont-validate:
> + description: Kernel attribute validation flags.
> + type: array
> + items:
> + enum: [ strict, dump ]
> + do: &subop-type
> + description: Main command handler.
> + type: object
> + additionalProperties: False
> + properties:
> + request: &subop-attr-list
> + description: Definition of the request message for a given command.
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: |
> + Names of attributes from the attribute-set (not full attribute
> + definitions, just names).
> + type: array
> + items:
> + type: string
> + reply: *subop-attr-list
> + pre:
> + description: Hook for a function to run before the main callback (pre_doit or start).
> + type: string
> + post:
> + description: Hook for a function to run after the main callback (post_doit or done).
> + type: string
> + dump: *subop-type
> + notify:
> + description: Name of the command sharing the reply type with this notification.
> + type: string
> + event:
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: Explicit list of the attributes for the notification.
> + type: array
> + items:
> + type: string
> + mcgrp:
> + description: Name of the multicast group generating given notification.
> + type: string
> + mcast-groups:
> + description: List of multicast groups.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + list:
> + description: List of groups.
> + type: array
> + items:
> + type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + The name for the group, used to form the define and the value of the define.
> + type: string
> + flags: *cmd_flags
> --
> 2.39.0
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-19 0:36 ` [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs) Jakub Kicinski
@ 2023-01-19 15:48 ` Vladimir Oltean
2023-01-19 20:29 ` Johannes Berg
1 sibling, 0 replies; 31+ messages in thread
From: Vladimir Oltean @ 2023-01-19 15:48 UTC (permalink / raw
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, robh, johannes, stephen,
ecree.xilinx, sdf, f.fainelli, fw, linux-doc, razor,
nicolas.dichtel, Bagas Sanjaya
On Wed, Jan 18, 2023 at 04:36:06PM -0800, Jakub Kicinski wrote:
> Add documentation about the upcoming Netlink protocol specs.
>
> Reviewed-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
> Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> --
I considered this an useful read, thanks.
Reviewed-by: Vladimir Oltean <olteanv@gmail.com>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 0/8] Netlink protocol specs
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
` (7 preceding siblings ...)
2023-01-19 0:36 ` [PATCH net-next v3 8/8] tools: ynl: add a completely generic client Jakub Kicinski
@ 2023-01-19 17:07 ` Stanislav Fomichev
8 siblings, 0 replies; 31+ messages in thread
From: Stanislav Fomichev @ 2023-01-19 17:07 UTC (permalink / raw
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, robh, johannes, stephen,
ecree.xilinx, f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Wed, Jan 18, 2023 at 4:36 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> I think the Netlink proto specs are far along enough to merge.
> Filling in all attribute types and quirks will be an ongoing
> effort but we have enough to cover FOU so it's somewhat complete.
>
> I fully intend to continue polishing the code but at the same
> time I'd like to start helping others base their work on the
> specs (e.g. DPLL) and need to start working on some new families
> myself.
>
> That's the progress / motivation for merging. The RFC [1] has more
> of a high level blurb, plus I created a lot of documentation, I'm
> not going to repeat it here. There was also the talk at LPC [2].
>
> [1] https://lore.kernel.org/all/20220811022304.583300-1-kuba@kernel.org/
> [2] https://youtu.be/9QkXIQXkaQk?t=2562
> v2: https://lore.kernel.org/all/20220930023418.1346263-1-kuba@kernel.org/
I think this is awesome and is definitely a step in the right
direction. I did a light review, nothing pops up, thanks for the spec
fields renamings.
If that helps feel free to attach:
Acked-by: Stanislav Fomichev <sdf@google.com>
> Jakub Kicinski (8):
> docs: add more netlink docs (incl. spec docs)
> netlink: add schemas for YAML specs
> net: add basic C code generators for Netlink
> netlink: add a proto specification for FOU
> net: fou: regenerate the uAPI from the spec
> net: fou: rename the source for linking
> net: fou: use policy and operation tables generated from the spec
> tools: ynl: add a completely generic client
>
> Documentation/core-api/index.rst | 1 +
> Documentation/core-api/netlink.rst | 99 +
> Documentation/netlink/genetlink-c.yaml | 318 +++
> Documentation/netlink/genetlink-legacy.yaml | 346 +++
> Documentation/netlink/genetlink.yaml | 284 ++
> Documentation/netlink/specs/fou.yaml | 128 +
> .../userspace-api/netlink/c-code-gen.rst | 107 +
> .../netlink/genetlink-legacy.rst | 96 +
> Documentation/userspace-api/netlink/index.rst | 5 +
> Documentation/userspace-api/netlink/specs.rst | 422 +++
> MAINTAINERS | 3 +
> include/uapi/linux/fou.h | 54 +-
> net/ipv4/Makefile | 1 +
> net/ipv4/fou-nl.c | 48 +
> net/ipv4/fou-nl.h | 25 +
> net/ipv4/{fou.c => fou_core.c} | 47 +-
> tools/net/ynl/samples/cli.py | 47 +
> tools/net/ynl/samples/ynl.py | 534 ++++
> tools/net/ynl/ynl-gen-c.py | 2376 +++++++++++++++++
> tools/net/ynl/ynl-regen.sh | 30 +
> 20 files changed, 4903 insertions(+), 68 deletions(-)
> create mode 100644 Documentation/core-api/netlink.rst
> create mode 100644 Documentation/netlink/genetlink-c.yaml
> create mode 100644 Documentation/netlink/genetlink-legacy.yaml
> create mode 100644 Documentation/netlink/genetlink.yaml
> create mode 100644 Documentation/netlink/specs/fou.yaml
> create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst
> create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst
> create mode 100644 Documentation/userspace-api/netlink/specs.rst
> create mode 100644 net/ipv4/fou-nl.c
> create mode 100644 net/ipv4/fou-nl.h
> rename net/ipv4/{fou.c => fou_core.c} (94%)
> create mode 100755 tools/net/ynl/samples/cli.py
> create mode 100644 tools/net/ynl/samples/ynl.py
> create mode 100755 tools/net/ynl/ynl-gen-c.py
> create mode 100755 tools/net/ynl/ynl-regen.sh
>
> --
> 2.39.0
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-19 0:36 ` [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs) Jakub Kicinski
2023-01-19 15:48 ` Vladimir Oltean
@ 2023-01-19 20:29 ` Johannes Berg
2023-01-20 0:23 ` Jacob Keller
2023-01-20 2:13 ` Jakub Kicinski
1 sibling, 2 replies; 31+ messages in thread
From: Johannes Berg @ 2023-01-19 20:29 UTC (permalink / raw
To: Jakub Kicinski, davem
Cc: netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel, Bagas Sanjaya
On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
>
> +Answer requests
> +---------------
> +
> +Older families do not reply to all of the commands, especially NEW / ADD
> +commands. User only gets information whether the operation succeeded or
> +not via the ACK. Try to find useful data to return. Once the command is
> +added whether it replies with a full message or only an ACK is uAPI and
> +cannot be changed. It's better to err on the side of replying.
> +
> +Specifically NEW and ADD commands should reply with information identifying
> +the created object such as the allocated object's ID (without having to
> +resort to using ``NLM_F_ECHO``).
I'm a bit on the fence on this recommendation (as written).
Yeah, it's nice to reply to things ... but!
In userspace, you often request and wait for the ACK to see if the
operation succeeded. This is basically necessary. But then it's
complicated to wait for *another* message to see the ID.
We've actually started using the "cookie" in the extack to report an ID
of an object/... back, see uses of nl_set_extack_cookie_u64() in the
tree.
So I'm not sure I wholeheartedly agree with the recommendation to send a
separate answer. We've done that, but it's ugly on both sender side in
the kernel (requiring an extra message allocation, ideally at the
beginning of the operation so you can fail gracefully, etc.) and on the
receiver (having to wait for another message if the operation was
successful; possibly actually having to check for that message *before*
the ACK arrives.)
> +Support dump consistency
> +------------------------
> +
> +If iterating over objects during dump may skip over objects or repeat
> +them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.
That could be a bit more fleshed out on _how_ to do that, if it's not
somewhere else?
> +kernel-policy
> +~~~~~~~~~~~~~
> +
> +Defines if the kernel validation policy is per operation (``per-op``)
> +or for the entire family (``global``). New families should use ``per-op``
> +(default) to be able to narrow down the attributes accepted by a specific
> +command.
Again I'm not sure I agree with that recommendation, but I know it's
your preference :-)
(IMHO some things become more complex, such as having a "ifindex" in
each one of them)
> +checks
> +------
> +
> +Documentation for the ``checks`` sub-sections of attribute specs.
> +
> +unterminated-ok
> +~~~~~~~~~~~~~~~
> +
> +Accept strings without the null-termination (for legacy families only).
> +Switches from the ``NLA_NUL_STRING`` to ``NLA_STRING`` policy type.
Should we even document all the legacy bits in such a prominent place?
(or just move it after max-len/min-len?)
> +Attribute type nests
> +--------------------
> +
> +New Netlink families should use ``multi-attr`` to define arrays.
Unrelated to this particular document, but ...
I'm all for this, btw, but maybe we should have a way of representing in
the policy that an attribute is used as multi-attr for an array, and a
way of exposing that in the policy export? Hmm. Haven't thought about
this for a while.
johannes
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 3/8] net: add basic C code generators for Netlink
2023-01-19 0:36 ` [PATCH net-next v3 3/8] net: add basic C code generators for Netlink Jakub Kicinski
@ 2023-01-19 20:53 ` Johannes Berg
2023-01-20 1:53 ` Jakub Kicinski
0 siblings, 1 reply; 31+ messages in thread
From: Johannes Berg @ 2023-01-19 20:53 UTC (permalink / raw
To: Jakub Kicinski, davem
Cc: netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
>
> +class BaseNlLib:
> + def __init__(self):
> + pass
That __init__ seems pointless?
> + self.name = attr['name'].replace('-', '_')
You have a _lot_ of these ".replace('-', '_')" invocations - maybe make
a function just like you have c_upper()/c_lower()?
> + if self.presence_type() == 'len':
> + pfx = '__' if space == 'user' else ''
this is how you define pfx
> + if member:
> + ptr = ''
> + if self. is_multi_val():
> + ptr = '*'
but this is how you define ptr - feels a bit inconsistent
Also note the extra space after "self." there
> + if not self.is_multi_val():
> + ri.cw.p(f"if (ynl_attr_validate(yarg, attr))")
> + ri.cw.p(f"return MNL_CB_ERROR;")
Those didn't need f-strings.
Does this "ri.cw.p()" do indentation for the code?
> + def attr_policy(self, cw):
> + if 'unterminated-ok' in self.checks and self.checks['unterminated-ok']:
maybe
if self.checks.get('unterminated-ok', 0):
> +class TypeBinary(Type):
> + def arg_member(self, ri):
> + return [f"const void *{self.c_name}", 'size_t len']
> +
> + def presence_type(self):
> + return 'len'
> +
> + def struct_member(self, ri):
> + ri.cw.p(f"void *{self.c_name};")
> +
> + def _attr_typol(self):
> + return f'.type = YNL_PT_BINARY,'
> +
> + def _attr_policy(self, policy):
> + mem = '{ '
> + if len(self.checks) == 1 and 'min-len' in self.checks:
> + mem += '.len = ' + str(self.checks['min-len'])
Why does the len(self.checks) matter?
> + elif len(self.checks) == 0:
> + mem += '.type = NLA_BINARY'
> + else:
> + raise Exception('Binary type not implemented, yet')
to get to that error messsage? The error messsage seems a bit misleading
too though. It's that 'max-len' isn't implemented, or such, no?
> + def _complex_member_type(self, ri):
> + if 'type' not in self.attr or self.attr['type'] == 'nest':
could do
if self.attr.get('type', 'nest') == 'nest':
again like above
> + return f"struct {self.nested_render_name}"
> + elif self.attr['type'] in scalars:
> + scalar_pfx = '__' if ri.ku_space == 'user' else ''
You also have this
... = '__' if ... == 'user' else ''
quite a few times, maybe add something like
def qualify_user(value, user):
if user == 'user':
return '__' + value
return value
> + return scalar_pfx + self.attr['type']
and then this is just
return qualify_user(self.attr['type'], ri.ku_space)
instead of the two lines
> + def free_needs_iter(self):
> + return 'type' not in self.attr or self.attr['type'] == 'nest'
> +
> + def free(self, ri, var, ref):
> + if 'type' not in self.attr or self.attr['type'] == 'nest':
two more places that could use the .get trick
but it's really up to you. Just that the line like that seems rather
long to me :-)
There are many more below but I'll stop commenting.
> + if self.c_name in c_kw:
> + self.c_name += '_'
You also have this pattern quite a lot. Maybe a "c_safe()" wrapper or
something :)
FWIW, I'm still looking for "c_kw", maybe "pythonically" that should be
"_C_KW" since it's a private global? I think? Haven't seen it yet :)
> +c_kw = {
> + 'do'
> +}
Aha, here it is :-)
> + if has_ntf:
> + cw.p('// --------------- Common notification parsing --------------- //')
You said you were using /* */ comments now but this is still there.
> + print_ntf_parse_prototype(parsed, cw)
> + cw.nl()
> + else:
> + cw.p('// Policies')
and here too, etc.
Whew. I think I skipped some bits ;-)
Doesn't look that bad overall, IMHO. :)
johannes
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec
2023-01-19 0:36 ` [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec Jakub Kicinski
@ 2023-01-19 20:56 ` Johannes Berg
2023-01-20 0:18 ` Jacob Keller
0 siblings, 1 reply; 31+ messages in thread
From: Johannes Berg @ 2023-01-19 20:56 UTC (permalink / raw
To: Jakub Kicinski, davem
Cc: netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
>
> +fou-y := fou_core.o fou-nl.o
I feel like you could've been consistent there and used "fou-core.o"
with a dash, but hey :-)
johannes
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs
2023-01-19 14:07 ` Rob Herring
@ 2023-01-19 21:49 ` Jakub Kicinski
2023-01-19 22:24 ` Jakub Kicinski
2023-01-19 23:02 ` Rob Herring
2023-01-20 0:08 ` Jakub Kicinski
1 sibling, 2 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 21:49 UTC (permalink / raw
To: Rob Herring
Cc: davem, netdev, edumazet, pabeni, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
> On Wed, Jan 18, 2023 at 6:36 PM Jakub Kicinski <kuba@kernel.org> wrote:
> >
> > Add schemas for Netlink spec files. As described in the docs
> > we have 4 "protocols" or compatibility levels, and each one
> > comes with its own schema, but the more general / legacy
> > schemas are superset of more modern ones: genetlink is
> > the smallest followed by genetlink-c and genetlink-legacy.
> > There is no schema for raw netlink, yet, I haven't found the time..
> >
> > I don't know enough jsonschema to do inheritance or something
> > but the repetition is not too bad. I hope.
>
> Generally you put common schemas under '$defs' and the then reference
> them with '$ref'.
>
> $defs:
> some-prop-type:
> type: integer
> minimum: 0
>
> properties:
> foo:
> $ref: '#/$defs/some-prop-type'
> bar:
> $ref: '#/$defs/some-prop-type'
Thanks! Is it possible to move the common definitions to a separate
file? I tried to create a file called defs.yaml and change the ref to:
$ref: "defs.yaml#/$defs/len-or-define"
But:
File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 257, in iter_errors
for error in errors:
File "/usr/lib/python3.11/site-packages/jsonschema/_validators.py", line 294, in ref
scope, resolved = validator.resolver.resolve(ref)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 856, in resolve
return url, self._remote_cache(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 870, in resolve_from_url
raise exceptions.RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: Expecting value: line 1 column 1 (char 0)
> If you have objects with common sets of properties, you can do the
> same thing, but then you need 'unevaluatedProperties' if you want to
> define a base set of properties and add to them. We do that frequently
> in DT schemas. Unlike typical inheritance, you can't override the
> 'base' schema. It's an AND operation.
This is hard to comprehend :o Most of the time I seem to need only the
ability to add a custom "description" to the object, so for example:
$defs:
len-or-define:
oneOf:
-
type: string
pattern: ^[0-9A-Za-z_-]*( - 1)?$
-
type: integer
minimum: 0
Then:
min-len:
description: Min length for a binary attribute.
$ref: '#/$defs/len-or-define'
And that seems to work. Should I be using unevaluatedProperties somehow
as well here?
> > + description: |
> > + Name used when referring to this space in other definitions, not used outside of YAML.
> > + type: string
> > + # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
>
> If one is required:
>
> oneOf:
> - required: [ name-prefix ]
> - required: [ subset-of ]
>
> Or if both are optional:
>
> dependencies:
> name-prefix:
> not:
> required: [ subset-of ]
> subset-of:
> not:
> required: [ name-prefix ]
Nice, let me try this.
> > + min-len:
> > + description: Min length for a binary attribute.
> > + oneOf:
> > + - type: string
> > + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> > + - type: integer
>
> How can a length be a string?
For readability in C I wanted to allow using a define for the length.
Then the name of the define goes here, and the value can be fetched
from the "definitions" section of the spec.
> Anyways, this is something you could pull out into a $defs entry and
> reference. It will also work without the oneOf because 'pattern' will
> just be ignored for an integer. That's one gotcha with json-schema. If
> a keyword doesn't apply to the instance, it is silently ignored. (That
> includes unknown keywords such as ones with typos. Fun!). 'oneOf' will
> give you pretty crappy error messages, so it's good to avoid when
> possible.
Oh, interesting. Changed to:
$defs:
len-or-define:
type: [ string, integer ]
pattern: ^[0-9A-Za-z_-]*( - 1)?$
minimum: 0
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs
2023-01-19 21:49 ` Jakub Kicinski
@ 2023-01-19 22:24 ` Jakub Kicinski
2023-01-19 23:02 ` Rob Herring
1 sibling, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-19 22:24 UTC (permalink / raw
To: Rob Herring
Cc: davem, netdev, edumazet, pabeni, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Thu, 19 Jan 2023 13:49:22 -0800 Jakub Kicinski wrote:
> > Generally you put common schemas under '$defs' and the then reference
> > them with '$ref'.
> >
> > $defs:
> > some-prop-type:
> > type: integer
> > minimum: 0
> >
> > properties:
> > foo:
> > $ref: '#/$defs/some-prop-type'
> > bar:
> > $ref: '#/$defs/some-prop-type'
>
> Thanks! Is it possible to move the common definitions to a separate
> file? I tried to create a file called defs.yaml and change the ref to:
>
> $ref: "defs.yaml#/$defs/len-or-define"
Oh, oh. Instead of trying to create 3 different "levels" of spec,
and having to pull out shared definitions maybe I can use the
if + unevaluatedProperties
to only allow certain properties depending on the value in the
"protocol" attribute...
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs
2023-01-19 21:49 ` Jakub Kicinski
2023-01-19 22:24 ` Jakub Kicinski
@ 2023-01-19 23:02 ` Rob Herring
1 sibling, 0 replies; 31+ messages in thread
From: Rob Herring @ 2023-01-19 23:02 UTC (permalink / raw
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Thu, Jan 19, 2023 at 3:49 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
> > On Wed, Jan 18, 2023 at 6:36 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > >
> > > Add schemas for Netlink spec files. As described in the docs
> > > we have 4 "protocols" or compatibility levels, and each one
> > > comes with its own schema, but the more general / legacy
> > > schemas are superset of more modern ones: genetlink is
> > > the smallest followed by genetlink-c and genetlink-legacy.
> > > There is no schema for raw netlink, yet, I haven't found the time..
> > >
> > > I don't know enough jsonschema to do inheritance or something
> > > but the repetition is not too bad. I hope.
> >
> > Generally you put common schemas under '$defs' and the then reference
> > them with '$ref'.
> >
> > $defs:
> > some-prop-type:
> > type: integer
> > minimum: 0
> >
> > properties:
> > foo:
> > $ref: '#/$defs/some-prop-type'
> > bar:
> > $ref: '#/$defs/some-prop-type'
>
> Thanks! Is it possible to move the common definitions to a separate
> file? I tried to create a file called defs.yaml and change the ref to:
>
> $ref: "defs.yaml#/$defs/len-or-define"
Yes, but...
> But:
>
> File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 257, in iter_errors
> for error in errors:
> File "/usr/lib/python3.11/site-packages/jsonschema/_validators.py", line 294, in ref
> scope, resolved = validator.resolver.resolve(ref)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 856, in resolve
> return url, self._remote_cache(url)
> ^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 870, in resolve_from_url
> raise exceptions.RefResolutionError(exc)
> jsonschema.exceptions.RefResolutionError: Expecting value: line 1 column 1 (char 0)
You either need the ref at the URL or a custom resolver to convert
http path to a file path. Also, the default resolver doesn't handle
YAML either. It's not much code to do and the DT tools do this. The
hardest part was just getting the path right. For a ref with just a
filename and no path, the URL will be the same path as the $id in the
referring schema.
> > If you have objects with common sets of properties, you can do the
> > same thing, but then you need 'unevaluatedProperties' if you want to
> > define a base set of properties and add to them. We do that frequently
> > in DT schemas. Unlike typical inheritance, you can't override the
> > 'base' schema. It's an AND operation.
>
> This is hard to comprehend :o Most of the time I seem to need only the
> ability to add a custom "description" to the object, so for example:
By object, schemas here which are 'type: object'. Those all have a
list of properties. If several schemas for 'objects' have the same set
of properties like 'name' and 'type' for example, then you could
define a schema for that and then add more properties for each
specific object. I didn't really look close enough whether that makes
sense here.
> $defs:
> len-or-define:
> oneOf:
> -
> type: string
> pattern: ^[0-9A-Za-z_-]*( - 1)?$
> -
> type: integer
> minimum: 0
>
> Then:
>
> min-len:
> description: Min length for a binary attribute.
> $ref: '#/$defs/len-or-define'
>
> And that seems to work. Should I be using unevaluatedProperties somehow
> as well here?
No, it looks fine. 'unevaluatedProperties' wouldn't make sense here as
'type' is a string or integer.
>
> > > + description: |
> > > + Name used when referring to this space in other definitions, not used outside of YAML.
> > > + type: string
> > > + # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
> >
> > If one is required:
> >
> > oneOf:
> > - required: [ name-prefix ]
> > - required: [ subset-of ]
> >
> > Or if both are optional:
> >
> > dependencies:
> > name-prefix:
> > not:
> > required: [ subset-of ]
> > subset-of:
> > not:
> > required: [ name-prefix ]
>
> Nice, let me try this.
>
> > > + min-len:
> > > + description: Min length for a binary attribute.
> > > + oneOf:
> > > + - type: string
> > > + pattern: ^[0-9A-Za-z_-]*( - 1)?$
> > > + - type: integer
> >
> > How can a length be a string?
>
> For readability in C I wanted to allow using a define for the length.
> Then the name of the define goes here, and the value can be fetched
> from the "definitions" section of the spec.
Ah, makes sense.
Don't you need to drop the '-' in the regex then? Also, the '*' should
be a '+' for 1 or more instead of 0 or more.
> > Anyways, this is something you could pull out into a $defs entry and
> > reference. It will also work without the oneOf because 'pattern' will
> > just be ignored for an integer. That's one gotcha with json-schema. If
> > a keyword doesn't apply to the instance, it is silently ignored. (That
> > includes unknown keywords such as ones with typos. Fun!). 'oneOf' will
> > give you pretty crappy error messages, so it's good to avoid when
> > possible.
>
> Oh, interesting. Changed to:
>
> $defs:
> len-or-define:
> type: [ string, integer ]
> pattern: ^[0-9A-Za-z_-]*( - 1)?$
> minimum: 0
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs
2023-01-19 14:07 ` Rob Herring
2023-01-19 21:49 ` Jakub Kicinski
@ 2023-01-20 0:08 ` Jakub Kicinski
2023-01-20 14:43 ` Rob Herring
1 sibling, 1 reply; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-20 0:08 UTC (permalink / raw
To: Rob Herring; +Cc: netdev, linux-doc
On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
> > +$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#"
> > +$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
>
> There's no core.yaml. If you don't have a custom meta-schema, then
> just set this to the schema for the json-schema version you are using.
> Then the tools can validate the schemas without your own validator
> class.
$schema: https://json-schema.org/draft/2020-12/schema
or
$schema: https://json-schema.org/draft-09/schema
?
It seems like the documentation suggests the former but the latter
appears widespread.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec
2023-01-19 20:56 ` Johannes Berg
@ 2023-01-20 0:18 ` Jacob Keller
2023-01-20 1:04 ` Jakub Kicinski
0 siblings, 1 reply; 31+ messages in thread
From: Jacob Keller @ 2023-01-20 0:18 UTC (permalink / raw
To: Johannes Berg, Jakub Kicinski, davem
Cc: netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On 1/19/2023 12:56 PM, Johannes Berg wrote:
> On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
>>
>> +fou-y := fou_core.o fou-nl.o
>
> I feel like you could've been consistent there and used "fou-core.o"
> with a dash, but hey :-)
>
> johannes
And here I lean towards using _ instead of - in file names but
consistency is good, all else being equal.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-19 20:29 ` Johannes Berg
@ 2023-01-20 0:23 ` Jacob Keller
2023-01-20 9:10 ` Johannes Berg
2023-01-20 2:13 ` Jakub Kicinski
1 sibling, 1 reply; 31+ messages in thread
From: Jacob Keller @ 2023-01-20 0:23 UTC (permalink / raw
To: Johannes Berg, Jakub Kicinski, davem
Cc: netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel, Bagas Sanjaya
On 1/19/2023 12:29 PM, Johannes Berg wrote:
> On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
>> +kernel-policy
>> +~~~~~~~~~~~~~
>> +
>> +Defines if the kernel validation policy is per operation (``per-op``)
>> +or for the entire family (``global``). New families should use ``per-op``
>> +(default) to be able to narrow down the attributes accepted by a specific
>> +command.
>
> Again I'm not sure I agree with that recommendation, but I know it's
> your preference :-)
>
> (IMHO some things become more complex, such as having a "ifindex" in
> each one of them)
>
Per op policy is important because otherwise it can become impossible to
safely extend a new attribute to commands over multiple kernel releases.
If you add an attribute like DEVLINK_ATTR_DRY_RUN in one kernel, and add
it to devlink_cmd_foo.. its no longer really possible to add it to
another command if the policy is global.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 8/8] tools: ynl: add a completely generic client
2023-01-19 0:36 ` [PATCH net-next v3 8/8] tools: ynl: add a completely generic client Jakub Kicinski
@ 2023-01-20 0:50 ` Jacob Keller
0 siblings, 0 replies; 31+ messages in thread
From: Jacob Keller @ 2023-01-20 0:50 UTC (permalink / raw
To: Jakub Kicinski, davem
Cc: netdev, edumazet, pabeni, robh, johannes, stephen, ecree.xilinx,
sdf, f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On 1/18/2023 4:36 PM, Jakub Kicinski wrote:
> Add a CLI sample which can take in arbitrary request
> in JSON format, convert it to Netlink and do the inverse
> for output.
>
> It's meant as a development tool primarily and perhaps
> for selftests which need to tickle netlink in a special way.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> tools/net/ynl/samples/cli.py | 47 +++
> tools/net/ynl/samples/ynl.py | 534 +++++++++++++++++++++++++++++++++++
> 2 files changed, 581 insertions(+)
> create mode 100755 tools/net/ynl/samples/cli.py
> create mode 100644 tools/net/ynl/samples/ynl.py
>
> diff --git a/tools/net/ynl/samples/cli.py b/tools/net/ynl/samples/cli.py
> new file mode 100755
> index 000000000000..b27159c70710
> --- /dev/null
> +++ b/tools/net/ynl/samples/cli.py
> @@ -0,0 +1,47 @@
> +#!/usr/bin/env python
> +# SPDX-License-Identifier: BSD-3-Clause
> +
> +import argparse
> +import json
> +import pprint
> +import time
> +
> +from ynl import YnlFamily
> +
> +
> +def main():
> + parser = argparse.ArgumentParser(description='YNL CLI sample')
> + parser.add_argument('--spec', dest='spec', type=str, required=True)
> + parser.add_argument('--schema', dest='schema', type=str)
> + parser.add_argument('--json', dest='json_text', type=str)
> + parser.add_argument('--do', dest='do', type=str)
> + parser.add_argument('--dump', dest='dump', type=str)
> + parser.add_argument('--sleep', dest='sleep', type=int)
> + parser.add_argument('--subscribe', dest='ntf', type=str)
> + args = parser.parse_args()
> +
> + attrs = {}
> + if args.json_text:
> + attrs = json.loads(args.json_text)
> +
> + ynl = YnlFamily(args.spec, args.schema)
> +
> + if args.ntf:
> + ynl.ntf_subscribe(args.ntf)
> +
> + if args.sleep:
> + time.sleep(args.sleep)
> +
> + if args.do or args.dump:
> + method = getattr(ynl, args.do if args.do else args.dump)
> +
> + reply = method(attrs, dump=bool(args.dump))
> + pprint.PrettyPrinter().pprint(reply)
> +
> + if args.ntf:
> + ynl.check_ntf()
> + pprint.PrettyPrinter().pprint(ynl.async_msg_queue)
> +
> +
> +if __name__ == "__main__":
> + main()
> diff --git a/tools/net/ynl/samples/ynl.py b/tools/net/ynl/samples/ynl.py
> new file mode 100644
> index 000000000000..b71523d71d46
> --- /dev/null
> +++ b/tools/net/ynl/samples/ynl.py
> @@ -0,0 +1,534 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +
> +import functools
> +import jsonschema
> +import os
> +import random
> +import socket
> +import struct
> +import yaml
> +
> +#
> +# Generic Netlink code which should really be in some library, but I can't quickly find one.
> +#
> +
There is pyroute2, but it might be overkill, and I recall it seeming to
do some things a bit inconsistently.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec
2023-01-20 0:18 ` Jacob Keller
@ 2023-01-20 1:04 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-20 1:04 UTC (permalink / raw
To: Jacob Keller
Cc: Johannes Berg, davem, netdev, edumazet, pabeni, robh, stephen,
ecree.xilinx, sdf, f.fainelli, fw, linux-doc, razor,
nicolas.dichtel
On Thu, 19 Jan 2023 16:18:04 -0800 Jacob Keller wrote:
> On 1/19/2023 12:56 PM, Johannes Berg wrote:
> > On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
> >> +fou-y := fou_core.o fou-nl.o
> >
> > I feel like you could've been consistent there and used "fou-core.o"
> > with a dash, but hey :-)
>
> And here I lean towards using _ instead of - in file names but
> consistency is good, all else being equal.
Done. IIRC the codegen used to assume the naming (it needs to know
the name of the header to include). I fixed that but forgot to rename
fou :S
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 3/8] net: add basic C code generators for Netlink
2023-01-19 20:53 ` Johannes Berg
@ 2023-01-20 1:53 ` Jakub Kicinski
2023-01-20 9:17 ` Johannes Berg
0 siblings, 1 reply; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-20 1:53 UTC (permalink / raw
To: Johannes Berg
Cc: davem, netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Thu, 19 Jan 2023 21:53:12 +0100 Johannes Berg wrote:
> > + def _attr_policy(self, policy):
> > + mem = '{ '
> > + if len(self.checks) == 1 and 'min-len' in self.checks:
> > + mem += '.len = ' + str(self.checks['min-len'])
>
> Why does the len(self.checks) matter?
Trying to throw an exception if someone starts using checks I haven't
gotten to implementing yet.
> > + def free_needs_iter(self):
> > + return 'type' not in self.attr or self.attr['type'] == 'nest'
> > +
> > + def free(self, ri, var, ref):
> > + if 'type' not in self.attr or self.attr['type'] == 'nest':
>
> two more places that could use the .get trick
>
> but it's really up to you. Just that the line like that seems rather
> long to me :-)
Better question is why attr would not have a type :S
Let me leave it be for now, I'll revisit when exercising this code
in the future.
> > + if has_ntf:
> > + cw.p('// --------------- Common notification parsing --------------- //')
>
> You said you were using /* */ comments now but this is still there.
Ugh, now I'm worried I lost something in a rebase :S
> > + print_ntf_parse_prototype(parsed, cw)
> > + cw.nl()
> > + else:
> > + cw.p('// Policies')
>
> and here too, etc.
>
> Whew. I think I skipped some bits ;-)
Thanks for looking!! :)
> Doesn't look that bad overall, IMHO. :)
I hope we can avoid over-focusing on the python tools :P
I'm no python expert and the code would use a lot of refactoring.
But there's only so many hours in the day and the alternative
seems that it will bit rot in my tree forever :(
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-19 20:29 ` Johannes Berg
2023-01-20 0:23 ` Jacob Keller
@ 2023-01-20 2:13 ` Jakub Kicinski
2023-01-20 9:15 ` Johannes Berg
1 sibling, 1 reply; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-20 2:13 UTC (permalink / raw
To: Johannes Berg
Cc: davem, netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel, Bagas Sanjaya
On Thu, 19 Jan 2023 21:29:22 +0100 Johannes Berg wrote:
> On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
> >
> > +Answer requests
> > +---------------
> > +
> > +Older families do not reply to all of the commands, especially NEW / ADD
> > +commands. User only gets information whether the operation succeeded or
> > +not via the ACK. Try to find useful data to return. Once the command is
> > +added whether it replies with a full message or only an ACK is uAPI and
> > +cannot be changed. It's better to err on the side of replying.
> > +
> > +Specifically NEW and ADD commands should reply with information identifying
> > +the created object such as the allocated object's ID (without having to
> > +resort to using ``NLM_F_ECHO``).
>
> I'm a bit on the fence on this recommendation (as written).
>
> Yeah, it's nice to reply to things ... but!
>
> In userspace, you often request and wait for the ACK to see if the
> operation succeeded. This is basically necessary. But then it's
> complicated to wait for *another* message to see the ID.
Maybe you're looking at this from the perspective of a person tired
of manually writing the user space code?
If the netlink message handling code is all auto-generated it makes
no difference to the user...
> We've actually started using the "cookie" in the extack to report an ID
> of an object/... back, see uses of nl_set_extack_cookie_u64() in the
> tree.
... and to the middle-layer / RPC / auto-generated library pulling
stuff out from protocol messages and interpreting them in a special
way is a typical netlink vortex of magic :(
> So I'm not sure I wholeheartedly agree with the recommendation to send a
> separate answer. We've done that, but it's ugly on both sender side in
> the kernel (requiring an extra message allocation, ideally at the
> beginning of the operation so you can fail gracefully, etc.) and on the
> receiver (having to wait for another message if the operation was
> successful; possibly actually having to check for that message *before*
> the ACK arrives.)
Right, response is before ACK. It's not different to a GET command,
really.
> > +Support dump consistency
> > +------------------------
> > +
> > +If iterating over objects during dump may skip over objects or repeat
> > +them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.
>
> That could be a bit more fleshed out on _how_ to do that, if it's not
> somewhere else?
I was thinking about adding a sentence like "To avoid consistency
issues store your objects in an Xarray and correctly use the ID during
iteration".. but it seems to hand-wavy. Really the coder needs to
understand dumps quite well to get what's going on, and then the
consistency is kinda obvious. IDK. Almost nobody gets this right :(
> > +checks
> > +------
> > +
> > +Documentation for the ``checks`` sub-sections of attribute specs.
> > +
> > +unterminated-ok
> > +~~~~~~~~~~~~~~~
> > +
> > +Accept strings without the null-termination (for legacy families only).
> > +Switches from the ``NLA_NUL_STRING`` to ``NLA_STRING`` policy type.
>
> Should we even document all the legacy bits in such a prominent place?
>
> (or just move it after max-len/min-len?)
This is kernel-only info (checks) so I figured it's good enough until
someone takes a more serious stab at supporting legacy families.
> > +Attribute type nests
> > +--------------------
> > +
> > +New Netlink families should use ``multi-attr`` to define arrays.
>
> Unrelated to this particular document, but ...
>
> I'm all for this, btw, but maybe we should have a way of representing in
> the policy that an attribute is used as multi-attr for an array, and a
> way of exposing that in the policy export? Hmm. Haven't thought about
> this for a while.
Informational-only or enforced? Enforcing this now would be another
backward-compat nightmare :(
FWIW I have a set parked on a branch to add "required" bit to policies,
so for per-op policies one can reject requests with missing attrs
during validation.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-20 0:23 ` Jacob Keller
@ 2023-01-20 9:10 ` Johannes Berg
2023-01-20 18:35 ` Keller, Jacob E
0 siblings, 1 reply; 31+ messages in thread
From: Johannes Berg @ 2023-01-20 9:10 UTC (permalink / raw
To: Jacob Keller, Jakub Kicinski, davem
Cc: netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel, Bagas Sanjaya
On Thu, 2023-01-19 at 16:23 -0800, Jacob Keller wrote:
>
> Per op policy is important because otherwise it can become impossible to
> safely extend a new attribute to commands over multiple kernel releases.
>
Yeah. I think I just realised that my issues is more with the fact that
per-op policy implies per-op attribute (identifier/number/name)space,
and if you don't have that you have attribute duplication etc.
Anyway, it just feels superfluous, not really dangerous I guess :)
johannes
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-20 2:13 ` Jakub Kicinski
@ 2023-01-20 9:15 ` Johannes Berg
2023-01-20 17:23 ` Jakub Kicinski
0 siblings, 1 reply; 31+ messages in thread
From: Johannes Berg @ 2023-01-20 9:15 UTC (permalink / raw
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel, Bagas Sanjaya
On Thu, 2023-01-19 at 18:13 -0800, Jakub Kicinski wrote:
> On Thu, 19 Jan 2023 21:29:22 +0100 Johannes Berg wrote:
> > On Wed, 2023-01-18 at 16:36 -0800, Jakub Kicinski wrote:
> > >
> > > +Answer requests
> > > +---------------
> > > +
> > > +Older families do not reply to all of the commands, especially NEW / ADD
> > > +commands. User only gets information whether the operation succeeded or
> > > +not via the ACK. Try to find useful data to return. Once the command is
> > > +added whether it replies with a full message or only an ACK is uAPI and
> > > +cannot be changed. It's better to err on the side of replying.
> > > +
> > > +Specifically NEW and ADD commands should reply with information identifying
> > > +the created object such as the allocated object's ID (without having to
> > > +resort to using ``NLM_F_ECHO``).
> >
> > I'm a bit on the fence on this recommendation (as written).
> >
> > Yeah, it's nice to reply to things ... but!
> >
> > In userspace, you often request and wait for the ACK to see if the
> > operation succeeded. This is basically necessary. But then it's
> > complicated to wait for *another* message to see the ID.
>
> Maybe you're looking at this from the perspective of a person tired
> of manually writing the user space code?
Heh, I guess :)
> If the netlink message handling code is all auto-generated it makes
> no difference to the user...
True. And I guess we can hope that'll be the case for most users, though
I doubt it :)
> > We've actually started using the "cookie" in the extack to report an ID
> > of an object/... back, see uses of nl_set_extack_cookie_u64() in the
> > tree.
>
> ... and to the middle-layer / RPC / auto-generated library pulling
> stuff out from protocol messages and interpreting them in a special
> way is a typical netlink vortex of magic :(
:)
> > So I'm not sure I wholeheartedly agree with the recommendation to send a
> > separate answer. We've done that, but it's ugly on both sender side in
> > the kernel (requiring an extra message allocation, ideally at the
> > beginning of the operation so you can fail gracefully, etc.) and on the
> > receiver (having to wait for another message if the operation was
> > successful; possibly actually having to check for that message *before*
> > the ACK arrives.)
>
> Right, response is before ACK. It's not different to a GET command,
> really.
True.
> > > +Support dump consistency
> > > +------------------------
> > > +
> > > +If iterating over objects during dump may skip over objects or repeat
> > > +them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.
> >
> > That could be a bit more fleshed out on _how_ to do that, if it's not
> > somewhere else?
>
> I was thinking about adding a sentence like "To avoid consistency
> issues store your objects in an Xarray and correctly use the ID during
> iteration".. but it seems to hand-wavy. Really the coder needs to
> understand dumps quite well to get what's going on, and then the
> consistency is kinda obvious. IDK. Almost nobody gets this right :(
Yeah agree, it's tricky one way or the other. To be honest I was
thinking less of documenting the mechanics of the underlying code to
ensure that, but rather of the mechanics of using the APIs to ensure
that, i.e. how to use cb->seq and friends.
> > Unrelated to this particular document, but ...
> >
> > I'm all for this, btw, but maybe we should have a way of representing in
> > the policy that an attribute is used as multi-attr for an array, and a
> > way of exposing that in the policy export? Hmm. Haven't thought about
> > this for a while.
>
> Informational-only or enforced? Enforcing this now would be another
> backward-compat nightmare :(
More informational - for userspace to know from policy dump that certain
attributes have that property. With nested it's easy to know (there's a
special nested-array type), but multi-attr there's no way to distinguish
"is this one" and "is this multiple".
Now ... you might say you don't really care now since you want
everything to be auto-generated and then you have it in the docs
(actually, do you?), and that's a fair point.
> FWIW I have a set parked on a branch to add "required" bit to policies,
> so for per-op policies one can reject requests with missing attrs
> during validation.
Nice. That might yet convince me of per-op policies ;-)
Though IMHO the namespace issue remains - I'd still not like to have 100
definitions of NL80211_ATTR_IFINDEX or similar.
johannes
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 3/8] net: add basic C code generators for Netlink
2023-01-20 1:53 ` Jakub Kicinski
@ 2023-01-20 9:17 ` Johannes Berg
0 siblings, 0 replies; 31+ messages in thread
From: Johannes Berg @ 2023-01-20 9:17 UTC (permalink / raw
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel
On Thu, 2023-01-19 at 17:53 -0800, Jakub Kicinski wrote:
>
> > Doesn't look that bad overall, IMHO. :)
>
> I hope we can avoid over-focusing on the python tools :P
Oh sure.
> I'm no python expert and the code would use a lot of refactoring.
> But there's only so many hours in the day and the alternative
> seems that it will bit rot in my tree forever :(
>
:)
Yeah I don't mind. The quality of the implementation matters to a point,
but honestly I was just reading through this to convince myself it
wasn't just a totally stinking pile of hacks ;-) Which you/I have
managed, so personally I'm happy with this.
johannes
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs
2023-01-20 0:08 ` Jakub Kicinski
@ 2023-01-20 14:43 ` Rob Herring
0 siblings, 0 replies; 31+ messages in thread
From: Rob Herring @ 2023-01-20 14:43 UTC (permalink / raw
To: Jakub Kicinski; +Cc: netdev, linux-doc
On Thu, Jan 19, 2023 at 6:08 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
> > > +$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#"
> > > +$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
> >
> > There's no core.yaml. If you don't have a custom meta-schema, then
> > just set this to the schema for the json-schema version you are using.
> > Then the tools can validate the schemas without your own validator
> > class.
>
> $schema: https://json-schema.org/draft/2020-12/schema
>
> or
>
> $schema: https://json-schema.org/draft-09/schema
s/draft-09/draft\/2019-09/
Or did you mean draft-07?
> ?
>
> It seems like the documentation suggests the former but the latter
> appears widespread.
Yes, 2019-09 is generally not recommended as it had some one off
features that 2020-12 replaced (nothing you'd care about). DT uses
2019-09 because we needed unevaluatedProperties, but not 'prefixItems'
replacing 'items' list variant in 2020-12. I think you only used the
schema version of 'items' (not the list version), so you are probably
compatible with any version. 'dependencies' changing to
'dependentSchema/dependentRequired' is the other difference you might
hit.
TLDR: draft-07 is probably sufficient for your needs.
Rob
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-20 9:15 ` Johannes Berg
@ 2023-01-20 17:23 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2023-01-20 17:23 UTC (permalink / raw
To: Johannes Berg
Cc: davem, netdev, edumazet, pabeni, robh, stephen, ecree.xilinx, sdf,
f.fainelli, fw, linux-doc, razor, nicolas.dichtel, Bagas Sanjaya
On Fri, 20 Jan 2023 10:15:39 +0100 Johannes Berg wrote:
> > > > +Support dump consistency
> > > > +------------------------
> > > > +
> > > > +If iterating over objects during dump may skip over objects or repeat
> > > > +them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.
> > >
> > > That could be a bit more fleshed out on _how_ to do that, if it's not
> > > somewhere else?
> >
> > I was thinking about adding a sentence like "To avoid consistency
> > issues store your objects in an Xarray and correctly use the ID during
> > iteration".. but it seems to hand-wavy. Really the coder needs to
> > understand dumps quite well to get what's going on, and then the
> > consistency is kinda obvious. IDK. Almost nobody gets this right :(
>
> Yeah agree, it's tricky one way or the other. To be honest I was
> thinking less of documenting the mechanics of the underlying code to
> ensure that, but rather of the mechanics of using the APIs to ensure
> that, i.e. how to use cb->seq and friends.
I see. Let me add that.
My hope was to steer people towards data structures with stable
indexes, so the problem doesn't occur. But I'll add a mention of
the helpers.
> > > Unrelated to this particular document, but ...
> > >
> > > I'm all for this, btw, but maybe we should have a way of representing in
> > > the policy that an attribute is used as multi-attr for an array, and a
> > > way of exposing that in the policy export? Hmm. Haven't thought about
> > > this for a while.
> >
> > Informational-only or enforced? Enforcing this now would be another
> > backward-compat nightmare :(
>
> More informational - for userspace to know from policy dump that certain
> attributes have that property. With nested it's easy to know (there's a
> special nested-array type), but multi-attr there's no way to distinguish
> "is this one" and "is this multiple".
Makes sense.
> Now ... you might say you don't really care now since you want
> everything to be auto-generated and then you have it in the docs
> (actually, do you?), and that's a fair point.
Have in the docs that we want everything to be auto-generated?
> > FWIW I have a set parked on a branch to add "required" bit to policies,
> > so for per-op policies one can reject requests with missing attrs
> > during validation.
>
> Nice. That might yet convince me of per-op policies ;-)
>
> Though IMHO the namespace issue remains - I'd still not like to have 100
> definitions of NL80211_ATTR_IFINDEX or similar.
Yeah, there's different ways of dealing with it. The ethtool way is
pretty neat - have a nest in each command for "common attrs" with
ifindex and stuff in it.
^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
2023-01-20 9:10 ` Johannes Berg
@ 2023-01-20 18:35 ` Keller, Jacob E
0 siblings, 0 replies; 31+ messages in thread
From: Keller, Jacob E @ 2023-01-20 18:35 UTC (permalink / raw
To: Johannes Berg, Jakub Kicinski, davem@davemloft.net
Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com,
robh@kernel.org, stephen@networkplumber.org,
ecree.xilinx@gmail.com, sdf@google.com, f.fainelli@gmail.com,
fw@strlen.de, linux-doc@vger.kernel.org, razor@blackwall.org,
nicolas.dichtel@6wind.com, Bagas Sanjaya
> -----Original Message-----
> From: Johannes Berg <johannes@sipsolutions.net>
> Sent: Friday, January 20, 2023 1:11 AM
> To: Keller, Jacob E <jacob.e.keller@intel.com>; Jakub Kicinski <kuba@kernel.org>;
> davem@davemloft.net
> Cc: netdev@vger.kernel.org; edumazet@google.com; pabeni@redhat.com;
> robh@kernel.org; stephen@networkplumber.org; ecree.xilinx@gmail.com;
> sdf@google.com; f.fainelli@gmail.com; fw@strlen.de; linux-
> doc@vger.kernel.org; razor@blackwall.org; nicolas.dichtel@6wind.com; Bagas
> Sanjaya <bagasdotme@gmail.com>
> Subject: Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)
>
> On Thu, 2023-01-19 at 16:23 -0800, Jacob Keller wrote:
> >
> > Per op policy is important because otherwise it can become impossible to
> > safely extend a new attribute to commands over multiple kernel releases.
> >
>
> Yeah. I think I just realised that my issues is more with the fact that
> per-op policy implies per-op attribute (identifier/number/name)space,
> and if you don't have that you have attribute duplication etc.
>
> Anyway, it just feels superfluous, not really dangerous I guess :)
>
> Johannes
I think we want per-family attribute and op IDs, but still per-op policy that indicates which attributes are supported for a command.
This way you can re-use attributes for multiple commands, and they should behave the same semantics for those commands which support them, but the kernel can properly express which attributes are valid for a given command using the per-op policy.
I know some commands do use separate attributes for each command, or separate attribute space for sub-attributes in nests, but I am not sure how widespread that is. Devlink uses a single space for all its attributes.
Thanks,
Jake
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2023-01-20 18:35 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-01-19 0:36 [PATCH net-next v3 0/8] Netlink protocol specs Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs) Jakub Kicinski
2023-01-19 15:48 ` Vladimir Oltean
2023-01-19 20:29 ` Johannes Berg
2023-01-20 0:23 ` Jacob Keller
2023-01-20 9:10 ` Johannes Berg
2023-01-20 18:35 ` Keller, Jacob E
2023-01-20 2:13 ` Jakub Kicinski
2023-01-20 9:15 ` Johannes Berg
2023-01-20 17:23 ` Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 2/8] netlink: add schemas for YAML specs Jakub Kicinski
2023-01-19 14:07 ` Rob Herring
2023-01-19 21:49 ` Jakub Kicinski
2023-01-19 22:24 ` Jakub Kicinski
2023-01-19 23:02 ` Rob Herring
2023-01-20 0:08 ` Jakub Kicinski
2023-01-20 14:43 ` Rob Herring
2023-01-19 0:36 ` [PATCH net-next v3 3/8] net: add basic C code generators for Netlink Jakub Kicinski
2023-01-19 20:53 ` Johannes Berg
2023-01-20 1:53 ` Jakub Kicinski
2023-01-20 9:17 ` Johannes Berg
2023-01-19 0:36 ` [PATCH net-next v3 4/8] netlink: add a proto specification for FOU Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 5/8] net: fou: regenerate the uAPI from the spec Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 6/8] net: fou: rename the source for linking Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec Jakub Kicinski
2023-01-19 20:56 ` Johannes Berg
2023-01-20 0:18 ` Jacob Keller
2023-01-20 1:04 ` Jakub Kicinski
2023-01-19 0:36 ` [PATCH net-next v3 8/8] tools: ynl: add a completely generic client Jakub Kicinski
2023-01-20 0:50 ` Jacob Keller
2023-01-19 17:07 ` [PATCH net-next v3 0/8] Netlink protocol specs Stanislav Fomichev
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.