diff options
| author | Niklas Cassel <cassel@kernel.org> | 2026-09-16 15:00:32 +0200 |
|---|---|---|
| committer | Niklas Cassel <cassel@kernel.org> | 2026-09-16 16:18:00 +0200 |
| commit | 8d836581f9b1f57ceaa2b47654754ef1260b410b (patch) | |
| tree | 1dbd4d0dfb99c2e238356cbe947e5b0be3e3a571 /tools/net | |
| download | linux-stable-8d836581f9b1f57ceaa2b47654754ef1260b410b.tar.gz linux-stable-8d836581f9b1f57ceaa2b47654754ef1260b410b.zip | |
ata: libata-scsi: fix ata_dsm_trim_pages() kernel-docgrafted
make htmldocs fails with:
Documentation/driver-api/libata:604: ./drivers/ata/libata-scsi.c:2301:
ERROR: Unexpected indentation. [docutils]
The Return: section of ata_dsm_trim_pages() ends its first sentence with
a colon and continues with an indented bullet list. reStructuredText
requires a blank line before an indented block, so docutils chokes on
the list.
Simply adding the missing blank line does not work either: Return: is a
kernel-doc "special section", which is terminated by the first blank
line, so the bullet list would end up in the Description section,
detached from the sentence introducing it.
Spell the two bounds out as prose instead, so that the Return: section
stays self-contained. Documentation-only change.
Fixes: e64e6b5dc867 ("ata: libata-scsi: scale DSM TRIM payload by MAX PAGES PER DSM COMMAND")
Reported-by: Thomas Huth <thuth@redhat.com>
Closes: https://lore.kernel.org/linux-ide/b0a0b8e8-cc4f-4c7b-8bc5-fee0712d405a@redhat.com/
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20260916130031.29990-2-cassel@kernel.org
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Diffstat (limited to 'tools/net')
231 files changed, 18577 insertions, 0 deletions
diff --git a/tools/net/sunrpc/extract.sh b/tools/net/sunrpc/extract.sh new file mode 100755 index 000000000..f944066f2 --- /dev/null +++ b/tools/net/sunrpc/extract.sh @@ -0,0 +1,11 @@ +#! /bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# Extract an RPC protocol specification from an RFC document. +# The version of this script comes from RFC 8166. +# +# Usage: +# $ extract.sh < rfcNNNN.txt > protocol.x +# + +grep '^ *///' | sed 's?^ */// ??' | sed 's?^ *///$??' diff --git a/tools/net/sunrpc/xdrgen/.gitignore b/tools/net/sunrpc/xdrgen/.gitignore new file mode 100644 index 000000000..d7366c2f9 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/.gitignore @@ -0,0 +1,2 @@ +__pycache__ +generators/__pycache__ diff --git a/tools/net/sunrpc/xdrgen/README b/tools/net/sunrpc/xdrgen/README new file mode 100644 index 000000000..2cf05d1e4 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/README @@ -0,0 +1,259 @@ +xdrgen - Linux Kernel XDR code generator + +Introduction +------------ + +SunRPC programs are typically specified using a language defined by +RFC 4506. In fact, all IETF-published NFS specifications provide a +description of the specified protocol using this language. + +Since the 1990's, user space consumers of SunRPC have had access to +a tool that could read such XDR specifications and then generate C +code that implements the RPC portions of that protocol. This tool is +called rpcgen. + +This RPC-level code is code that handles input directly from the +network, and thus a high degree of memory safety and sanity checking +is needed to help ensure proper levels of security. Bugs in this +code can have significant impact on security and performance. + +However, it is code that is repetitive and tedious to write by hand. + +The C code generated by rpcgen makes extensive use of the facilities +of the user space TI-RPC library and libc. Furthermore, the dialect +of the generated code is very traditional K&R C. + +The Linux kernel's implementation of SunRPC-based protocols hand-roll +their XDR implementation. There are two main reasons for this: + +1. libtirpc (and its predecessors) operate only in user space. The + kernel's RPC implementation and its API are significantly + different than libtirpc. + +2. rpcgen-generated code is believed to be less efficient than code + that is hand-written. + +These days, gcc and its kin are capable of optimizing code better +than human authors. There are only a few instances where writing +XDR code by hand will make a measurable performance different. + +In addition, the current hand-written code in the Linux kernel is +difficult to audit and prove that it implements exactly what is in +the protocol specification. + +In order to accrue the benefits of machine-generated XDR code in the +kernel, a tool is needed that will output C code that works against +the kernel's SunRPC implementation rather than libtirpc. + +Enter xdrgen. + + +Dependencies +------------ + +These dependencies are typically packaged by Linux distributions: + +- python3 +- python3-lark +- python3-jinja2 + +These dependencies are available via PyPi: + +- pip install 'lark[interegular]' + + +XDR Specifications +------------------ + +When adding a new protocol implementation to the kernel, the XDR +specification can be derived by feeding a .txt copy of the RFC to +the script located in tools/net/sunrpc/extract.sh. + + $ extract.sh < rfc0001.txt > new2.x + + +Operation +--------- + +Once a .x file is available, use xdrgen to generate source and +header files containing an implementation of XDR encoding and +decoding functions for the specified protocol. + + $ ./xdrgen definitions new2.x > include/linux/sunrpc/xdrgen/new2.h + $ ./xdrgen declarations new2.x > new2xdr_gen.h + +and + + $ ./xdrgen source new2.x > new2xdr_gen.c + +The files are ready to use for a server-side protocol implementation, +or may be used as a guide for implementing these routines by hand. + +By default, the only comments added to this code are kdoc comments +that appear directly in front of the public per-procedure APIs. For +deeper introspection, specifying the "--annotate" flag will insert +additional comments in the generated code to help readers match the +generated code to specific parts of the XDR specification. + +Because the generated code is targeted for the Linux kernel, it +is tagged with a GPLv2-only license. + +The xdrgen tool can also provide lexical and syntax checking of +an XDR specification: + + $ ./xdrgen lint xdr/new.x + + +How It Works +------------ + +xdrgen does not use machine learning to generate source code. The +translation is entirely deterministic. + +RFC 4506 Section 6 contains a BNF grammar of the XDR specification +language. The grammar has been adapted for use by the Python Lark +module. + +The xdr.ebnf file in this directory contains the grammar used to +parse XDR specifications. xdrgen configures Lark using the grammar +in xdr.ebnf. Lark parses the target XDR specification using this +grammar, creating a parse tree. + +xdrgen then transforms the parse tree into an abstract syntax tree. +This tree is passed to a series of code generators. + +The generators are implemented as Python classes residing in the +generators/ directory. Each generator emits code created from Jinja2 +templates stored in the templates/ directory. + +The source code is generated in the same order in which they appear +in the specification to ensure the generated code compiles. This +conforms with the behavior of rpcgen. + +xdrgen assumes that the generated source code is further compiled by +a compiler that can optimize in a number of ways, including: + + - Unused functions are discarded (ie, not added to the executable) + + - Aggressive function inlining removes unnecessary stack frames + + - Single-arm switch statements are replaced by a single conditional + branch + +And so on. + + +Pragmas +------- + +Pragma directives specify exceptions to the normal generation of +encoding and decoding functions. Currently one directive is +implemented: "public". + +Pragma big_endian +------ ---------- + + pragma big_endian <enum> ; + +For variables that might contain only a small number values, it +is more efficient to avoid the byte-swap when encoding or decoding +on little-endian machines. Such is often the case with error status +codes. For example: + + pragma big_endian nfsstat3; + +In this case, when generating an XDR struct or union containing a +field of type "nfsstat3", xdrgen will make the type of that field +"__be32" instead of "enum nfsstat3". XDR unions then switch on the +non-byte-swapped value of that field. + +Pragma exclude +------ ------- + + pragma exclude <RPC procedure> ; + +In some cases, a procedure encoder or decoder function might need +special processing that cannot be automatically generated. The +automatically-generated functions might conflict or interfere with +the hand-rolled function. To avoid editing the generated source code +by hand, a pragma can specify that the procedure's encoder and +decoder functions are not included in the generated header and +source. + +For example: + + pragma exclude NFSPROC3_READDIRPLUS; + +Excludes the decoder function for the READDIRPLUS argument and the +encoder function for the READDIRPLUS result. + +Note that because data item encoder and decoder functions are +defined "static __maybe_unused", subsequent compilation +automatically excludes data item encoder and decoder functions that +are used only by excluded procedure. + +Pragma header +------ ------ + + pragma header <string> ; + +Provide a name to use for the header file. For example: + + pragma header nlm4; + +Adds + + #include "nlm4xdr_gen.h" + +to the generated source file. + +Pragma public +------ ------ + + pragma public <XDR data item> ; + +Normally XDR encoder and decoder functions are "static". In case an +implementer wants to call these functions from other source code, +s/he can add a public pragma in the input .x file to indicate a set +of functions that should get a prototype in the generated header, +and the function definitions will not be declared static. + +For example: + + pragma public nfsstat3; + +Adds these prototypes in the generated header: + + bool xdrgen_decode_nfsstat3(struct xdr_stream *xdr, enum nfsstat3 *ptr); + bool xdrgen_encode_nfsstat3(struct xdr_stream *xdr, enum nfsstat3 value); + +And, in the generated source code, both of these functions appear +without the "static __maybe_unused" modifiers. + + +Future Work +----------- + +Finish implementing XDR pointer and list types. + +Generate client-side procedure functions + +Expand the README into a user guide similar to rpcgen(1) + +Add more pragma directives: + + * @pages -- use xdr_read/write_pages() for the specified opaque + field + * @skip -- do not decode, but rather skip, the specified argument + field + +Enable something like a #include to dynamically insert the content +of other specification files + +Build a unit test suite for verifying translation of XDR language +into compilable code + +Add a command-line option to insert trace_printk call sites in the +generated source code, for improved (temporary) observability + +Generate kernel Rust code as well as C code diff --git a/tools/net/sunrpc/xdrgen/__init__.py b/tools/net/sunrpc/xdrgen/__init__.py new file mode 100644 index 000000000..c940e9275 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/__init__.py @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +# Just to make sphinx-apidoc document this directory diff --git a/tools/net/sunrpc/xdrgen/generators/__init__.py b/tools/net/sunrpc/xdrgen/generators/__init__.py new file mode 100644 index 000000000..5c3a4a47d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/__init__.py @@ -0,0 +1,121 @@ +# SPDX-License-Identifier: GPL-2.0 + +"""Define a base code generator class""" + +from pathlib import Path +from jinja2 import Environment, FileSystemLoader, Template + +from xdr_ast import _XdrAst, Specification, _RpcProgram, _XdrTypeSpecifier +from xdr_ast import public_apis, pass_by_reference, structs, get_header_name +from xdr_parse import get_xdr_annotate + + +def create_jinja2_environment(language: str, xdr_type: str) -> Environment: + """Open a set of templates based on output language""" + match language: + case "C": + templates_dir = ( + Path(__file__).parent.parent / "templates" / language / xdr_type + ) + environment = Environment( + loader=FileSystemLoader(templates_dir), + trim_blocks=True, + lstrip_blocks=True, + ) + environment.globals["annotate"] = get_xdr_annotate() + environment.globals["public_apis"] = public_apis + environment.globals["pass_by_reference"] = pass_by_reference + environment.globals["structs"] = structs + return environment + case _: + raise NotImplementedError("Language not supported") + + +def get_jinja2_template( + environment: Environment, template_type: str, template_name: str +) -> Template: + """Retrieve a Jinja2 template for emitting source code""" + return environment.get_template(template_type + "/" + template_name + ".j2") + + +def find_xdr_program_name(root: Specification) -> str: + """Retrieve the RPC program name from an abstract syntax tree""" + raw_name = get_header_name() + if raw_name != "none": + return raw_name.lower() + for definition in root.definitions: + if isinstance(definition.value, _RpcProgram): + raw_name = definition.value.name + return raw_name.lower().removesuffix("_program").removesuffix("_prog") + return "noprog" + + +def header_guard_infix(filename: str) -> str: + """Extract the header guard infix from the specification filename""" + return Path(filename).stem.upper() + + +def kernel_c_type(spec: _XdrTypeSpecifier) -> str: + """Return name of C type""" + builtin_native_c_type = { + "bool": "bool", + "short": "s16", + "unsigned_short": "u16", + "int": "s32", + "unsigned_int": "u32", + "long": "s32", + "unsigned_long": "u32", + "hyper": "s64", + "unsigned_hyper": "u64", + } + if spec.type_name in builtin_native_c_type: + return builtin_native_c_type[spec.type_name] + return spec.type_name + + +class Boilerplate: + """Base class to generate boilerplate for source files""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + raise NotImplementedError("No language support defined") + + def emit_declaration(self, filename: str, root: Specification) -> None: + """Emit declaration header boilerplate""" + raise NotImplementedError("Header boilerplate generation not supported") + + def emit_definition(self, filename: str, root: Specification) -> None: + """Emit definition header boilerplate""" + raise NotImplementedError("Header boilerplate generation not supported") + + def emit_source(self, filename: str, root: Specification) -> None: + """Emit generic source code for this XDR type""" + raise NotImplementedError("Source boilerplate generation not supported") + + +class SourceGenerator: + """Base class to generate header and source code for XDR types""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + raise NotImplementedError("No language support defined") + + def emit_declaration(self, node: _XdrAst) -> None: + """Emit one function declaration for this XDR type""" + raise NotImplementedError("Declaration generation not supported") + + def emit_decoder(self, node: _XdrAst) -> None: + """Emit one decoder function for this XDR type""" + raise NotImplementedError("Decoder generation not supported") + + def emit_definition(self, node: _XdrAst) -> None: + """Emit one definition for this XDR type""" + raise NotImplementedError("Definition generation not supported") + + def emit_encoder(self, node: _XdrAst) -> None: + """Emit one encoder function for this XDR type""" + raise NotImplementedError("Encoder generation not supported") + + def emit_maxsize(self, node: _XdrAst) -> None: + """Emit one maxsize macro for this XDR type""" + raise NotImplementedError("Maxsize macro generation not supported") diff --git a/tools/net/sunrpc/xdrgen/generators/constant.py b/tools/net/sunrpc/xdrgen/generators/constant.py new file mode 100644 index 000000000..f2339caf0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/constant.py @@ -0,0 +1,20 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code to handle XDR constants""" + +from generators import SourceGenerator, create_jinja2_environment +from xdr_ast import _XdrConstant + +class XdrConstantGenerator(SourceGenerator): + """Generate source code for XDR constants""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "constants") + self.peer = peer + + def emit_definition(self, node: _XdrConstant) -> None: + """Emit one definition for a constant""" + template = self.environment.get_template("definition.j2") + print(template.render(name=node.name, value=node.value)) diff --git a/tools/net/sunrpc/xdrgen/generators/enum.py b/tools/net/sunrpc/xdrgen/generators/enum.py new file mode 100644 index 000000000..b4ed3ed64 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/enum.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code to handle XDR enum types""" + +from generators import SourceGenerator, create_jinja2_environment +from xdr_ast import _XdrEnum, public_apis, big_endian, get_header_name +from xdr_parse import get_xdr_enum_validation + + +class XdrEnumGenerator(SourceGenerator): + """Generate source code for XDR enum types""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "enum") + self.peer = peer + + def emit_declaration(self, node: _XdrEnum) -> None: + """Emit one declaration pair for an XDR enum type""" + if node.name in public_apis: + template = self.environment.get_template("declaration/enum.j2") + print(template.render(name=node.name)) + + def emit_definition(self, node: _XdrEnum) -> None: + """Emit one definition for an XDR enum type""" + template = self.environment.get_template("definition/open.j2") + print(template.render(name=node.name)) + + template = self.environment.get_template("definition/enumerator.j2") + for enumerator in node.enumerators: + print(template.render(name=enumerator.name, value=enumerator.value)) + + if node.name in big_endian: + template = self.environment.get_template("definition/close_be.j2") + else: + template = self.environment.get_template("definition/close.j2") + print(template.render(name=node.name)) + + def emit_decoder(self, node: _XdrEnum) -> None: + """Emit one decoder function for an XDR enum type""" + if node.name in big_endian: + template = self.environment.get_template("decoder/enum_be.j2") + else: + template = self.environment.get_template("decoder/enum.j2") + print( + template.render( + name=node.name, + enumerators=node.enumerators, + validate=get_xdr_enum_validation(), + ) + ) + + def emit_encoder(self, node: _XdrEnum) -> None: + """Emit one encoder function for an XDR enum type""" + if node.name in big_endian: + template = self.environment.get_template("encoder/enum_be.j2") + else: + template = self.environment.get_template("encoder/enum.j2") + print(template.render(name=node.name)) + + def emit_maxsize(self, node: _XdrEnum) -> None: + """Emit one maxsize macro for an XDR enum type""" + macro_name = get_header_name().upper() + "_" + node.name + "_sz" + template = self.environment.get_template("maxsize/enum.j2") + print( + template.render( + macro=macro_name, + width=" + ".join(node.symbolic_width()), + ) + ) diff --git a/tools/net/sunrpc/xdrgen/generators/header_bottom.py b/tools/net/sunrpc/xdrgen/generators/header_bottom.py new file mode 100644 index 000000000..4b55b282d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/header_bottom.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate header bottom boilerplate""" + +import os.path +import time + +from generators import Boilerplate, header_guard_infix +from generators import create_jinja2_environment, get_jinja2_template +from xdr_ast import Specification + + +class XdrHeaderBottomGenerator(Boilerplate): + """Generate header boilerplate""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "header_bottom") + self.peer = peer + + def emit_declaration(self, filename: str, root: Specification) -> None: + """Emit the bottom header guard""" + template = get_jinja2_template(self.environment, "declaration", "header") + print(template.render(infix=header_guard_infix(filename))) + + def emit_definition(self, filename: str, root: Specification) -> None: + """Emit the bottom header guard""" + template = get_jinja2_template(self.environment, "definition", "header") + print(template.render(infix=header_guard_infix(filename))) + + def emit_source(self, filename: str, root: Specification) -> None: + pass diff --git a/tools/net/sunrpc/xdrgen/generators/header_top.py b/tools/net/sunrpc/xdrgen/generators/header_top.py new file mode 100644 index 000000000..c6bc21c71 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/header_top.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate header top boilerplate""" + +import os.path +import time + +from generators import Boilerplate, header_guard_infix +from generators import create_jinja2_environment, get_jinja2_template +from xdr_ast import Specification + + +class XdrHeaderTopGenerator(Boilerplate): + """Generate header boilerplate""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "header_top") + self.peer = peer + + def emit_declaration(self, filename: str, root: Specification) -> None: + """Emit the top header guard""" + template = get_jinja2_template(self.environment, "declaration", "header") + print( + template.render( + infix=header_guard_infix(filename), + filename=filename, + mtime=time.ctime(os.path.getmtime(filename)), + ) + ) + + def emit_definition(self, filename: str, root: Specification) -> None: + """Emit the top header guard""" + template = get_jinja2_template(self.environment, "definition", "header") + print( + template.render( + infix=header_guard_infix(filename), + filename=filename, + mtime=time.ctime(os.path.getmtime(filename)), + ) + ) + + def emit_source(self, filename: str, root: Specification) -> None: + pass diff --git a/tools/net/sunrpc/xdrgen/generators/passthru.py b/tools/net/sunrpc/xdrgen/generators/passthru.py new file mode 100644 index 000000000..cb17bd977 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/passthru.py @@ -0,0 +1,26 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code for XDR pass-through lines""" + +from generators import SourceGenerator, create_jinja2_environment +from xdr_ast import _XdrPassthru + + +class XdrPassthruGenerator(SourceGenerator): + """Generate source code for XDR pass-through content""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "passthru") + self.peer = peer + + def emit_definition(self, node: _XdrPassthru) -> None: + """Emit one pass-through line""" + template = self.environment.get_template("definition.j2") + print(template.render(content=node.content)) + + def emit_decoder(self, node: _XdrPassthru) -> None: + """Emit one pass-through line""" + template = self.environment.get_template("source.j2") + print(template.render(content=node.content)) diff --git a/tools/net/sunrpc/xdrgen/generators/pointer.py b/tools/net/sunrpc/xdrgen/generators/pointer.py new file mode 100644 index 000000000..6dbda60ad --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/pointer.py @@ -0,0 +1,288 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code to handle XDR pointer types""" + +from jinja2 import Environment + +from generators import SourceGenerator, kernel_c_type +from generators import create_jinja2_environment, get_jinja2_template + +from xdr_ast import _XdrBasic, _XdrString +from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque +from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray +from xdr_ast import _XdrOptionalData, _XdrPointer, _XdrDeclaration +from xdr_ast import public_apis, get_header_name + + +def emit_pointer_declaration(environment: Environment, node: _XdrPointer) -> None: + """Emit a declaration pair for an XDR pointer type""" + if node.name in public_apis: + template = get_jinja2_template(environment, "declaration", "close") + print(template.render(name=node.name)) + + +def emit_pointer_member_definition( + environment: Environment, field: _XdrDeclaration +) -> None: + """Emit a definition for one field in an XDR struct""" + if isinstance(field, _XdrBasic): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "definition", field.template) + print(template.render(name=field.name)) + elif isinstance(field, _XdrString): + template = get_jinja2_template(environment, "definition", field.template) + print(template.render(name=field.name)) + elif isinstance(field, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrOptionalData): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + classifier=field.spec.c_classifier, + ) + ) + + +def emit_pointer_definition(environment: Environment, node: _XdrPointer) -> None: + """Emit a definition for an XDR pointer type""" + template = get_jinja2_template(environment, "definition", "open") + print(template.render(name=node.name)) + + for field in node.fields[0:-1]: + emit_pointer_member_definition(environment, field) + + template = get_jinja2_template(environment, "definition", "close") + print(template.render(name=node.name)) + + +def emit_pointer_member_decoder( + environment: Environment, field: _XdrDeclaration +) -> None: + """Emit a decoder for one field in an XDR pointer""" + if isinstance(field, _XdrBasic): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrString): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + size=field.size, + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + maxsize=field.maxsize, + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrOptionalData): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + classifier=field.spec.c_classifier, + ) + ) + + +def emit_pointer_decoder(environment: Environment, node: _XdrPointer) -> None: + """Emit one decoder function for an XDR pointer type""" + template = get_jinja2_template(environment, "decoder", "open") + print(template.render(name=node.name)) + + for field in node.fields[0:-1]: + emit_pointer_member_decoder(environment, field) + + template = get_jinja2_template(environment, "decoder", "close") + print(template.render()) + + +def emit_pointer_member_encoder( + environment: Environment, field: _XdrDeclaration +) -> None: + """Emit an encoder for one field in a XDR pointer""" + if isinstance(field, _XdrBasic): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + ) + ) + elif isinstance(field, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrString): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrOptionalData): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + classifier=field.spec.c_classifier, + ) + ) + + +def emit_pointer_encoder(environment: Environment, node: _XdrPointer) -> None: + """Emit one encoder function for an XDR pointer type""" + template = get_jinja2_template(environment, "encoder", "open") + print(template.render(name=node.name)) + + for field in node.fields[0:-1]: + emit_pointer_member_encoder(environment, field) + + template = get_jinja2_template(environment, "encoder", "close") + print(template.render()) + + +def emit_pointer_maxsize(environment: Environment, node: _XdrPointer) -> None: + """Emit one maxsize macro for an XDR pointer type""" + macro_name = get_header_name().upper() + "_" + node.name + "_sz" + template = get_jinja2_template(environment, "maxsize", "pointer") + print( + template.render( + macro=macro_name, + width=" + ".join(node.symbolic_width()), + ) + ) + + +class XdrPointerGenerator(SourceGenerator): + """Generate source code for XDR pointer""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "pointer") + self.peer = peer + + def emit_declaration(self, node: _XdrPointer) -> None: + """Emit one declaration pair for an XDR pointer type""" + emit_pointer_declaration(self.environment, node) + + def emit_definition(self, node: _XdrPointer) -> None: + """Emit one declaration for an XDR pointer type""" + emit_pointer_definition(self.environment, node) + + def emit_decoder(self, node: _XdrPointer) -> None: + """Emit one decoder function for an XDR pointer type""" + emit_pointer_decoder(self.environment, node) + + def emit_encoder(self, node: _XdrPointer) -> None: + """Emit one encoder function for an XDR pointer type""" + emit_pointer_encoder(self.environment, node) + + def emit_maxsize(self, node: _XdrPointer) -> None: + """Emit one maxsize macro for an XDR pointer type""" + emit_pointer_maxsize(self.environment, node) diff --git a/tools/net/sunrpc/xdrgen/generators/program.py b/tools/net/sunrpc/xdrgen/generators/program.py new file mode 100644 index 000000000..37f9655c8 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/program.py @@ -0,0 +1,212 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code for an RPC program's procedures""" + +from jinja2 import Environment + +from generators import SourceGenerator, create_jinja2_environment, get_jinja2_template +from xdr_ast import _RpcProgram, _RpcVersion, excluded_apis +from xdr_ast import max_widths, get_header_name + + +def emit_version_definitions( + environment: Environment, program: str, version: _RpcVersion +) -> None: + """Emit procedure numbers for each RPC version's procedures""" + template = environment.get_template("definition/open.j2") + print(template.render(program=program.upper())) + + template = environment.get_template("definition/procedure.j2") + for procedure in version.procedures: + if procedure.name not in excluded_apis: + print( + template.render( + name=procedure.name, + value=procedure.number, + ) + ) + + template = environment.get_template("definition/close.j2") + print(template.render()) + + +def emit_version_declarations( + environment: Environment, program: str, version: _RpcVersion +) -> None: + """Emit declarations for each RPC version's procedures""" + arguments = dict.fromkeys([]) + for procedure in version.procedures: + if procedure.name not in excluded_apis: + if procedure.argument.type_name == "void": + continue + arguments[procedure.argument.type_name] = None + if len(arguments) > 0: + print("") + template = environment.get_template("declaration/argument.j2") + for argument in arguments: + print(template.render(program=program, argument=argument)) + + results = dict.fromkeys([]) + for procedure in version.procedures: + if procedure.name not in excluded_apis: + if procedure.result.type_name == "void": + continue + results[procedure.result.type_name] = None + if len(results) > 0: + print("") + template = environment.get_template("declaration/result.j2") + for result in results: + print(template.render(program=program, result=result)) + + +def emit_version_argument_decoders( + environment: Environment, program: str, version: _RpcVersion +) -> None: + """Emit server argument decoders for each RPC version's procedures""" + arguments = dict.fromkeys([]) + for procedure in version.procedures: + if procedure.name not in excluded_apis: + if procedure.argument.type_name == "void": + continue + arguments[procedure.argument.type_name] = None + + template = environment.get_template("decoder/argument.j2") + for argument in arguments: + print(template.render(program=program, argument=argument)) + + +def emit_version_result_decoders( + environment: Environment, program: str, version: _RpcVersion +) -> None: + """Emit client result decoders for each RPC version's procedures""" + results = dict.fromkeys([]) + for procedure in version.procedures: + if procedure.name not in excluded_apis: + results[procedure.result.type_name] = None + + template = environment.get_template("decoder/result.j2") + for result in results: + print(template.render(program=program, result=result)) + + +def emit_version_argument_encoders( + environment: Environment, program: str, version: _RpcVersion +) -> None: + """Emit client argument encoders for each RPC version's procedures""" + arguments = dict.fromkeys([]) + for procedure in version.procedures: + if procedure.name not in excluded_apis: + arguments[procedure.argument.type_name] = None + + template = environment.get_template("encoder/argument.j2") + for argument in arguments: + print(template.render(program=program, argument=argument)) + + +def emit_version_result_encoders( + environment: Environment, program: str, version: _RpcVersion +) -> None: + """Emit server result encoders for each RPC version's procedures""" + results = dict.fromkeys([]) + for procedure in version.procedures: + if procedure.name not in excluded_apis: + if procedure.result.type_name == "void": + continue + results[procedure.result.type_name] = None + + template = environment.get_template("encoder/result.j2") + for result in results: + print(template.render(program=program, result=result)) + + +class XdrProgramGenerator(SourceGenerator): + """Generate source code for an RPC program's procedures""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "program") + self.peer = peer + + def emit_definition(self, node: _RpcProgram) -> None: + """Emit procedure numbers for each of an RPC programs's procedures""" + raw_name = node.name + program = raw_name.lower().removesuffix("_program").removesuffix("_prog") + + for version in node.versions: + emit_version_definitions(self.environment, program, version) + + template = self.environment.get_template("definition/program.j2") + print(template.render(name=raw_name, value=node.number)) + + def emit_declaration(self, node: _RpcProgram) -> None: + """Emit a declaration pair for each of an RPC programs's procedures""" + raw_name = node.name + program = raw_name.lower().removesuffix("_program").removesuffix("_prog") + + for version in node.versions: + emit_version_declarations(self.environment, program, version) + + def emit_decoder(self, node: _RpcProgram) -> None: + """Emit all decoder functions for an RPC program's procedures""" + raw_name = node.name + program = raw_name.lower().removesuffix("_program").removesuffix("_prog") + match self.peer: + case "server": + for version in node.versions: + emit_version_argument_decoders( + self.environment, program, version, + ) + case "client": + for version in node.versions: + emit_version_result_decoders( + self.environment, program, version, + ) + + def emit_encoder(self, node: _RpcProgram) -> None: + """Emit all encoder functions for an RPC program's procedures""" + raw_name = node.name + program = raw_name.lower().removesuffix("_program").removesuffix("_prog") + match self.peer: + case "server": + for version in node.versions: + emit_version_result_encoders( + self.environment, program, version, + ) + case "client": + for version in node.versions: + emit_version_argument_encoders( + self.environment, program, version, + ) + + def emit_maxsize(self, node: _RpcProgram) -> None: + """Emit maxsize macro for maximum RPC argument size""" + header = get_header_name().upper() + + # Find the largest argument across all versions + max_arg_width = 0 + max_arg_name = None + for version in node.versions: + for procedure in version.procedures: + if procedure.name in excluded_apis: + continue + arg_name = procedure.argument.type_name + if arg_name == "void": + continue + if arg_name not in max_widths: + continue + if max_widths[arg_name] > max_arg_width: + max_arg_width = max_widths[arg_name] + max_arg_name = arg_name + + if max_arg_name is None: + return + + macro_name = header + "_MAX_ARGS_SZ" + template = get_jinja2_template(self.environment, "maxsize", "max_args") + print( + template.render( + macro=macro_name, + width=header + "_" + max_arg_name + "_sz", + ) + ) diff --git a/tools/net/sunrpc/xdrgen/generators/source_top.py b/tools/net/sunrpc/xdrgen/generators/source_top.py new file mode 100644 index 000000000..bcf47d93d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/source_top.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate source code boilerplate""" + +import os.path +import time + +from generators import Boilerplate +from generators import find_xdr_program_name, create_jinja2_environment +from xdr_ast import _RpcProgram, Specification, get_header_name + + +class XdrSourceTopGenerator(Boilerplate): + """Generate source code boilerplate""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "source_top") + self.peer = peer + + def emit_source(self, filename: str, root: Specification) -> None: + """Emit the top source boilerplate""" + name = find_xdr_program_name(root) + template = self.environment.get_template(self.peer + ".j2") + print( + template.render( + program=name, + filename=filename, + mtime=time.ctime(os.path.getmtime(filename)), + ) + ) diff --git a/tools/net/sunrpc/xdrgen/generators/struct.py b/tools/net/sunrpc/xdrgen/generators/struct.py new file mode 100644 index 000000000..64911de46 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/struct.py @@ -0,0 +1,288 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code to handle XDR struct types""" + +from jinja2 import Environment + +from generators import SourceGenerator, kernel_c_type +from generators import create_jinja2_environment, get_jinja2_template + +from xdr_ast import _XdrBasic, _XdrString +from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque +from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray +from xdr_ast import _XdrOptionalData, _XdrStruct, _XdrDeclaration +from xdr_ast import public_apis, get_header_name + + +def emit_struct_declaration(environment: Environment, node: _XdrStruct) -> None: + """Emit one declaration pair for an XDR struct type""" + if node.name in public_apis: + template = get_jinja2_template(environment, "declaration", "close") + print(template.render(name=node.name)) + + +def emit_struct_member_definition( + environment: Environment, field: _XdrDeclaration +) -> None: + """Emit a definition for one field in an XDR struct""" + if isinstance(field, _XdrBasic): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "definition", field.template) + print(template.render(name=field.name)) + elif isinstance(field, _XdrString): + template = get_jinja2_template(environment, "definition", field.template) + print(template.render(name=field.name)) + elif isinstance(field, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrOptionalData): + template = get_jinja2_template(environment, "definition", field.template) + print( + template.render( + name=field.name, + type=kernel_c_type(field.spec), + classifier=field.spec.c_classifier, + ) + ) + + +def emit_struct_definition(environment: Environment, node: _XdrStruct) -> None: + """Emit one definition for an XDR struct type""" + template = get_jinja2_template(environment, "definition", "open") + print(template.render(name=node.name)) + + for field in node.fields: + emit_struct_member_definition(environment, field) + + template = get_jinja2_template(environment, "definition", "close") + print(template.render(name=node.name)) + + +def emit_struct_member_decoder( + environment: Environment, field: _XdrDeclaration +) -> None: + """Emit a decoder for one field in an XDR struct""" + if isinstance(field, _XdrBasic): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrString): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + size=field.size, + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + maxsize=field.maxsize, + classifier=field.spec.c_classifier, + ) + ) + elif isinstance(field, _XdrOptionalData): + template = get_jinja2_template(environment, "decoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + classifier=field.spec.c_classifier, + ) + ) + + +def emit_struct_decoder(environment: Environment, node: _XdrStruct) -> None: + """Emit one decoder function for an XDR struct type""" + template = get_jinja2_template(environment, "decoder", "open") + print(template.render(name=node.name)) + + for field in node.fields: + emit_struct_member_decoder(environment, field) + + template = get_jinja2_template(environment, "decoder", "close") + print(template.render()) + + +def emit_struct_member_encoder( + environment: Environment, field: _XdrDeclaration +) -> None: + """Emit an encoder for one field in an XDR struct""" + if isinstance(field, _XdrBasic): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + ) + ) + elif isinstance(field, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrString): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + size=field.size, + ) + ) + elif isinstance(field, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + maxsize=field.maxsize, + ) + ) + elif isinstance(field, _XdrOptionalData): + template = get_jinja2_template(environment, "encoder", field.template) + print( + template.render( + name=field.name, + type=field.spec.type_name, + classifier=field.spec.c_classifier, + ) + ) + + +def emit_struct_encoder(environment: Environment, node: _XdrStruct) -> None: + """Emit one encoder function for an XDR struct type""" + template = get_jinja2_template(environment, "encoder", "open") + print(template.render(name=node.name)) + + for field in node.fields: + emit_struct_member_encoder(environment, field) + + template = get_jinja2_template(environment, "encoder", "close") + print(template.render()) + + +def emit_struct_maxsize(environment: Environment, node: _XdrStruct) -> None: + """Emit one maxsize macro for an XDR struct type""" + macro_name = get_header_name().upper() + "_" + node.name + "_sz" + template = get_jinja2_template(environment, "maxsize", "struct") + print( + template.render( + macro=macro_name, + width=" + ".join(node.symbolic_width()), + ) + ) + + +class XdrStructGenerator(SourceGenerator): + """Generate source code for XDR structs""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "struct") + self.peer = peer + + def emit_declaration(self, node: _XdrStruct) -> None: + """Emit one declaration pair for an XDR struct type""" + emit_struct_declaration(self.environment, node) + + def emit_definition(self, node: _XdrStruct) -> None: + """Emit one definition for an XDR struct type""" + emit_struct_definition(self.environment, node) + + def emit_decoder(self, node: _XdrStruct) -> None: + """Emit one decoder function for an XDR struct type""" + emit_struct_decoder(self.environment, node) + + def emit_encoder(self, node: _XdrStruct) -> None: + """Emit one encoder function for an XDR struct type""" + emit_struct_encoder(self.environment, node) + + def emit_maxsize(self, node: _XdrStruct) -> None: + """Emit one maxsize macro for an XDR struct type""" + emit_struct_maxsize(self.environment, node) diff --git a/tools/net/sunrpc/xdrgen/generators/typedef.py b/tools/net/sunrpc/xdrgen/generators/typedef.py new file mode 100644 index 000000000..75e3a40e1 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/typedef.py @@ -0,0 +1,271 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code to handle XDR typedefs""" + +from jinja2 import Environment + +from generators import SourceGenerator, kernel_c_type +from generators import create_jinja2_environment, get_jinja2_template + +from xdr_ast import _XdrBasic, _XdrTypedef, _XdrString +from xdr_ast import _XdrFixedLengthOpaque, _XdrVariableLengthOpaque +from xdr_ast import _XdrFixedLengthArray, _XdrVariableLengthArray +from xdr_ast import _XdrOptionalData, _XdrVoid, _XdrDeclaration +from xdr_ast import public_apis, get_header_name + + +def emit_typedef_declaration(environment: Environment, node: _XdrDeclaration) -> None: + """Emit a declaration pair for one XDR typedef""" + if node.name not in public_apis: + return + if isinstance(node, _XdrBasic): + template = get_jinja2_template(environment, "declaration", node.template) + print( + template.render( + name=node.name, + type=kernel_c_type(node.spec), + classifier=node.spec.c_classifier, + ) + ) + elif isinstance(node, _XdrString): + template = get_jinja2_template(environment, "declaration", node.template) + print(template.render(name=node.name)) + elif isinstance(node, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "declaration", node.template) + print(template.render(name=node.name, size=node.size)) + elif isinstance(node, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "declaration", node.template) + print(template.render(name=node.name)) + elif isinstance(node, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "declaration", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + size=node.size, + ) + ) + elif isinstance(node, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "declaration", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + classifier=node.spec.c_classifier, + ) + ) + elif isinstance(node, _XdrOptionalData): + raise NotImplementedError("<optional_data> typedef not yet implemented") + elif isinstance(node, _XdrVoid): + raise ValueError("invalid void usage in RPC Specification") + else: + raise NotImplementedError("typedef: type not recognized") + + +def emit_type_definition(environment: Environment, node: _XdrDeclaration) -> None: + """Emit a definition for one XDR typedef""" + if isinstance(node, _XdrBasic): + template = get_jinja2_template(environment, "definition", node.template) + print( + template.render( + name=node.name, + type=kernel_c_type(node.spec), + classifier=node.spec.c_classifier, + ) + ) + elif isinstance(node, _XdrString): + template = get_jinja2_template(environment, "definition", node.template) + print(template.render(name=node.name)) + elif isinstance(node, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "definition", node.template) + print(template.render(name=node.name, size=node.size)) + elif isinstance(node, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "definition", node.template) + print(template.render(name=node.name)) + elif isinstance(node, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "definition", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + size=node.size, + ) + ) + elif isinstance(node, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "definition", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + classifier=node.spec.c_classifier, + ) + ) + elif isinstance(node, _XdrOptionalData): + raise NotImplementedError("<optional_data> typedef not yet implemented") + elif isinstance(node, _XdrVoid): + raise ValueError("invalid void usage in RPC Specification") + else: + raise NotImplementedError("typedef: type not recognized") + + +def emit_typedef_decoder(environment: Environment, node: _XdrDeclaration) -> None: + """Emit a decoder function for one XDR typedef""" + if isinstance(node, _XdrBasic): + template = get_jinja2_template(environment, "decoder", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + ) + ) + elif isinstance(node, _XdrString): + template = get_jinja2_template(environment, "decoder", node.template) + print( + template.render( + name=node.name, + maxsize=node.maxsize, + ) + ) + elif isinstance(node, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "decoder", node.template) + print( + template.render( + name=node.name, + size=node.size, + ) + ) + elif isinstance(node, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "decoder", node.template) + print( + template.render( + name=node.name, + maxsize=node.maxsize, + ) + ) + elif isinstance(node, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "decoder", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + size=node.size, + classifier=node.spec.c_classifier, + ) + ) + elif isinstance(node, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "decoder", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + maxsize=node.maxsize, + ) + ) + elif isinstance(node, _XdrOptionalData): + raise NotImplementedError("<optional_data> typedef not yet implemented") + elif isinstance(node, _XdrVoid): + raise ValueError("invalid void usage in RPC Specification") + else: + raise NotImplementedError("typedef: type not recognized") + + +def emit_typedef_encoder(environment: Environment, node: _XdrDeclaration) -> None: + """Emit an encoder function for one XDR typedef""" + if isinstance(node, _XdrBasic): + template = get_jinja2_template(environment, "encoder", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + ) + ) + elif isinstance(node, _XdrString): + template = get_jinja2_template(environment, "encoder", node.template) + print( + template.render( + name=node.name, + maxsize=node.maxsize, + ) + ) + elif isinstance(node, _XdrFixedLengthOpaque): + template = get_jinja2_template(environment, "encoder", node.template) + print( + template.render( + name=node.name, + size=node.size, + ) + ) + elif isinstance(node, _XdrVariableLengthOpaque): + template = get_jinja2_template(environment, "encoder", node.template) + print( + template.render( + name=node.name, + maxsize=node.maxsize, + ) + ) + elif isinstance(node, _XdrFixedLengthArray): + template = get_jinja2_template(environment, "encoder", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + size=node.size, + ) + ) + elif isinstance(node, _XdrVariableLengthArray): + template = get_jinja2_template(environment, "encoder", node.template) + print( + template.render( + name=node.name, + type=node.spec.type_name, + maxsize=node.maxsize, + ) + ) + elif isinstance(node, _XdrOptionalData): + raise NotImplementedError("<optional_data> typedef not yet implemented") + elif isinstance(node, _XdrVoid): + raise ValueError("invalid void usage in RPC Specification") + else: + raise NotImplementedError("typedef: type not recognized") + + +def emit_typedef_maxsize(environment: Environment, node: _XdrDeclaration) -> None: + """Emit a maxsize macro for an XDR typedef""" + macro_name = get_header_name().upper() + "_" + node.name + "_sz" + template = get_jinja2_template(environment, "maxsize", node.template) + print( + template.render( + macro=macro_name, + width=" + ".join(node.symbolic_width()), + ) + ) + + +class XdrTypedefGenerator(SourceGenerator): + """Generate source code for XDR typedefs""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "typedef") + self.peer = peer + + def emit_declaration(self, node: _XdrTypedef) -> None: + """Emit one declaration pair for an XDR enum type""" + emit_typedef_declaration(self.environment, node.declaration) + + def emit_definition(self, node: _XdrTypedef) -> None: + """Emit one definition for an XDR typedef""" + emit_type_definition(self.environment, node.declaration) + + def emit_decoder(self, node: _XdrTypedef) -> None: + """Emit one decoder function for an XDR typedef""" + emit_typedef_decoder(self.environment, node.declaration) + + def emit_encoder(self, node: _XdrTypedef) -> None: + """Emit one encoder function for an XDR typedef""" + emit_typedef_encoder(self.environment, node.declaration) + + def emit_maxsize(self, node: _XdrTypedef) -> None: + """Emit one maxsize macro for an XDR typedef""" + emit_typedef_maxsize(self.environment, node.declaration) diff --git a/tools/net/sunrpc/xdrgen/generators/union.py b/tools/net/sunrpc/xdrgen/generators/union.py new file mode 100644 index 000000000..d15837dae --- /dev/null +++ b/tools/net/sunrpc/xdrgen/generators/union.py @@ -0,0 +1,366 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Generate code to handle XDR unions""" + +from jinja2 import Environment + +from generators import SourceGenerator +from generators import create_jinja2_environment, get_jinja2_template + +from xdr_ast import _XdrBasic, _XdrUnion, _XdrVoid, _XdrString, get_header_name +from xdr_ast import _XdrDeclaration, _XdrCaseSpec, public_apis, big_endian + + +def emit_union_declaration(environment: Environment, node: _XdrUnion) -> None: + """Emit one declaration pair for an XDR union type""" + if node.name in public_apis: + template = get_jinja2_template(environment, "declaration", "close") + print(template.render(name=node.name)) + + +def emit_union_switch_spec_definition( + environment: Environment, node: _XdrDeclaration +) -> None: + """Emit a definition for an XDR union's discriminant""" + assert isinstance(node, _XdrBasic) + template = get_jinja2_template(environment, "definition", "switch_spec") + print( + template.render( + name=node.name, + type=node.spec.type_name, + classifier=node.spec.c_classifier, + ) + ) + + +def emit_union_case_spec_definition( + environment: Environment, node: _XdrDeclaration +) -> None: + """Emit a definition for an XDR union's case arm""" + if isinstance(node.arm, _XdrVoid): + return + if isinstance(node.arm, _XdrString): + type_name = "char *" + classifier = "" + else: + type_name = node.arm.spec.type_name + classifier = node.arm.spec.c_classifier + + assert isinstance(node.arm, (_XdrBasic, _XdrString)) + template = get_jinja2_template(environment, "definition", "case_spec") + print( + template.render( + name=node.arm.name, + type=type_name, + classifier=classifier, + ) + ) + + +def emit_union_definition(environment: Environment, node: _XdrUnion) -> None: + """Emit one XDR union definition""" + template = get_jinja2_template(environment, "definition", "open") + print(template.render(name=node.name)) + + emit_union_switch_spec_definition(environment, node.discriminant) + + for case in node.cases: + emit_union_case_spec_definition(environment, case) + + if node.default is not None: + emit_union_case_spec_definition(environment, node.default) + + template = get_jinja2_template(environment, "definition", "close") + print(template.render(name=node.name)) + + +def emit_union_switch_spec_decoder( + environment: Environment, node: _XdrDeclaration +) -> None: + """Emit a decoder for an XDR union's discriminant""" + assert isinstance(node, _XdrBasic) + template = get_jinja2_template(environment, "decoder", "switch_spec") + print(template.render(name=node.name, type=node.spec.type_name)) + + +def emit_union_arm_decoder( + environment: Environment, node: _XdrCaseSpec +) -> None: + """Emit decoder for an XDR union's arm (data only, no case/break)""" + + if isinstance(node.arm, _XdrVoid): + return + if isinstance(node.arm, _XdrString): + type_name = "char *" + classifier = "" + else: + type_name = node.arm.spec.type_name + classifier = node.arm.spec.c_classifier + + assert isinstance(node.arm, (_XdrBasic, _XdrString)) + template = get_jinja2_template(environment, "decoder", node.arm.template) + print( + template.render( + name=node.arm.name, + type=type_name, + classifier=classifier, + ) + ) + + +def emit_union_case_spec_decoder( + environment: Environment, node: _XdrCaseSpec, big_endian_discriminant: bool +) -> None: + """Emit decoder functions for an XDR union's case arm""" + + if isinstance(node.arm, _XdrVoid): + return + if isinstance(node.arm, _XdrString): + type_name = "char *" + classifier = "" + else: + type_name = node.arm.spec.type_name + classifier = node.arm.spec.c_classifier + + if big_endian_discriminant: + template = get_jinja2_template(environment, "decoder", "case_spec_be") + else: + template = get_jinja2_template(environment, "decoder", "case_spec") + for case in node.values: + print(template.render(case=case)) + + assert isinstance(node.arm, (_XdrBasic, _XdrString)) + template = get_jinja2_template(environment, "decoder", node.arm.template) + print( + template.render( + name=node.arm.name, + type=type_name, + classifier=classifier, + ) + ) + + template = get_jinja2_template(environment, "decoder", "break") + print(template.render()) + + +def emit_union_default_spec_decoder(environment: Environment, node: _XdrUnion) -> None: + """Emit a decoder function for an XDR union's default arm""" + default_case = node.default + + # Avoid a gcc warning about a default case with boolean discriminant + if default_case is None and node.discriminant.spec.type_name == "bool": + return + + template = get_jinja2_template(environment, "decoder", "default_spec") + print(template.render()) + + if default_case is None or isinstance(default_case.arm, _XdrVoid): + template = get_jinja2_template(environment, "decoder", "break") + print(template.render()) + return + + assert isinstance(default_case.arm, _XdrBasic) + template = get_jinja2_template(environment, "decoder", default_case.arm.template) + print( + template.render( + name=default_case.arm.name, + type=default_case.arm.spec.type_name, + classifier=default_case.arm.spec.c_classifier, + ) + ) + + +def emit_union_decoder(environment: Environment, node: _XdrUnion) -> None: + """Emit one XDR union decoder""" + template = get_jinja2_template(environment, "decoder", "open") + print(template.render(name=node.name)) + + # For boolean discriminants, use if statement instead of switch + if node.discriminant.spec.type_name == "bool": + template = get_jinja2_template(environment, "decoder", "bool_spec") + print(template.render(name=node.discriminant.name, type=node.discriminant.spec.type_name)) + + # Find and emit the TRUE case + for case in node.cases: + if case.values and case.values[0] == "TRUE": + emit_union_arm_decoder(environment, case) + break + + template = get_jinja2_template(environment, "decoder", "close") + print(template.render()) + else: + emit_union_switch_spec_decoder(environment, node.discriminant) + + for case in node.cases: + emit_union_case_spec_decoder( + environment, + case, + node.discriminant.spec.type_name in big_endian, + ) + + emit_union_default_spec_decoder(environment, node) + + template = get_jinja2_template(environment, "decoder", "close") + print(template.render()) + + +def emit_union_switch_spec_encoder( + environment: Environment, node: _XdrDeclaration +) -> None: + """Emit an encoder for an XDR union's discriminant""" + assert isinstance(node, _XdrBasic) + template = get_jinja2_template(environment, "encoder", "switch_spec") + print(template.render(name=node.name, type=node.spec.type_name)) + + +def emit_union_arm_encoder( + environment: Environment, node: _XdrCaseSpec +) -> None: + """Emit encoder for an XDR union's arm (data only, no case/break)""" + + if isinstance(node.arm, _XdrVoid): + return + if isinstance(node.arm, _XdrString): + type_name = "char *" + else: + type_name = node.arm.spec.type_name + + assert isinstance(node.arm, (_XdrBasic, _XdrString)) + template = get_jinja2_template(environment, "encoder", node.arm.template) + print( + template.render( + name=node.arm.name, + type=type_name, + ) + ) + + +def emit_union_case_spec_encoder( + environment: Environment, node: _XdrCaseSpec, big_endian_discriminant: bool +) -> None: + """Emit encoder functions for an XDR union's case arm""" + + if isinstance(node.arm, _XdrVoid): + return + if isinstance(node.arm, _XdrString): + type_name = "char *" + else: + type_name = node.arm.spec.type_name + if big_endian_discriminant: + template = get_jinja2_template(environment, "encoder", "case_spec_be") + else: + template = get_jinja2_template(environment, "encoder", "case_spec") + for case in node.values: + print(template.render(case=case)) + + template = get_jinja2_template(environment, "encoder", node.arm.template) + print( + template.render( + name=node.arm.name, + type=type_name, + ) + ) + + template = get_jinja2_template(environment, "encoder", "break") + print(template.render()) + + +def emit_union_default_spec_encoder(environment: Environment, node: _XdrUnion) -> None: + """Emit an encoder function for an XDR union's default arm""" + default_case = node.default + + # Avoid a gcc warning about a default case with boolean discriminant + if default_case is None and node.discriminant.spec.type_name == "bool": + return + + template = get_jinja2_template(environment, "encoder", "default_spec") + print(template.render()) + + if default_case is None or isinstance(default_case.arm, _XdrVoid): + template = get_jinja2_template(environment, "encoder", "break") + print(template.render()) + return + + template = get_jinja2_template(environment, "encoder", default_case.arm.template) + print( + template.render( + name=default_case.arm.name, + type=default_case.arm.spec.type_name, + ) + ) + + +def emit_union_encoder(environment, node: _XdrUnion) -> None: + """Emit one XDR union encoder""" + template = get_jinja2_template(environment, "encoder", "open") + print(template.render(name=node.name)) + + # For boolean discriminants, use if statement instead of switch + if node.discriminant.spec.type_name == "bool": + template = get_jinja2_template(environment, "encoder", "bool_spec") + print(template.render(name=node.discriminant.name, type=node.discriminant.spec.type_name)) + + # Find and emit the TRUE case + for case in node.cases: + if case.values and case.values[0] == "TRUE": + emit_union_arm_encoder(environment, case) + break + + template = get_jinja2_template(environment, "encoder", "close") + print(template.render()) + else: + emit_union_switch_spec_encoder(environment, node.discriminant) + + for case in node.cases: + emit_union_case_spec_encoder( + environment, + case, + node.discriminant.spec.type_name in big_endian, + ) + + emit_union_default_spec_encoder(environment, node) + + template = get_jinja2_template(environment, "encoder", "close") + print(template.render()) + + +def emit_union_maxsize(environment: Environment, node: _XdrUnion) -> None: + """Emit one maxsize macro for an XDR union type""" + macro_name = get_header_name().upper() + "_" + node.name + "_sz" + template = get_jinja2_template(environment, "maxsize", "union") + print( + template.render( + macro=macro_name, + width=" + ".join(node.symbolic_width()), + ) + ) + + +class XdrUnionGenerator(SourceGenerator): + """Generate source code for XDR unions""" + + def __init__(self, language: str, peer: str): + """Initialize an instance of this class""" + self.environment = create_jinja2_environment(language, "union") + self.peer = peer + + def emit_declaration(self, node: _XdrUnion) -> None: + """Emit one declaration pair for an XDR union""" + emit_union_declaration(self.environment, node) + + def emit_definition(self, node: _XdrUnion) -> None: + """Emit one definition for an XDR union""" + emit_union_definition(self.environment, node) + + def emit_decoder(self, node: _XdrUnion) -> None: + """Emit one decoder function for an XDR union""" + emit_union_decoder(self.environment, node) + + def emit_encoder(self, node: _XdrUnion) -> None: + """Emit one encoder function for an XDR union""" + emit_union_encoder(self.environment, node) + + def emit_maxsize(self, node: _XdrUnion) -> None: + """Emit one maxsize macro for an XDR union""" + emit_union_maxsize(self.environment, node) diff --git a/tools/net/sunrpc/xdrgen/grammars/xdr.lark b/tools/net/sunrpc/xdrgen/grammars/xdr.lark new file mode 100644 index 000000000..1d2afff98 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/grammars/xdr.lark @@ -0,0 +1,127 @@ +// A Lark grammar for the XDR specification language based on +// https://tools.ietf.org/html/rfc4506 Section 6.3 + +declaration : "opaque" identifier "[" value "]" -> fixed_length_opaque + | "opaque" identifier "<" [ value ] ">" -> variable_length_opaque + | "string" identifier "<" [ value ] ">" -> string + | type_specifier identifier "[" value "]" -> fixed_length_array + | type_specifier identifier "<" [ value ] ">" -> variable_length_array + | type_specifier "*" identifier -> optional_data + | type_specifier identifier -> basic + | "void" -> void + +value : decimal_constant + | hexadecimal_constant + | octal_constant + | identifier + +constant : decimal_constant | hexadecimal_constant | octal_constant + +type_specifier : unsigned_hyper + | unsigned_long + | unsigned_int + | unsigned_short + | hyper + | long + | int + | short + | float + | double + | quadruple + | bool + | enum_type_spec + | struct_type_spec + | union_type_spec + | identifier + +unsigned_hyper : "unsigned" "hyper" +unsigned_long : "unsigned" "long" +unsigned_int : "unsigned" "int" +unsigned_short : "unsigned" "short" +hyper : "hyper" +long : "long" +int : "int" +short : "short" +float : "float" +double : "double" +quadruple : "quadruple" +bool : "bool" + +enum_type_spec : "enum" enum_body + +enum_body : "{" ( identifier "=" value ) ( "," identifier "=" value )* "}" + +struct_type_spec : "struct" struct_body + +struct_body : "{" ( declaration ";" )+ "}" + +union_type_spec : "union" union_body + +union_body : switch_spec "{" case_spec+ [ default_spec ] "}" + +switch_spec : "switch" "(" declaration ")" + +case_spec : ( "case" value ":" )+ declaration ";" + +default_spec : "default" ":" declaration ";" + +constant_def : "const" identifier "=" value ";" + +type_def : "typedef" declaration ";" -> typedef + | "enum" identifier enum_body ";" -> enum + | "struct" identifier struct_body ";" -> struct + | "union" identifier union_body ";" -> union + +specification : definition* + +definition : constant_def + | type_def + | program_def + | pragma_def + | passthru_def + +passthru_def : PASSTHRU + +// +// RPC program definitions not specified in RFC 4506 +// + +program_def : "program" identifier "{" version_def+ "}" "=" constant ";" + +version_def : "version" identifier "{" procedure_def+ "}" "=" constant ";" + +procedure_def : type_specifier identifier "(" type_specifier ")" "=" constant ";" + +pragma_def : "pragma" directive identifier [ identifier ] ";" + +directive : big_endian_directive + | exclude_directive + | header_directive + | pages_directive + | public_directive + | skip_directive + +big_endian_directive : "big_endian" +exclude_directive : "exclude" +header_directive : "header" +pages_directive : "pages" +public_directive : "public" +skip_directive : "skip" + +// +// XDR language primitives +// + +identifier : /([a-z]|[A-Z])(_|[a-z]|[A-Z]|[0-9])*/ + +decimal_constant : /[\+-]?(0|[1-9][0-9]*)/ +hexadecimal_constant : /0x([a-f]|[A-F]|[0-9])+/ +octal_constant : /0[0-7]+/ + +PASSTHRU : /%.*/ + +%import common.C_COMMENT +%ignore C_COMMENT + +%import common.WS +%ignore WS diff --git a/tools/net/sunrpc/xdrgen/subcmds/__init__.py b/tools/net/sunrpc/xdrgen/subcmds/__init__.py new file mode 100644 index 000000000..c940e9275 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/subcmds/__init__.py @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +# Just to make sphinx-apidoc document this directory diff --git a/tools/net/sunrpc/xdrgen/subcmds/declarations.py b/tools/net/sunrpc/xdrgen/subcmds/declarations.py new file mode 100644 index 000000000..f18761146 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/subcmds/declarations.py @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Translate an XDR specification into executable code that +can be compiled for the Linux kernel.""" + +import logging + +from argparse import Namespace +from lark import logger +from lark.exceptions import VisitError + +from generators.enum import XdrEnumGenerator +from generators.header_bottom import XdrHeaderBottomGenerator +from generators.header_top import XdrHeaderTopGenerator +from generators.pointer import XdrPointerGenerator +from generators.program import XdrProgramGenerator +from generators.typedef import XdrTypedefGenerator +from generators.struct import XdrStructGenerator +from generators.union import XdrUnionGenerator + +from xdr_ast import transform_parse_tree, _RpcProgram, Specification +from xdr_ast import _XdrEnum, _XdrPointer, _XdrTypedef, _XdrStruct, _XdrUnion +from xdr_ast import XdrSemanticError +from xdr_parse import xdr_parser, set_xdr_annotate +from xdr_parse import make_error_handler, XdrParseError +from xdr_parse import handle_transform_error, handle_semantic_error + +logger.setLevel(logging.INFO) + + +def emit_header_declarations(root: Specification, language: str, peer: str) -> None: + """Emit header declarations""" + for definition in root.definitions: + if isinstance(definition.value, _XdrEnum): + gen = XdrEnumGenerator(language, peer) + elif isinstance(definition.value, _XdrPointer): + gen = XdrPointerGenerator(language, peer) + elif isinstance(definition.value, _XdrTypedef): + gen = XdrTypedefGenerator(language, peer) + elif isinstance(definition.value, _XdrStruct): + gen = XdrStructGenerator(language, peer) + elif isinstance(definition.value, _XdrUnion): + gen = XdrUnionGenerator(language, peer) + elif isinstance(definition.value, _RpcProgram): + gen = XdrProgramGenerator(language, peer) + else: + continue + gen.emit_declaration(definition.value) + + +def subcmd(args: Namespace) -> int: + """Generate definitions and declarations""" + + set_xdr_annotate(args.annotate) + parser = xdr_parser() + with open(args.filename, encoding="utf-8") as f: + source = f.read() + try: + parse_tree = parser.parse( + source, on_error=make_error_handler(source, args.filename) + ) + except XdrParseError: + return 1 + try: + ast = transform_parse_tree(parse_tree) + except VisitError as e: + handle_transform_error(e, source, args.filename) + return 1 + except XdrSemanticError as e: + handle_semantic_error(e, source, args.filename) + return 1 + + gen = XdrHeaderTopGenerator(args.language, args.peer) + gen.emit_declaration(args.filename, ast) + + emit_header_declarations(ast, args.language, args.peer) + + gen = XdrHeaderBottomGenerator(args.language, args.peer) + gen.emit_declaration(args.filename, ast) + + return 0 diff --git a/tools/net/sunrpc/xdrgen/subcmds/definitions.py b/tools/net/sunrpc/xdrgen/subcmds/definitions.py new file mode 100644 index 000000000..77b666943 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/subcmds/definitions.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Translate an XDR specification into executable code that +can be compiled for the Linux kernel.""" + +import logging + +from argparse import Namespace +from lark import logger +from lark.exceptions import VisitError + +from generators.constant import XdrConstantGenerator +from generators.enum import XdrEnumGenerator +from generators.header_bottom import XdrHeaderBottomGenerator +from generators.header_top import XdrHeaderTopGenerator +from generators.passthru import XdrPassthruGenerator +from generators.pointer import XdrPointerGenerator +from generators.program import XdrProgramGenerator +from generators.typedef import XdrTypedefGenerator +from generators.struct import XdrStructGenerator +from generators.union import XdrUnionGenerator + +from xdr_ast import transform_parse_tree, Specification, XdrSemanticError +from xdr_ast import _RpcProgram, _XdrConstant, _XdrEnum, _XdrPassthru, _XdrPointer +from xdr_ast import _XdrTypedef, _XdrStruct, _XdrUnion +from xdr_parse import xdr_parser, set_xdr_annotate +from xdr_parse import make_error_handler, XdrParseError +from xdr_parse import handle_transform_error, handle_semantic_error + +logger.setLevel(logging.INFO) + + +def emit_header_definitions(root: Specification, language: str, peer: str) -> None: + """Emit header definitions""" + for definition in root.definitions: + if isinstance(definition.value, _XdrConstant): + gen = XdrConstantGenerator(language, peer) + elif isinstance(definition.value, _XdrEnum): + gen = XdrEnumGenerator(language, peer) + elif isinstance(definition.value, _XdrPointer): + gen = XdrPointerGenerator(language, peer) + elif isinstance(definition.value, _RpcProgram): + gen = XdrProgramGenerator(language, peer) + elif isinstance(definition.value, _XdrTypedef): + gen = XdrTypedefGenerator(language, peer) + elif isinstance(definition.value, _XdrStruct): + gen = XdrStructGenerator(language, peer) + elif isinstance(definition.value, _XdrUnion): + gen = XdrUnionGenerator(language, peer) + elif isinstance(definition.value, _XdrPassthru): + gen = XdrPassthruGenerator(language, peer) + else: + continue + gen.emit_definition(definition.value) + + +def emit_header_maxsize(root: Specification, language: str, peer: str) -> None: + """Emit header maxsize macros""" + print("") + for definition in root.definitions: + if isinstance(definition.value, _XdrEnum): + gen = XdrEnumGenerator(language, peer) + elif isinstance(definition.value, _XdrPointer): + gen = XdrPointerGenerator(language, peer) + elif isinstance(definition.value, _XdrTypedef): + gen = XdrTypedefGenerator(language, peer) + elif isinstance(definition.value, _XdrStruct): + gen = XdrStructGenerator(language, peer) + elif isinstance(definition.value, _XdrUnion): + gen = XdrUnionGenerator(language, peer) + elif isinstance(definition.value, _RpcProgram): + gen = XdrProgramGenerator(language, peer) + else: + continue + gen.emit_maxsize(definition.value) + + +def subcmd(args: Namespace) -> int: + """Generate definitions""" + + set_xdr_annotate(args.annotate) + parser = xdr_parser() + with open(args.filename, encoding="utf-8") as f: + source = f.read() + try: + parse_tree = parser.parse( + source, on_error=make_error_handler(source, args.filename) + ) + except XdrParseError: + return 1 + try: + ast = transform_parse_tree(parse_tree) + except VisitError as e: + handle_transform_error(e, source, args.filename) + return 1 + except XdrSemanticError as e: + handle_semantic_error(e, source, args.filename) + return 1 + + gen = XdrHeaderTopGenerator(args.language, args.peer) + gen.emit_definition(args.filename, ast) + + emit_header_definitions(ast, args.language, args.peer) + emit_header_maxsize(ast, args.language, args.peer) + + gen = XdrHeaderBottomGenerator(args.language, args.peer) + gen.emit_definition(args.filename, ast) + + return 0 diff --git a/tools/net/sunrpc/xdrgen/subcmds/lint.py b/tools/net/sunrpc/xdrgen/subcmds/lint.py new file mode 100644 index 000000000..b4ea0f55f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/subcmds/lint.py @@ -0,0 +1,41 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Translate an XDR specification into executable code that +can be compiled for the Linux kernel.""" + +import logging + +from argparse import Namespace +from lark import logger +from lark.exceptions import VisitError + +from xdr_parse import xdr_parser, make_error_handler, XdrParseError +from xdr_parse import handle_transform_error, handle_semantic_error +from xdr_ast import transform_parse_tree, XdrSemanticError + +logger.setLevel(logging.DEBUG) + + +def subcmd(args: Namespace) -> int: + """Lexical and syntax check of an XDR specification""" + + parser = xdr_parser() + with open(args.filename, encoding="utf-8") as f: + source = f.read() + try: + parse_tree = parser.parse( + source, on_error=make_error_handler(source, args.filename) + ) + except XdrParseError: + return 1 + try: + transform_parse_tree(parse_tree) + except VisitError as e: + handle_transform_error(e, source, args.filename) + return 1 + except XdrSemanticError as e: + handle_semantic_error(e, source, args.filename) + return 1 + + return 0 diff --git a/tools/net/sunrpc/xdrgen/subcmds/source.py b/tools/net/sunrpc/xdrgen/subcmds/source.py new file mode 100644 index 000000000..56eba34d8 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/subcmds/source.py @@ -0,0 +1,137 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Translate an XDR specification into executable code that +can be compiled for the Linux kernel.""" + +import logging + +from argparse import Namespace +from lark import logger +from lark.exceptions import VisitError + +from generators.source_top import XdrSourceTopGenerator +from generators.enum import XdrEnumGenerator +from generators.passthru import XdrPassthruGenerator +from generators.pointer import XdrPointerGenerator +from generators.program import XdrProgramGenerator +from generators.typedef import XdrTypedefGenerator +from generators.struct import XdrStructGenerator +from generators.union import XdrUnionGenerator + +from xdr_ast import transform_parse_tree, _RpcProgram, Specification +from xdr_ast import _XdrAst, _XdrEnum, _XdrPassthru, _XdrPointer +from xdr_ast import _XdrStruct, _XdrTypedef, _XdrUnion, XdrSemanticError + +from xdr_parse import xdr_parser, set_xdr_annotate, set_xdr_enum_validation +from xdr_parse import make_error_handler, XdrParseError +from xdr_parse import handle_transform_error, handle_semantic_error + +logger.setLevel(logging.INFO) + + +def emit_source_decoder(node: _XdrAst, language: str, peer: str) -> None: + """Emit one XDR decoder function for a source file""" + if isinstance(node, _XdrEnum): + gen = XdrEnumGenerator(language, peer) + elif isinstance(node, _XdrPointer): + gen = XdrPointerGenerator(language, peer) + elif isinstance(node, _XdrTypedef): + gen = XdrTypedefGenerator(language, peer) + elif isinstance(node, _XdrStruct): + gen = XdrStructGenerator(language, peer) + elif isinstance(node, _XdrUnion): + gen = XdrUnionGenerator(language, peer) + elif isinstance(node, _RpcProgram): + gen = XdrProgramGenerator(language, peer) + else: + return + gen.emit_decoder(node) + + +def emit_source_encoder(node: _XdrAst, language: str, peer: str) -> None: + """Emit one XDR encoder function for a source file""" + if isinstance(node, _XdrEnum): + gen = XdrEnumGenerator(language, peer) + elif isinstance(node, _XdrPointer): + gen = XdrPointerGenerator(language, peer) + elif isinstance(node, _XdrTypedef): + gen = XdrTypedefGenerator(language, peer) + elif isinstance(node, _XdrStruct): + gen = XdrStructGenerator(language, peer) + elif isinstance(node, _XdrUnion): + gen = XdrUnionGenerator(language, peer) + elif isinstance(node, _RpcProgram): + gen = XdrProgramGenerator(language, peer) + else: + return + gen.emit_encoder(node) + + +def generate_server_source(filename: str, root: Specification, language: str) -> None: + """Generate server-side source code""" + + gen = XdrSourceTopGenerator(language, "server") + gen.emit_source(filename, root) + + for definition in root.definitions: + if isinstance(definition.value, _XdrPassthru): + passthru_gen = XdrPassthruGenerator(language, "server") + passthru_gen.emit_decoder(definition.value) + else: + emit_source_decoder(definition.value, language, "server") + for definition in root.definitions: + if not isinstance(definition.value, _XdrPassthru): + emit_source_encoder(definition.value, language, "server") + + +def generate_client_source(filename: str, root: Specification, language: str) -> None: + """Generate client-side source code""" + + gen = XdrSourceTopGenerator(language, "client") + gen.emit_source(filename, root) + + for definition in root.definitions: + if isinstance(definition.value, _XdrPassthru): + passthru_gen = XdrPassthruGenerator(language, "client") + passthru_gen.emit_decoder(definition.value) + else: + emit_source_encoder(definition.value, language, "client") + for definition in root.definitions: + if not isinstance(definition.value, _XdrPassthru): + emit_source_decoder(definition.value, language, "client") + + # cel: todo: client needs PROC macros + + +def subcmd(args: Namespace) -> int: + """Generate encoder and decoder functions""" + + set_xdr_annotate(args.annotate) + set_xdr_enum_validation(not args.no_enum_validation) + parser = xdr_parser() + with open(args.filename, encoding="utf-8") as f: + source = f.read() + try: + parse_tree = parser.parse( + source, on_error=make_error_handler(source, args.filename) + ) + except XdrParseError: + return 1 + try: + ast = transform_parse_tree(parse_tree) + except VisitError as e: + handle_transform_error(e, source, args.filename) + return 1 + except XdrSemanticError as e: + handle_semantic_error(e, source, args.filename) + return 1 + match args.peer: + case "server": + generate_server_source(args.filename, ast, args.language) + case "client": + generate_client_source(args.filename, ast, args.language) + case _: + print("Code generation for", args.peer, "is not yet supported") + + return 0 diff --git a/tools/net/sunrpc/xdrgen/templates/C/constants/definition.j2 b/tools/net/sunrpc/xdrgen/templates/C/constants/definition.j2 new file mode 100644 index 000000000..d648ca419 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/constants/definition.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +enum { {{ name }} = {{ value }} }; diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/declaration/enum.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/declaration/enum.j2 new file mode 100644 index 000000000..d1405c7c5 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/declaration/enum.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, {{ name }} value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2 new file mode 100644 index 000000000..735a34157 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2 @@ -0,0 +1,30 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* enum {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr) +{ + u32 val; + + if (xdr_stream_decode_u32(xdr, &val) < 0) + return false; +{% if validate and enumerators %} + /* Compiler may optimize to a range check for dense enums */ + switch (val) { +{% for e in enumerators %} + case {{ e.name }}: +{% endfor %} + break; + default: + return false; + } +{% endif %} + *ptr = val; + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum_be.j2 new file mode 100644 index 000000000..82782a510 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum_be.j2 @@ -0,0 +1,34 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* enum {{ name }} (big-endian) */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr) +{ +{% if validate and enumerators %} + __be32 raw; + u32 val; + + if (xdr_stream_decode_be32(xdr, &raw) < 0) + return false; + val = be32_to_cpu(raw); + /* Compiler may optimize to a range check for dense enums */ + switch (val) { +{% for e in enumerators %} + case {{ e.name }}: +{% endfor %} + break; + default: + return false; + } + *ptr = raw; + return true; +{% else %} + return xdr_stream_decode_be32(xdr, ptr) == 0; +{% endif %} +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close.j2 new file mode 100644 index 000000000..446266ad6 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +}; + +typedef enum {{ name }} {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close_be.j2 new file mode 100644 index 000000000..cfeee2287 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close_be.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +}; + +typedef __be32 {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/definition/enumerator.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/enumerator.j2 new file mode 100644 index 000000000..ff0b893b8 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/enumerator.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + {{ name }} = {{ value }}, diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/definition/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/open.j2 new file mode 100644 index 000000000..b25335221 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/open.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +enum {{ name }} { diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2 new file mode 100644 index 000000000..67245b9a9 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2 @@ -0,0 +1,14 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* enum {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, {{ name }} value) +{ + return xdr_stream_encode_u32(xdr, value) == XDR_UNIT; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum_be.j2 new file mode 100644 index 000000000..fbbcc4594 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum_be.j2 @@ -0,0 +1,14 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* enum {{ name }} (big-endian) */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, {{ name }} value) +{ + return xdr_stream_encode_be32(xdr, value) == XDR_UNIT; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/maxsize/enum.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/maxsize/enum.j2 new file mode 100644 index 000000000..45c1d4c21 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/enum/maxsize/enum.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/header_bottom/declaration/header.j2 b/tools/net/sunrpc/xdrgen/templates/C/header_bottom/declaration/header.j2 new file mode 100644 index 000000000..0bb8c6fc0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/header_bottom/declaration/header.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +#endif /* _LINUX_XDRGEN_{{ infix }}_DECL_H */ diff --git a/tools/net/sunrpc/xdrgen/templates/C/header_bottom/definition/header.j2 b/tools/net/sunrpc/xdrgen/templates/C/header_bottom/definition/header.j2 new file mode 100644 index 000000000..69069d08d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/header_bottom/definition/header.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +#endif /* _LINUX_XDRGEN_{{ infix }}_DEF_H */ diff --git a/tools/net/sunrpc/xdrgen/templates/C/header_top/declaration/header.j2 b/tools/net/sunrpc/xdrgen/templates/C/header_top/declaration/header.j2 new file mode 100644 index 000000000..ebb4e1d32 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/header_top/declaration/header.j2 @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Generated by xdrgen. Manual edits will be lost. */ +/* XDR specification file: {{ filename }} */ +/* XDR specification modification time: {{ mtime }} */ + +#ifndef _LINUX_XDRGEN_{{ infix }}_DECL_H +#define _LINUX_XDRGEN_{{ infix }}_DECL_H + +#include <linux/types.h> + +#include <linux/sunrpc/xdr.h> +#include <linux/sunrpc/xdrgen/_defs.h> +#include <linux/sunrpc/xdrgen/_builtins.h> +#include <linux/sunrpc/xdrgen/{{ infix.lower() }}.h> diff --git a/tools/net/sunrpc/xdrgen/templates/C/header_top/definition/header.j2 b/tools/net/sunrpc/xdrgen/templates/C/header_top/definition/header.j2 new file mode 100644 index 000000000..92f1fd4ba --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/header_top/definition/header.j2 @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Generated by xdrgen. Manual edits will be lost. */ +/* XDR specification file: {{ filename }} */ +/* XDR specification modification time: {{ mtime }} */ + +#ifndef _LINUX_XDRGEN_{{ infix }}_DEF_H +#define _LINUX_XDRGEN_{{ infix }}_DEF_H + +#include <linux/types.h> +#include <linux/sunrpc/xdrgen/_defs.h> diff --git a/tools/net/sunrpc/xdrgen/templates/C/passthru/definition.j2 b/tools/net/sunrpc/xdrgen/templates/C/passthru/definition.j2 new file mode 100644 index 000000000..900c7516a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/passthru/definition.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{{ content }} diff --git a/tools/net/sunrpc/xdrgen/templates/C/passthru/source.j2 b/tools/net/sunrpc/xdrgen/templates/C/passthru/source.j2 new file mode 100644 index 000000000..900c7516a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/passthru/source.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{{ content }} diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/declaration/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/declaration/close.j2 new file mode 100644 index 000000000..816291184 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/declaration/close.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, struct {{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const struct {{ name }} *value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/basic.j2 new file mode 100644 index 000000000..cde4ab53f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/basic.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (basic) */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/close.j2 new file mode 100644 index 000000000..3dbd724d7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/close.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/fixed_length_array.j2 new file mode 100644 index 000000000..cfd64217a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/fixed_length_array.j2 @@ -0,0 +1,8 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length array) */ +{% endif %} + for (u32 i = 0; i < {{ size }}; i++) { + if (xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }}.items[i]) < 0) + return false; + } diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/fixed_length_opaque.j2 new file mode 100644 index 000000000..b4695ece1 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/fixed_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length opaque) */ +{% endif %} + if (xdr_stream_decode_opaque_fixed(xdr, ptr->{{ name }}, {{ size }}) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/open.j2 new file mode 100644 index 000000000..c093d9e3c --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/open.j2 @@ -0,0 +1,22 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* pointer {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, struct {{ name }} *ptr) +{ + bool opted; + +{% if annotate %} + /* opted */ +{% endif %} + if (!xdrgen_decode_bool(xdr, &opted)) + return false; + if (!opted) + return true; + diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/optional_data.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/optional_data.j2 new file mode 100644 index 000000000..b6834299a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/optional_data.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (optional data) */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, ptr->{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/string.j2 new file mode 100644 index 000000000..12d20b143 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/string.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length string) */ +{% endif %} + if (!xdrgen_decode_string(xdr, (string *)ptr, {{ maxsize }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/variable_length_array.j2 new file mode 100644 index 000000000..2f943909c --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/variable_length_array.j2 @@ -0,0 +1,13 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length array) */ +{% endif %} + if (xdr_stream_decode_u32(xdr, &ptr->{{ name }}.count) < 0) + return false; +{% if maxsize != "0" %} + if (ptr->{{ name }}.count > {{ maxsize }}) + return false; +{% endif %} + for (u32 i = 0; i < ptr->{{ name }}.count; i++) + if (!xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }}.element[i])) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/variable_length_opaque.j2 new file mode 100644 index 000000000..9a814de54 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/decoder/variable_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length opaque) */ +{% endif %} + if (!xdrgen_decode_opaque(xdr, (opaque *)ptr, {{ maxsize }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/basic.j2 new file mode 100644 index 000000000..b3430895f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/basic.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (basic) */ +{% endif %} + {{ classifier }}{{ type }} {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/close.j2 new file mode 100644 index 000000000..9e62344a9 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/close.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/fixed_length_array.j2 new file mode 100644 index 000000000..66be83682 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/fixed_length_array.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (fixed-length array) */ +{% endif %} + {{ type }} {{ name }}[{{ size }}]; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/fixed_length_opaque.j2 new file mode 100644 index 000000000..0daba19aa --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/fixed_length_opaque.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (fixed-length opaque) */ +{% endif %} + u8 {{ name }}[{{ size }}]; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/open.j2 new file mode 100644 index 000000000..bc886b818 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/open.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* pointer {{ name }} */ +{% endif %} +struct {{ name }} { diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/optional_data.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/optional_data.j2 new file mode 100644 index 000000000..a33341f45 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/optional_data.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (optional data) */ +{% endif %} + {{ classifier }}{{ type }} *{{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/string.j2 new file mode 100644 index 000000000..2de2feec7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/string.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (variable-length string) */ +{% endif %} + string {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/variable_length_array.j2 new file mode 100644 index 000000000..5d767f9b3 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/variable_length_array.j2 @@ -0,0 +1,8 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (variable-length array) */ +{% endif %} + struct { + u32 count; + {{ classifier }}{{ type }} *element; + } {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/variable_length_opaque.j2 new file mode 100644 index 000000000..4d0cd84be --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/definition/variable_length_opaque.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (variable-length opaque) */ +{% endif %} + opaque {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/basic.j2 new file mode 100644 index 000000000..a7d3695c5 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/basic.j2 @@ -0,0 +1,10 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (basic) */ +{% endif %} +{% if type in pass_by_reference %} + if (!xdrgen_encode_{{ type }}(xdr, &value->{{ name }})) +{% else %} + if (!xdrgen_encode_{{ type }}(xdr, value->{{ name }})) +{% endif %} + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/close.j2 new file mode 100644 index 000000000..3dbd724d7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/close.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/fixed_length_array.j2 new file mode 100644 index 000000000..b01833a2c --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/fixed_length_array.j2 @@ -0,0 +1,12 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length array) */ +{% endif %} + for (u32 i = 0; i < {{ size }}; i++) { +{% if type in pass_by_reference %} + if (xdrgen_encode_{{ type }}(xdr, &value->items[i]) < 0) +{% else %} + if (xdrgen_encode_{{ type }}(xdr, value->items[i]) < 0) +{% endif %} + return false; + } diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/fixed_length_opaque.j2 new file mode 100644 index 000000000..07bc91919 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/fixed_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length opaque) */ +{% endif %} + if (xdr_stream_encode_opaque_fixed(xdr, value->{{ name }}, {{ size }}) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/open.j2 new file mode 100644 index 000000000..d67fae200 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/open.j2 @@ -0,0 +1,20 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* pointer {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const struct {{ name }} *value) +{ +{% if annotate %} + /* opted */ +{% endif %} + if (!xdrgen_encode_bool(xdr, value != NULL)) + return false; + if (!value) + return true; + diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/optional_data.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/optional_data.j2 new file mode 100644 index 000000000..16fb3e09b --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/optional_data.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (optional data) */ +{% endif %} + if (!xdrgen_encode_{{ type }}(xdr, value->{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/string.j2 new file mode 100644 index 000000000..7ddc2bf3e --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/string.j2 @@ -0,0 +1,10 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length string) */ +{% endif %} +{% if maxsize != "0" %} + if (value->{{ name }}.len > {{ maxsize }}) + return false; +{% endif %} + if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_array.j2 new file mode 100644 index 000000000..b21476629 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_array.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length array) */ +{% endif %} +{% if maxsize != "0" %} + if (value->{{ name }}.count > {{ maxsize }}) + return false; +{% endif %} + if (xdr_stream_encode_u32(xdr, value->{{ name }}.count) != XDR_UNIT) + return false; + for (u32 i = 0; i < value->{{ name }}.count; i++) +{% if type in pass_by_reference %} + if (!xdrgen_encode_{{ type }}(xdr, &value->{{ name }}.element[i])) +{% else %} + if (!xdrgen_encode_{{ type }}(xdr, value->{{ name }}.element[i])) +{% endif %} + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_opaque.j2 new file mode 100644 index 000000000..5bf00070a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_opaque.j2 @@ -0,0 +1,10 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length opaque) */ +{% endif %} +{% if maxsize != "0" %} + if (value->{{ name }}.len > {{ maxsize }}) + return false; +{% endif %} + if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/maxsize/pointer.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/maxsize/pointer.j2 new file mode 100644 index 000000000..9f3bfb47d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/maxsize/pointer.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} \ + ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/declaration/argument.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/declaration/argument.j2 new file mode 100644 index 000000000..4364fed19 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/declaration/argument.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +bool {{ program }}_svc_decode_{{ argument }}(struct svc_rqst *rqstp, struct xdr_stream *xdr); diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/declaration/result.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/declaration/result.j2 new file mode 100644 index 000000000..e0ea1e849 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/declaration/result.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +bool {{ program }}_svc_encode_{{ result }}(struct svc_rqst *rqstp, struct xdr_stream *xdr); diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/decoder/argument.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/decoder/argument.j2 new file mode 100644 index 000000000..096d553b2 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/decoder/argument.j2 @@ -0,0 +1,21 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +/** + * {{ program }}_svc_decode_{{ argument }} - Decode a {{ argument }} argument + * @rqstp: RPC transaction context + * @xdr: source XDR data stream + * + * Return values: + * %true: procedure arguments decoded successfully + * %false: decode failed + */ +bool {{ program }}_svc_decode_{{ argument }}(struct svc_rqst *rqstp, struct xdr_stream *xdr) +{ +{% if argument in structs %} + struct {{ argument }} *argp = rqstp->rq_argp; +{% else %} + {{ argument }} *argp = rqstp->rq_argp; +{% endif %} + + return xdrgen_decode_{{ argument }}(xdr, argp); +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/decoder/result.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/decoder/result.j2 new file mode 100644 index 000000000..aa9940e32 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/decoder/result.j2 @@ -0,0 +1,18 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* Decode {{ result }} results */ +{% endif %} +static int {{ program }}_xdr_dec_{{ result }}(struct rpc_rqst *req, + struct xdr_stream *xdr, void *data) +{ +{% if result == 'void' %} + xdrgen_decode_void(xdr); +{% else %} + struct {{ result }} *result = data; + + if (!xdrgen_decode_{{ result }}(xdr, result)) + return -EIO; +{% endif %} + return 0; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/definition/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/definition/close.j2 new file mode 100644 index 000000000..9e62344a9 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/definition/close.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/definition/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/definition/open.j2 new file mode 100644 index 000000000..f9a6d439f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/definition/open.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* procedure numbers for {{ program }} */ +{% endif %} +enum { diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/definition/procedure.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/definition/procedure.j2 new file mode 100644 index 000000000..ff0b893b8 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/definition/procedure.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + {{ name }} = {{ value }}, diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/definition/program.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/definition/program.j2 new file mode 100644 index 000000000..320663ffc --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/definition/program.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +#ifndef {{ name }} +#define {{ name }} ({{ value }}) +#endif diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/encoder/argument.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/encoder/argument.j2 new file mode 100644 index 000000000..2fbb5bd13 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/encoder/argument.j2 @@ -0,0 +1,16 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{%if annotate %} +/* Encode {{ argument }} arguments */ +{% endif %} +static void {{ program }}_xdr_enc_{{ argument }}(struct rpc_rqst *req, + struct xdr_stream *xdr, const void *data) +{ +{% if argument == 'void' %} + xdrgen_encode_void(xdr); +{% else %} + const struct {{ argument }} *args = data; + + xdrgen_encode_{{ argument }}(xdr, args); +{% endif %} +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/encoder/result.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/encoder/result.j2 new file mode 100644 index 000000000..4243d9196 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/encoder/result.j2 @@ -0,0 +1,23 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +/** + * {{ program }}_svc_encode_{{ result }} - Encode a {{ result }} result + * @rqstp: RPC transaction context + * @xdr: target XDR data stream + * + * Return values: + * %true: procedure results encoded successfully + * %false: encode failed + */ +bool {{ program }}_svc_encode_{{ result }}(struct svc_rqst *rqstp, struct xdr_stream *xdr) +{ +{% if result in structs %} + struct {{ result }} *resp = rqstp->rq_resp; + + return xdrgen_encode_{{ result }}(xdr, resp); +{% else %} + {{ result }} *resp = rqstp->rq_resp; + + return xdrgen_encode_{{ result }}(xdr, *resp); +{% endif %} +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/program/maxsize/max_args.j2 b/tools/net/sunrpc/xdrgen/templates/C/program/maxsize/max_args.j2 new file mode 100644 index 000000000..9f3bfb47d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/program/maxsize/max_args.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} \ + ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/source_top/client.j2 b/tools/net/sunrpc/xdrgen/templates/C/source_top/client.j2 new file mode 100644 index 000000000..df3598c38 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/source_top/client.j2 @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +// Generated by xdrgen. Manual edits will be lost. +// XDR specification file: {{ filename }} +// XDR specification modification time: {{ mtime }} + +#include <linux/types.h> + +#include <linux/sunrpc/xdr.h> +#include <linux/sunrpc/xdrgen/_defs.h> +#include <linux/sunrpc/xdrgen/_builtins.h> + +#include <linux/sunrpc/clnt.h> diff --git a/tools/net/sunrpc/xdrgen/templates/C/source_top/server.j2 b/tools/net/sunrpc/xdrgen/templates/C/source_top/server.j2 new file mode 100644 index 000000000..974e1d971 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/source_top/server.j2 @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 +// Generated by xdrgen. Manual edits will be lost. +// XDR specification file: {{ filename }} +// XDR specification modification time: {{ mtime }} + +#include <linux/sunrpc/svc.h> + +#include "{{ program }}xdr_gen.h" diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/declaration/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/declaration/close.j2 new file mode 100644 index 000000000..816291184 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/declaration/close.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, struct {{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const struct {{ name }} *value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/basic.j2 new file mode 100644 index 000000000..cde4ab53f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/basic.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (basic) */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/close.j2 new file mode 100644 index 000000000..3dbd724d7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/close.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_array.j2 new file mode 100644 index 000000000..cfd64217a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_array.j2 @@ -0,0 +1,8 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length array) */ +{% endif %} + for (u32 i = 0; i < {{ size }}; i++) { + if (xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }}.items[i]) < 0) + return false; + } diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_opaque.j2 new file mode 100644 index 000000000..b4695ece1 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/fixed_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length opaque) */ +{% endif %} + if (xdr_stream_decode_opaque_fixed(xdr, ptr->{{ name }}, {{ size }}) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/open.j2 new file mode 100644 index 000000000..289e67259 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/open.j2 @@ -0,0 +1,12 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* struct {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, struct {{ name }} *ptr) +{ diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/optional_data.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/optional_data.j2 new file mode 100644 index 000000000..b6834299a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/optional_data.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (optional data) */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, ptr->{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/string.j2 new file mode 100644 index 000000000..12d20b143 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/string.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length string) */ +{% endif %} + if (!xdrgen_decode_string(xdr, (string *)ptr, {{ maxsize }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_array.j2 new file mode 100644 index 000000000..2f943909c --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_array.j2 @@ -0,0 +1,13 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length array) */ +{% endif %} + if (xdr_stream_decode_u32(xdr, &ptr->{{ name }}.count) < 0) + return false; +{% if maxsize != "0" %} + if (ptr->{{ name }}.count > {{ maxsize }}) + return false; +{% endif %} + for (u32 i = 0; i < ptr->{{ name }}.count; i++) + if (!xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }}.element[i])) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_opaque.j2 new file mode 100644 index 000000000..65698e20d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/decoder/variable_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length opaque) */ +{% endif %} + if (!xdrgen_decode_opaque(xdr, &ptr->{{ name }}, {{ maxsize }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/basic.j2 new file mode 100644 index 000000000..b3430895f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/basic.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (basic) */ +{% endif %} + {{ classifier }}{{ type }} {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/close.j2 new file mode 100644 index 000000000..9e62344a9 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/close.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/fixed_length_array.j2 new file mode 100644 index 000000000..66be83682 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/fixed_length_array.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (fixed-length array) */ +{% endif %} + {{ type }} {{ name }}[{{ size }}]; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/fixed_length_opaque.j2 new file mode 100644 index 000000000..0daba19aa --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/fixed_length_opaque.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (fixed-length opaque) */ +{% endif %} + u8 {{ name }}[{{ size }}]; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/open.j2 new file mode 100644 index 000000000..07cbf5424 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/open.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* struct {{ name }} */ +{% endif %} +struct {{ name }} { diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/optional_data.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/optional_data.j2 new file mode 100644 index 000000000..a33341f45 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/optional_data.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (optional data) */ +{% endif %} + {{ classifier }}{{ type }} *{{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/string.j2 new file mode 100644 index 000000000..2de2feec7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/string.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (variable-length string) */ +{% endif %} + string {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/variable_length_array.j2 new file mode 100644 index 000000000..5d767f9b3 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/variable_length_array.j2 @@ -0,0 +1,8 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (variable-length array) */ +{% endif %} + struct { + u32 count; + {{ classifier }}{{ type }} *element; + } {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/definition/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/variable_length_opaque.j2 new file mode 100644 index 000000000..4d0cd84be --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/definition/variable_length_opaque.j2 @@ -0,0 +1,5 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* (variable-length opaque) */ +{% endif %} + opaque {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/basic.j2 new file mode 100644 index 000000000..a7d3695c5 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/basic.j2 @@ -0,0 +1,10 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (basic) */ +{% endif %} +{% if type in pass_by_reference %} + if (!xdrgen_encode_{{ type }}(xdr, &value->{{ name }})) +{% else %} + if (!xdrgen_encode_{{ type }}(xdr, value->{{ name }})) +{% endif %} + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/close.j2 new file mode 100644 index 000000000..3dbd724d7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/close.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_array.j2 new file mode 100644 index 000000000..b01833a2c --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_array.j2 @@ -0,0 +1,12 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length array) */ +{% endif %} + for (u32 i = 0; i < {{ size }}; i++) { +{% if type in pass_by_reference %} + if (xdrgen_encode_{{ type }}(xdr, &value->items[i]) < 0) +{% else %} + if (xdrgen_encode_{{ type }}(xdr, value->items[i]) < 0) +{% endif %} + return false; + } diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_opaque.j2 new file mode 100644 index 000000000..07bc91919 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/fixed_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (fixed-length opaque) */ +{% endif %} + if (xdr_stream_encode_opaque_fixed(xdr, value->{{ name }}, {{ size }}) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/open.j2 new file mode 100644 index 000000000..2286a3adf --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/open.j2 @@ -0,0 +1,12 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* struct {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const struct {{ name }} *value) +{ diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/optional_data.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/optional_data.j2 new file mode 100644 index 000000000..16fb3e09b --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/optional_data.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (optional data) */ +{% endif %} + if (!xdrgen_encode_{{ type }}(xdr, value->{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/string.j2 new file mode 100644 index 000000000..7ddc2bf3e --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/string.j2 @@ -0,0 +1,10 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length string) */ +{% endif %} +{% if maxsize != "0" %} + if (value->{{ name }}.len > {{ maxsize }}) + return false; +{% endif %} + if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_array.j2 new file mode 100644 index 000000000..b21476629 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_array.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length array) */ +{% endif %} +{% if maxsize != "0" %} + if (value->{{ name }}.count > {{ maxsize }}) + return false; +{% endif %} + if (xdr_stream_encode_u32(xdr, value->{{ name }}.count) != XDR_UNIT) + return false; + for (u32 i = 0; i < value->{{ name }}.count; i++) +{% if type in pass_by_reference %} + if (!xdrgen_encode_{{ type }}(xdr, &value->{{ name }}.element[i])) +{% else %} + if (!xdrgen_encode_{{ type }}(xdr, value->{{ name }}.element[i])) +{% endif %} + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2 new file mode 100644 index 000000000..5bf00070a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2 @@ -0,0 +1,10 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length opaque) */ +{% endif %} +{% if maxsize != "0" %} + if (value->{{ name }}.len > {{ maxsize }}) + return false; +{% endif %} + if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/maxsize/struct.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/maxsize/struct.j2 new file mode 100644 index 000000000..9f3bfb47d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/struct/maxsize/struct.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} \ + ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/basic.j2 new file mode 100644 index 000000000..455b10bd9 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/basic.j2 @@ -0,0 +1,8 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr); +{% if name in pass_by_reference %} +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ name }} *value); +{%- else -%} +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ name }} value); +{% endif %} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_array.j2 new file mode 100644 index 000000000..3fe3ddd9f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_array.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_opaque.j2 new file mode 100644 index 000000000..3fe3ddd9f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/fixed_length_opaque.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/string.j2 new file mode 100644 index 000000000..3fe3ddd9f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/string.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_array.j2 new file mode 100644 index 000000000..3fe3ddd9f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_array.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_opaque.j2 new file mode 100644 index 000000000..3fe3ddd9f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/declaration/variable_length_opaque.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/basic.j2 new file mode 100644 index 000000000..b215e157d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/basic.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr) +{ +{% if annotate %} + /* (basic) */ +{% endif %} + return xdrgen_decode_{{ type }}(xdr, ptr); +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_array.j2 new file mode 100644 index 000000000..c8953719e --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_array.j2 @@ -0,0 +1,25 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr) +{ +{% if annotate %} + /* (fixed-length array) */ +{% endif %} + for (u32 i = 0; i < {{ size }}; i++) { +{%- if classifier == '' %} + if (xdrgen_decode_{{ type }}(xdr, ptr->items[i]) < 0) +{% else %} + if (xdrgen_decode_{{ type }}(xdr, &ptr->items[i]) < 0) +{% endif %} + return false; + } + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_opaque.j2 new file mode 100644 index 000000000..c854fc8c7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/fixed_length_opaque.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr) +{ +{% if annotate %} + /* (fixed-length opaque) */ +{% endif %} + return xdr_stream_decode_opaque_fixed(xdr, ptr, {{ size }}) == 0; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/string.j2 new file mode 100644 index 000000000..bcbc1758a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/string.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr) +{ +{% if annotate %} + /* (variable-length string) */ +{% endif %} + return xdrgen_decode_string(xdr, ptr, {{ maxsize }}); +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_array.j2 new file mode 100644 index 000000000..a59cc1f38 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_array.j2 @@ -0,0 +1,26 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr) +{ +{% if annotate %} + /* (variable-length array) */ +{% endif %} + if (xdr_stream_decode_u32(xdr, &ptr->count) < 0) + return false; +{% if maxsize != "0" %} + if (ptr->count > {{ maxsize }}) + return false; +{% endif %} + for (u32 i = 0; i < ptr->count; i++) + if (!xdrgen_decode_{{ type }}(xdr, &ptr->element[i])) + return false; + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_opaque.j2 new file mode 100644 index 000000000..eb05f53e1 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/decoder/variable_length_opaque.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ classifier }}{{ name }} *ptr) +{ +{% if annotate %} + /* (variable-length opaque) */ +{% endif %} + return xdrgen_decode_opaque(xdr, ptr, {{ maxsize }}); +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/basic.j2 new file mode 100644 index 000000000..1c5f28135 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/basic.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} (basic) */ +{% endif %} +typedef {{ classifier }}{{ type }} {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/fixed_length_array.j2 new file mode 100644 index 000000000..c3a67c952 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/fixed_length_array.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} (fixed-length array) */ +{% endif %} +typedef {{ type }}{{ name }}[{{ size }}]; diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/fixed_length_opaque.j2 new file mode 100644 index 000000000..8788b02fe --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/fixed_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} (fixed-length opaque) */ +{% endif %} +typedef u8 {{ name }}[{{ size }}]; diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/string.j2 new file mode 100644 index 000000000..c03c2df8e --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/string.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} (variable-length string) */ +{% endif %} +typedef string {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/variable_length_array.j2 new file mode 100644 index 000000000..f03393760 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/variable_length_array.j2 @@ -0,0 +1,9 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} (variable-length array) */ +{% endif %} +typedef struct { + u32 count; + {{ classifier }}{{ type }} *element; +} {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/variable_length_opaque.j2 new file mode 100644 index 000000000..162f2610a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/definition/variable_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} (variable-length opaque) */ +{% endif %} +typedef opaque {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/basic.j2 new file mode 100644 index 000000000..0d21dd0b7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/basic.j2 @@ -0,0 +1,21 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +{% if name in pass_by_reference %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} *value) +{% else %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value) +{% endif %} +{ +{% if annotate %} + /* (basic) */ +{% endif %} + return xdrgen_encode_{{ type }}(xdr, value); +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_array.j2 new file mode 100644 index 000000000..ec8cd6509 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_array.j2 @@ -0,0 +1,25 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value) +{ +{% if annotate %} + /* (fixed-length array) */ +{% endif %} + for (u32 i = 0; i < {{ size }}; i++) { +{% if type in pass_by_reference %} + if (xdrgen_encode_{{ type }}(xdr, &value->items[i]) < 0) +{% else %} + if (xdrgen_encode_{{ type }}(xdr, value->items[i]) < 0) +{% endif %} + return false; + } + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_opaque.j2 new file mode 100644 index 000000000..b53fa87e1 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/fixed_length_opaque.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value) +{ +{% if annotate %} + /* (fixed-length opaque) */ +{% endif %} + return xdr_stream_encode_opaque_fixed(xdr, value, {{ size }}) >= 0; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/string.j2 new file mode 100644 index 000000000..28b81f1d0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/string.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value) +{ +{% if annotate %} + /* (variable-length string) */ +{% endif %} + return xdr_stream_encode_opaque(xdr, value.data, value.len) >= 0; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_array.j2 new file mode 100644 index 000000000..ff093c281 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_array.j2 @@ -0,0 +1,30 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value) +{ +{% if annotate %} + /* (variable-length array) */ +{% endif %} +{% if maxsize != "0" %} + if (unlikely(value.count > {{ maxsize }})) + return false; +{% endif %} + if (xdr_stream_encode_u32(xdr, value.count) != XDR_UNIT) + return false; + for (u32 i = 0; i < value.count; i++) +{% if type in pass_by_reference %} + if (!xdrgen_encode_{{ type }}(xdr, &value.element[i])) +{% else %} + if (!xdrgen_encode_{{ type }}(xdr, value.element[i])) +{% endif %} + return false; + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_opaque.j2 new file mode 100644 index 000000000..2e89592fa --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/encoder/variable_length_opaque.j2 @@ -0,0 +1,17 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* typedef {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const {{ classifier }}{{ name }} value) +{ +{% if annotate %} + /* (variable-length opaque) */ +{% endif %} + return xdr_stream_encode_opaque(xdr, value.data, value.len) >= 0; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/basic.j2 new file mode 100644 index 000000000..9f3bfb47d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/basic.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} \ + ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/fixed_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/fixed_length_opaque.j2 new file mode 100644 index 000000000..45c1d4c21 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/fixed_length_opaque.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/string.j2 new file mode 100644 index 000000000..45c1d4c21 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/string.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/variable_length_array.j2 new file mode 100644 index 000000000..45c1d4c21 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/variable_length_array.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/variable_length_opaque.j2 new file mode 100644 index 000000000..45c1d4c21 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/typedef/maxsize/variable_length_opaque.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/declaration/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/declaration/close.j2 new file mode 100644 index 000000000..816291184 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/declaration/close.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, struct {{ name }} *ptr); +bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const struct {{ name }} *value); diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/basic.j2 new file mode 100644 index 000000000..4d97cc539 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/basic.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (basic) */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, &ptr->u.{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/bool_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/bool_spec.j2 new file mode 100644 index 000000000..05ad491f7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/bool_spec.j2 @@ -0,0 +1,7 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* discriminant {{ name }} */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }})) + return false; + if (ptr->{{ name }}) { diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/break.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/break.j2 new file mode 100644 index 000000000..b286d1407 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/break.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + break; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec.j2 new file mode 100644 index 000000000..5fa2163f0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + case {{ case }}: diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec_be.j2 new file mode 100644 index 000000000..917f3a1c4 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec_be.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + case __constant_cpu_to_be32({{ case }}): diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/close.j2 new file mode 100644 index 000000000..39d8d6c50 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/close.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + } + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/default_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/default_spec.j2 new file mode 100644 index 000000000..044a002d0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/default_spec.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + default: diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/open.j2 new file mode 100644 index 000000000..eb9941376 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/open.j2 @@ -0,0 +1,12 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* union {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_decode_{{ name }}(struct xdr_stream *xdr, struct {{ name }} *ptr) +{ diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/optional_data.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/optional_data.j2 new file mode 100644 index 000000000..e4476f5fd --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/optional_data.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (optional data) */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, &ptr->u.{{ name }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/string.j2 new file mode 100644 index 000000000..83b6e5a14 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/string.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length string) */ +{% endif %} + if (!xdrgen_decode_string(xdr, (struct string *)ptr->u.{{ name }}, {{ maxsize }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/switch_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/switch_spec.j2 new file mode 100644 index 000000000..99b3067ef --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/switch_spec.j2 @@ -0,0 +1,7 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* discriminant {{ name }} */ +{% endif %} + if (!xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }})) + return false; + switch (ptr->{{ name }}) { diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_array.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_array.j2 new file mode 100644 index 000000000..53dfaf9ce --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_array.j2 @@ -0,0 +1,15 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length array) */ +{% endif %} + if (xdr_stream_decode_u32(xdr, &count) < 0) + return false; +{% if maxsize != "0" %} + if (count > {{ maxsize }}) + return false; +{% endif %} + for (u32 i = 0; i < count; i++) { + if (xdrgen_decode_{{ type }}(xdr, &ptr->{{ name }}.items[i]) < 0) + return false; + } + ptr->{{ name }}.len = count; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_opaque.j2 new file mode 100644 index 000000000..c9d88ed29 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/variable_length_opaque.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length opaque) */ +{% endif %} + if (!xdrgen_decode_opaque(xdr, (struct opaque *)ptr->u.{{ name }}, {{ maxsize }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/void.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/void.j2 new file mode 100644 index 000000000..65205ce37 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/void.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + if (!xdrgen_decode_void(xdr)) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/definition/case_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/definition/case_spec.j2 new file mode 100644 index 000000000..52f8d131b --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/definition/case_spec.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + {{ classifier }}{{ type }} {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/definition/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/definition/close.j2 new file mode 100644 index 000000000..19ee759d7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/definition/close.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + } u; +}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/definition/default_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/definition/default_spec.j2 new file mode 100644 index 000000000..52f8d131b --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/definition/default_spec.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + {{ classifier }}{{ type }} {{ name }}; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/definition/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/definition/open.j2 new file mode 100644 index 000000000..20fcfd1fc --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/definition/open.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* union {{ name }} */ +{% endif %} +struct {{ name }} { diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/definition/switch_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/definition/switch_spec.j2 new file mode 100644 index 000000000..3e5527325 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/definition/switch_spec.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + {{ classifier }}{{ type }} {{ name }}; + union { diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/basic.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/basic.j2 new file mode 100644 index 000000000..6452d75c6 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/basic.j2 @@ -0,0 +1,10 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (basic) */ +{% endif %} +{% if type in pass_by_reference %} + if (!xdrgen_encode_{{ type }}(xdr, &ptr->u.{{ name }})) +{% else %} + if (!xdrgen_encode_{{ type }}(xdr, ptr->u.{{ name }})) +{% endif %} + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/bool_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/bool_spec.j2 new file mode 100644 index 000000000..e5135ed64 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/bool_spec.j2 @@ -0,0 +1,7 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* discriminant {{ name }} */ +{% endif %} + if (!xdrgen_encode_{{ type }}(xdr, ptr->{{ name }})) + return false; + if (ptr->{{ name }}) { diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/break.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/break.j2 new file mode 100644 index 000000000..b286d1407 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/break.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + break; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec.j2 new file mode 100644 index 000000000..5fa2163f0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + case {{ case }}: diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec_be.j2 new file mode 100644 index 000000000..917f3a1c4 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec_be.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + case __constant_cpu_to_be32({{ case }}): diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/close.j2 new file mode 100644 index 000000000..39d8d6c50 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/close.j2 @@ -0,0 +1,4 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + } + return true; +} diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/default_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/default_spec.j2 new file mode 100644 index 000000000..044a002d0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/default_spec.j2 @@ -0,0 +1,2 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + default: diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/open.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/open.j2 new file mode 100644 index 000000000..e5a206df1 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/open.j2 @@ -0,0 +1,12 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + +{% if annotate %} +/* union {{ name }} */ +{% endif %} +{% if name in public_apis %} +bool +{% else %} +static bool __maybe_unused +{% endif %} +xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const struct {{ name }} *ptr) +{ diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/string.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/string.j2 new file mode 100644 index 000000000..2f035a64f --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/string.j2 @@ -0,0 +1,6 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* member {{ name }} (variable-length string) */ +{% endif %} + if (!xdrgen_encode_string(xdr, ptr->u.{{ name }}, {{ maxsize }})) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/switch_spec.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/switch_spec.j2 new file mode 100644 index 000000000..c8c3ecbe0 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/switch_spec.j2 @@ -0,0 +1,7 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +{% if annotate %} + /* discriminant {{ name }} */ +{% endif %} + if (!xdrgen_encode_{{ type }}(xdr, ptr->{{ name }})) + return false; + switch (ptr->{{ name }}) { diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/void.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/void.j2 new file mode 100644 index 000000000..84e7c2127 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/void.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} + if (!xdrgen_encode_void(xdr)) + return false; diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/maxsize/union.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/maxsize/union.j2 new file mode 100644 index 000000000..9f3bfb47d --- /dev/null +++ b/tools/net/sunrpc/xdrgen/templates/C/union/maxsize/union.j2 @@ -0,0 +1,3 @@ +{# SPDX-License-Identifier: GPL-2.0 #} +#define {{ '{:<31}'.format(macro) }} \ + ({{ width }}) diff --git a/tools/net/sunrpc/xdrgen/tests/bad-procedure-number-negative.x b/tools/net/sunrpc/xdrgen/tests/bad-procedure-number-negative.x new file mode 100644 index 000000000..33ed272c3 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/tests/bad-procedure-number-negative.x @@ -0,0 +1,20 @@ +/* + * NEGATIVE TEST CASE -- xdrgen must REJECT this specification. + * + * RFC 5531 assigns only unsigned constants to program, version, and + * procedure numbers (Section 12.3). This spec gives a procedure a + * negative number, which the front end must reject. + * + * Expected diagnostic: + * negative procedure number -5 in version 'BADVERS' + * + * The tests directory has no automated runner; exercise by hand: + * ./xdrgen definitions tests/bad-procedure-number-negative.x (must fail) + */ + +program BADPROG { + version BADVERS { + void BADPROC_NULL(void) = 0; + void BADPROC_FOO(void) = -5; + } = 1; +} = 100000; diff --git a/tools/net/sunrpc/xdrgen/tests/bad-procedure-number-too-large.x b/tools/net/sunrpc/xdrgen/tests/bad-procedure-number-too-large.x new file mode 100644 index 000000000..521581c57 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/tests/bad-procedure-number-too-large.x @@ -0,0 +1,20 @@ +/* + * NEGATIVE TEST CASE -- xdrgen must REJECT this specification. + * + * RFC 5531 encodes program, version, and procedure numbers as unsigned + * 32-bit integers (Section 9). This spec gives a procedure a number one + * past the 32-bit maximum, which the front end must reject. + * + * Expected diagnostic: + * procedure number 4294967296 in version 'BADVERS' exceeds 4294967295 + * + * The tests directory has no automated runner; exercise by hand: + * ./xdrgen definitions tests/bad-procedure-number-too-large.x (must fail) + */ + +program BADPROG { + version BADVERS { + void BADPROC_NULL(void) = 0; + void BADPROC_FOO(void) = 4294967296; + } = 1; +} = 100000; diff --git a/tools/net/sunrpc/xdrgen/tests/bad-program-number-negative.x b/tools/net/sunrpc/xdrgen/tests/bad-program-number-negative.x new file mode 100644 index 000000000..f7b71ee07 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/tests/bad-program-number-negative.x @@ -0,0 +1,19 @@ +/* + * NEGATIVE TEST CASE -- xdrgen must REJECT this specification. + * + * RFC 5531 assigns only unsigned constants to program, version, and + * procedure numbers (Section 12.3). This spec gives the program a + * negative number, which the front end must reject. + * + * Expected diagnostic: + * negative program number -100000 in program 'BADPROG' + * + * The tests directory has no automated runner; exercise by hand: + * ./xdrgen definitions tests/bad-program-number-negative.x (must fail) + */ + +program BADPROG { + version BADVERS { + void BADPROC_NULL(void) = 0; + } = 1; +} = -100000; diff --git a/tools/net/sunrpc/xdrgen/tests/bad-program-number-too-large.x b/tools/net/sunrpc/xdrgen/tests/bad-program-number-too-large.x new file mode 100644 index 000000000..c761584e7 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/tests/bad-program-number-too-large.x @@ -0,0 +1,19 @@ +/* + * NEGATIVE TEST CASE -- xdrgen must REJECT this specification. + * + * RFC 5531 encodes program, version, and procedure numbers as unsigned + * 32-bit integers (Section 9). This spec gives the program a number one + * past the 32-bit maximum, which the front end must reject. + * + * Expected diagnostic: + * program number 4294967296 in program 'BADPROG' exceeds 4294967295 + * + * The tests directory has no automated runner; exercise by hand: + * ./xdrgen definitions tests/bad-program-number-too-large.x (must fail) + */ + +program BADPROG { + version BADVERS { + void BADPROC_NULL(void) = 0; + } = 1; +} = 4294967296; diff --git a/tools/net/sunrpc/xdrgen/tests/bad-version-number-negative.x b/tools/net/sunrpc/xdrgen/tests/bad-version-number-negative.x new file mode 100644 index 000000000..dd9c77343 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/tests/bad-version-number-negative.x @@ -0,0 +1,19 @@ +/* + * NEGATIVE TEST CASE -- xdrgen must REJECT this specification. + * + * RFC 5531 assigns only unsigned constants to program, version, and + * procedure numbers (Section 12.3). This spec gives the version a + * negative number, which the front end must reject. + * + * Expected diagnostic: + * negative version number -1 in program 'BADPROG' + * + * The tests directory has no automated runner; exercise by hand: + * ./xdrgen definitions tests/bad-version-number-negative.x (must fail) + */ + +program BADPROG { + version BADVERS { + void BADPROC_NULL(void) = 0; + } = -1; +} = 100000; diff --git a/tools/net/sunrpc/xdrgen/tests/bad-version-number-too-large.x b/tools/net/sunrpc/xdrgen/tests/bad-version-number-too-large.x new file mode 100644 index 000000000..dd44f6eed --- /dev/null +++ b/tools/net/sunrpc/xdrgen/tests/bad-version-number-too-large.x @@ -0,0 +1,19 @@ +/* + * NEGATIVE TEST CASE -- xdrgen must REJECT this specification. + * + * RFC 5531 encodes program, version, and procedure numbers as unsigned + * 32-bit integers (Section 9). This spec gives the version a number one + * past the 32-bit maximum, which the front end must reject. + * + * Expected diagnostic: + * version number 4294967296 in program 'BADPROG' exceeds 4294967295 + * + * The tests directory has no automated runner; exercise by hand: + * ./xdrgen definitions tests/bad-version-number-too-large.x (must fail) + */ + +program BADPROG { + version BADVERS { + void BADPROC_NULL(void) = 0; + } = 4294967296; +} = 100000; diff --git a/tools/net/sunrpc/xdrgen/tests/test.x b/tools/net/sunrpc/xdrgen/tests/test.x new file mode 100644 index 000000000..90c8587f6 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/tests/test.x @@ -0,0 +1,36 @@ +/* Sample XDR specification from RFC 1832 Section 5.5 */ + +const MAXUSERNAME = 32; /* max length of a user name */ +const MAXFILELEN = 65535; /* max length of a file */ +const MAXNAMELEN = 255; /* max length of a file name */ + +/* + * Types of files: + */ +enum filekind { + TEXT = 0, /* ascii data */ + DATA = 1, /* raw data */ + EXEC = 2 /* executable */ +}; + +/* + * File information, per kind of file: + */ +union filetype switch (filekind kind) { +case TEXT: + void; /* no extra information */ +case DATA: + string creator<MAXNAMELEN>; /* data creator */ +case EXEC: + string interpretor<MAXNAMELEN>; /* program interpretor */ +}; + +/* + * A complete file: + */ +struct file { + string filename<MAXNAMELEN>; /* name of file */ + filetype type; /* info about file */ + string owner<MAXUSERNAME>; /* owner of file */ + opaque data<MAXFILELEN>; /* file data */ +}; diff --git a/tools/net/sunrpc/xdrgen/xdr_ast.py b/tools/net/sunrpc/xdrgen/xdr_ast.py new file mode 100644 index 000000000..9dab8bc54 --- /dev/null +++ b/tools/net/sunrpc/xdrgen/xdr_ast.py @@ -0,0 +1,1004 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Define and implement the Abstract Syntax Tree for the XDR language.""" + +import sys +from typing import List +from dataclasses import dataclass, KW_ONLY + +from lark import ast_utils, Transformer +from lark.tree import Meta + +this_module = sys.modules[__name__] + +big_endian = [] +excluded_apis = [] +header_name = "none" +public_apis = [] +structs = set() +pass_by_reference = set() + +constants = {} + + +def xdr_quadlen(val: str) -> int: + """Return integer XDR width of an XDR type""" + if val in constants: + octets = constants[val] + else: + octets = int(val) + return int((octets + 3) / 4) + + +symbolic_widths = { + "void": ["XDR_void"], + "bool": ["XDR_bool"], + "short": ["XDR_short"], + "unsigned_short": ["XDR_unsigned_short"], + "int": ["XDR_int"], + "unsigned_int": ["XDR_unsigned_int"], + "long": ["XDR_long"], + "unsigned_long": ["XDR_unsigned_long"], + "hyper": ["XDR_hyper"], + "unsigned_hyper": ["XDR_unsigned_hyper"], +} + +# Numeric XDR widths are tracked in a dictionary that is keyed +# by type_name because sometimes a caller has nothing more than +# the type_name to use to figure out the numeric width. +max_widths = { + "void": 0, + "bool": 1, + "short": 1, + "unsigned_short": 1, + "int": 1, + "unsigned_int": 1, + "long": 1, + "unsigned_long": 1, + "hyper": 2, + "unsigned_hyper": 2, +} + + +@dataclass +class _XdrAst(ast_utils.Ast): + """Base class for the XDR abstract syntax tree""" + + # Source position of the construct's declared identifier, when + # the transformer records one, so semantic diagnostics can point + # at the exact declaration. The KW_ONLY marker makes the fields + # keyword-only, so they never disturb the positional child + # ordering lark uses to build each node; 0 means the position was + # not recorded. + _: KW_ONLY + line: int = 0 + column: int = 0 + + +@dataclass +class _XdrIdentifier(_XdrAst): + """Corresponds to 'identifier' in the XDR language grammar""" + + symbol: str + + +@dataclass +class _XdrValue(_XdrAst): + """Corresponds to 'value' in the XDR language grammar""" + + value: str + + +@dataclass +class _XdrConstantValue(_XdrAst): + """Corresponds to 'constant' in the XDR language grammar""" + + value: int + + +@dataclass +class _XdrTypeSpecifier(_XdrAst): + """Corresponds to 'type_specifier' in the XDR language grammar""" + + type_name: str + c_classifier: str = "" + + +@dataclass +class _XdrDefinedType(_XdrTypeSpecifier): + """Corresponds to a type defined by the input specification""" + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return [get_header_name().upper() + "_" + self.type_name + "_sz"] + + def __post_init__(self): + if self.type_name in structs: + self.c_classifier = "struct " + symbolic_widths[self.type_name] = self.symbolic_width() + + +@dataclass +class _XdrBuiltInType(_XdrTypeSpecifier): + """Corresponds to a built-in XDR type""" + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return symbolic_widths[self.type_name] + + +@dataclass +class _XdrDeclaration(_XdrAst): + """Base class of XDR type declarations""" + + +@dataclass +class _XdrFixedLengthOpaque(_XdrDeclaration): + """A fixed-length opaque declaration""" + + name: str + size: str + template: str = "fixed_length_opaque" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return xdr_quadlen(self.size) + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return ["XDR_QUADLEN(" + self.size + ")"] + + def __post_init__(self): + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrVariableLengthOpaque(_XdrDeclaration): + """A variable-length opaque declaration""" + + name: str + maxsize: str + template: str = "variable_length_opaque" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return 1 + xdr_quadlen(self.maxsize) + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + widths = ["XDR_unsigned_int"] + if self.maxsize != "0": + widths.append("XDR_QUADLEN(" + self.maxsize + ")") + return widths + + def __post_init__(self): + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrString(_XdrDeclaration): + """A (NUL-terminated) variable-length string declaration""" + + name: str + maxsize: str + template: str = "string" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return 1 + xdr_quadlen(self.maxsize) + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + widths = ["XDR_unsigned_int"] + if self.maxsize != "0": + widths.append("XDR_QUADLEN(" + self.maxsize + ")") + return widths + + def __post_init__(self): + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrFixedLengthArray(_XdrDeclaration): + """A fixed-length array declaration""" + + name: str + spec: _XdrTypeSpecifier + size: str + template: str = "fixed_length_array" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return xdr_quadlen(self.size) * max_widths[self.spec.type_name] + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + item_width = " + ".join(symbolic_widths[self.spec.type_name]) + return ["(" + self.size + " * (" + item_width + "))"] + + def __post_init__(self): + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrVariableLengthArray(_XdrDeclaration): + """A variable-length array declaration""" + + name: str + spec: _XdrTypeSpecifier + maxsize: str + template: str = "variable_length_array" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return 1 + (xdr_quadlen(self.maxsize) * max_widths[self.spec.type_name]) + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + widths = ["XDR_unsigned_int"] + if self.maxsize != "0": + item_width = " + ".join(symbolic_widths[self.spec.type_name]) + widths.append("(" + self.maxsize + " * (" + item_width + "))") + return widths + + def __post_init__(self): + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrOptionalData(_XdrDeclaration): + """An 'optional_data' declaration""" + + name: str + spec: _XdrTypeSpecifier + template: str = "optional_data" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return 1 + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return ["XDR_bool"] + + def __post_init__(self): + structs.add(self.name) + pass_by_reference.add(self.name) + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrBasic(_XdrDeclaration): + """A 'basic' declaration""" + + name: str + spec: _XdrTypeSpecifier + template: str = "basic" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return max_widths[self.spec.type_name] + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return symbolic_widths[self.spec.type_name] + + def __post_init__(self): + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrVoid(_XdrDeclaration): + """A void declaration""" + + name: str = "void" + template: str = "void" + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return 0 + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return [] + + +@dataclass +class _XdrConstant(_XdrAst): + """Corresponds to 'constant_def' in the grammar""" + + name: str + value: str + + def __post_init__(self): + if self.value not in constants: + constants[self.name] = int(self.value, 0) + + +@dataclass +class _XdrEnumerator(_XdrAst): + """An 'identifier = value' enumerator""" + + name: str + value: str + + def __post_init__(self): + if self.value not in constants: + constants[self.name] = int(self.value, 0) + + +@dataclass +class _XdrEnum(_XdrAst): + """An XDR enum definition""" + + name: str + enumerators: List[_XdrEnumerator] + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return 1 + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return ["XDR_int"] + + def __post_init__(self): + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrStruct(_XdrAst): + """An XDR struct definition""" + + name: str + fields: List[_XdrDeclaration] + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + width = 0 + for field in self.fields: + width += field.max_width() + return width + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + widths = [] + for field in self.fields: + widths += field.symbolic_width() + return widths + + def __post_init__(self): + structs.add(self.name) + pass_by_reference.add(self.name) + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrPointer(_XdrAst): + """An XDR pointer definition""" + + name: str + fields: List[_XdrDeclaration] + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + width = 1 + for field in self.fields[0:-1]: + width += field.max_width() + return width + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + widths = [] + widths += ["XDR_bool"] + for field in self.fields[0:-1]: + widths += field.symbolic_width() + return widths + + def __post_init__(self): + structs.add(self.name) + pass_by_reference.add(self.name) + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _XdrTypedef(_XdrAst): + """An XDR typedef""" + + declaration: _XdrDeclaration + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + return self.declaration.max_width() + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + return self.declaration.symbolic_width() + + def __post_init__(self): + if isinstance(self.declaration, _XdrBasic): + new_type = self.declaration + if isinstance(new_type.spec, _XdrDefinedType): + if new_type.spec.type_name in pass_by_reference: + pass_by_reference.add(new_type.name) + max_widths[new_type.name] = self.max_width() + symbolic_widths[new_type.name] = self.symbolic_width() + + +@dataclass +class _XdrCaseSpec(_XdrAst): + """One case in an XDR union""" + + values: List[str] + arm: _XdrDeclaration + template: str = "case_spec" + + +@dataclass +class _XdrDefaultSpec(_XdrAst): + """Default case in an XDR union""" + + arm: _XdrDeclaration + template: str = "default_spec" + + +@dataclass +class _XdrUnion(_XdrAst): + """An XDR union""" + + name: str + discriminant: _XdrDeclaration + cases: List[_XdrCaseSpec] + default: _XdrDeclaration + + def max_width(self) -> int: + """Return width of type in XDR_UNITS""" + max_width = 0 + for case in self.cases: + if case.arm.max_width() > max_width: + max_width = case.arm.max_width() + if self.default: + if self.default.arm.max_width() > max_width: + max_width = self.default.arm.max_width() + return 1 + max_width + + def symbolic_width(self) -> List: + """Return list containing XDR width of type's components""" + max_width = 0 + for case in self.cases: + if case.arm.max_width() > max_width: + max_width = case.arm.max_width() + width = case.arm.symbolic_width() + if self.default: + if self.default.arm.max_width() > max_width: + max_width = self.default.arm.max_width() + width = self.default.arm.symbolic_width() + return symbolic_widths[self.discriminant.name] + width + + def __post_init__(self): + structs.add(self.name) + pass_by_reference.add(self.name) + max_widths[self.name] = self.max_width() + symbolic_widths[self.name] = self.symbolic_width() + + +@dataclass +class _RpcProcedure(_XdrAst): + """RPC procedure definition""" + + name: str + number: int + argument: _XdrTypeSpecifier + result: _XdrTypeSpecifier + + +@dataclass +class _RpcVersion(_XdrAst): + """RPC version definition""" + + name: str + number: int + procedures: List[_RpcProcedure] + + +@dataclass +class _RpcProgram(_XdrAst): + """RPC program definition""" + + name: str + number: int + versions: List[_RpcVersion] + + +@dataclass +class _Pragma(_XdrAst): + """Empty class for pragma directives""" + + +@dataclass +class _XdrPassthru(_XdrAst): + """Passthrough line to emit verbatim in output""" + + content: str + + +@dataclass +class Definition(_XdrAst, ast_utils.WithMeta): + """Corresponds to 'definition' in the grammar""" + + meta: Meta + value: _XdrAst + + +@dataclass +class Specification(_XdrAst, ast_utils.AsList): + """Corresponds to 'specification' in the grammar""" + + definitions: List[Definition] + + +class ParseToAst(Transformer): + """Functions that transform productions into AST nodes""" + + def identifier(self, children): + """Instantiate one _XdrIdentifier object""" + token = children[0] + return _XdrIdentifier(token.value, line=token.line, column=token.column) + + def value(self, children): + """Instantiate one _XdrValue object""" + if isinstance(children[0], _XdrIdentifier): + return _XdrValue(children[0].symbol) + return _XdrValue(children[0].children[0].value) + + def constant(self, children): + """Instantiate one _XdrConstantValue object""" + match children[0].data: + case "decimal_constant": + value = int(children[0].children[0].value, base=10) + case "hexadecimal_constant": + value = int(children[0].children[0].value, base=16) + case "octal_constant": + value = int(children[0].children[0].value, base=8) + return _XdrConstantValue(value) + + def type_specifier(self, children): + """Instantiate one _XdrTypeSpecifier object""" + if isinstance(children[0], _XdrIdentifier): + name = children[0].symbol + return _XdrDefinedType(type_name=name) + + name = children[0].data.value + return _XdrBuiltInType(type_name=name) + + def constant_def(self, children): + """Instantiate one _XdrConstant object""" + ident = children[0] + value = children[1].value + return _XdrConstant(ident.symbol, value, line=ident.line, column=ident.column) + + def enum(self, children): + """Instantiate one _XdrEnum object""" + name_ident = children[0] + + i = 0 + enumerators = [] + body = children[1] + while i < len(body.children): + ident = body.children[i] + value = body.children[i + 1].value + enumerators.append( + _XdrEnumerator( + ident.symbol, value, line=ident.line, column=ident.column + ) + ) + i = i + 2 + + return _XdrEnum( + name_ident.symbol, + enumerators, + line=name_ident.line, + column=name_ident.column, + ) + + def fixed_length_opaque(self, children): + """Instantiate one _XdrFixedLengthOpaque declaration object""" + ident = children[0] + size = children[1].value + + return _XdrFixedLengthOpaque( + ident.symbol, size, line=ident.line, column=ident.column + ) + + def variable_length_opaque(self, children): + """Instantiate one _XdrVariableLengthOpaque declaration object""" + ident = children[0] + if children[1] is not None: + maxsize = children[1].value + else: + maxsize = "0" + + return _XdrVariableLengthOpaque( + ident.symbol, maxsize, line=ident.line, column=ident.column + ) + + def string(self, children): + """Instantiate one _XdrString declaration object""" + ident = children[0] + if children[1] is not None: + maxsize = children[1].value + else: + maxsize = "0" + + return _XdrString(ident.symbol, maxsize, line=ident.line, column=ident.column) + + def fixed_length_array(self, children): + """Instantiate one _XdrFixedLengthArray declaration object""" + spec = children[0] + ident = children[1] + size = children[2].value + + return _XdrFixedLengthArray( + ident.symbol, spec, size, line=ident.line, column=ident.column + ) + + def variable_length_array(self, children): + """Instantiate one _XdrVariableLengthArray declaration object""" + spec = children[0] + ident = children[1] + if children[2] is not None: + maxsize = children[2].value + else: + maxsize = "0" + + return _XdrVariableLengthArray( + ident.symbol, spec, maxsize, line=ident.line, column=ident.column + ) + + def optional_data(self, children): + """Instantiate one _XdrOptionalData declaration object""" + spec = children[0] + ident = children[1] + + return _XdrOptionalData( + ident.symbol, spec, line=ident.line, column=ident.column + ) + + def basic(self, children): + """Instantiate one _XdrBasic object""" + spec = children[0] + ident = children[1] + + return _XdrBasic(ident.symbol, spec, line=ident.line, column=ident.column) + + def void(self, children): + """Instantiate one _XdrVoid declaration object""" + + return _XdrVoid() + + def struct(self, children): + """Instantiate one _XdrStruct object""" + ident = children[0] + name = ident.symbol + fields = children[1].children + pos = {"line": ident.line, "column": ident.column} + + last_field = fields[-1] + if ( + isinstance(last_field, _XdrOptionalData) + and name == last_field.spec.type_name + ): + return _XdrPointer(name, fields, **pos) + + return _XdrStruct(name, fields, **pos) + + def typedef(self, children): + """Instantiate one _XdrTypedef object""" + new_type = children[0] + + return _XdrTypedef(new_type) + + def case_spec(self, children): + """Instantiate one _XdrCaseSpec object""" + values = [] + for item in children[0:-1]: + values.append(item.value) + arm = children[-1] + + return _XdrCaseSpec(values, arm) + + def default_spec(self, children): + """Instantiate one _XdrDefaultSpec object""" + arm = children[0] + + return _XdrDefaultSpec(arm) + + def union(self, children): + """Instantiate one _XdrUnion object""" + ident = children[0] + + body = children[1] + discriminant = body.children[0].children[0] + cases = body.children[1:-1] + default = body.children[-1] + + return _XdrUnion( + ident.symbol, + discriminant, + cases, + default, + line=ident.line, + column=ident.column, + ) + + def procedure_def(self, children): + """Instantiate one _RpcProcedure object""" + result = children[0] + ident = children[1] + argument = children[2] + number = children[3].value + + return _RpcProcedure( + ident.symbol, + number, + argument, + result, + line=ident.line, + column=ident.column, + ) + + def version_def(self, children): + """Instantiate one _RpcVersion object""" + ident = children[0] + number = children[-1].value + procedures = children[1:-1] + + return _RpcVersion( + ident.symbol, number, procedures, line=ident.line, column=ident.column + ) + + def program_def(self, children): + """Instantiate one _RpcProgram object""" + ident = children[0] + number = children[-1].value + versions = children[1:-1] + + return _RpcProgram( + ident.symbol, number, versions, line=ident.line, column=ident.column + ) + + def pragma_def(self, children): + """Instantiate one _Pragma object""" + directive = children[0].children[0].data + match directive: + case "big_endian_directive": + big_endian.append(children[1].symbol) + case "exclude_directive": + excluded_apis.append(children[1].symbol) + case "header_directive": + global header_name + header_name = children[1].symbol + case "public_directive": + public_apis.append(children[1].symbol) + case _: + raise NotImplementedError("Directive not supported") + return _Pragma() + + def passthru_def(self, children): + """Instantiate one _XdrPassthru object""" + token = children[0] + content = token.value[1:] + return _XdrPassthru(content) + + +transformer = ast_utils.create_transformer(this_module, ParseToAst()) + + +def _merge_consecutive_passthru(definitions: List[Definition]) -> List[Definition]: + """Merge consecutive passthru definitions into single nodes""" + result = [] + i = 0 + while i < len(definitions): + if isinstance(definitions[i].value, _XdrPassthru): + lines = [definitions[i].value.content] + meta = definitions[i].meta + j = i + 1 + while j < len(definitions) and isinstance( + definitions[j].value, _XdrPassthru + ): + lines.append(definitions[j].value.content) + j += 1 + merged = _XdrPassthru("\n".join(lines)) + result.append(Definition(meta, merged)) + i = j + else: + result.append(definitions[i]) + i += 1 + return result + + +def _meta_line(meta) -> int: + """Return the 1-based source line for a node's meta, or 0 if unknown""" + try: + return meta.line + except AttributeError: + return 0 + + +class XdrSemanticError(Exception): + """A specification that parses but violates an XDR semantic rule. + + Detection lives in the language-independent front end because a + duplicate name is malformed XDR regardless of the output language. + """ + + def __init__(self, message: str, meta): + super().__init__(message) + self.message = message + self.line = _meta_line(meta) + self.column = getattr(meta, "column", 0) + + +def _introduced_names(value): + """Yield (name, node) for each identifier a definition introduces.""" + if isinstance(value, (_XdrStruct, _XdrUnion, _XdrPointer)): + yield value.name, value + elif isinstance(value, _XdrEnum): + yield value.name, value + for enumerator in value.enumerators: + yield enumerator.name, enumerator + elif isinstance(value, _XdrTypedef): + yield value.declaration.name, value.declaration + elif isinstance(value, _XdrConstant): + yield value.name, value + elif isinstance(value, _RpcProgram): + yield value.name, value + + +def _check_rpc_scope_names(program: "_RpcProgram") -> None: + """Enforce RFC 5531 Section 12.3 scoping within an RPC program. + + A version name and number are unique within the program and a + procedure name and number are unique within its version. + """ + version_names = set() + version_numbers = set() + for version in program.versions: + if version.name in version_names: + raise XdrSemanticError( + f"duplicate version name '{version.name}'" + f" in program '{program.name}'", + version, + ) + version_names.add(version.name) + if version.number in version_numbers: + raise XdrSemanticError( + f"duplicate version number {version.number}" + f" in program '{program.name}'", + version, + ) + version_numbers.add(version.number) + procedure_names = set() + procedure_numbers = set() + for procedure in version.procedures: + if procedure.name in procedure_names: + raise XdrSemanticError( + f"duplicate procedure name '{procedure.name}'" + f" in version '{version.name}'", + procedure, + ) + procedure_names.add(procedure.name) + if procedure.number in procedure_numbers: + raise XdrSemanticError( + f"duplicate procedure number {procedure.number}" + f" in version '{version.name}'", + procedure, + ) + procedure_numbers.add(procedure.number) + + +def check_duplicate_definitions(root: "Specification") -> None: + """Reject a spec that declares an identifier more than once. + + RFC 4506 Section 6.4 places constant and type identifiers in a + single name space that must be unique within a specification. + RFC 5531 Section 12.3 adds RPC program names to that name space + and scopes version names and numbers to their program and + procedure names and numbers to their version. + """ + seen = {} + for definition in root.definitions: + for name, node in _introduced_names(definition.value): + where = node if node.line else definition.meta + first = seen.get(name) + if first is not None: + raise XdrSemanticError( + f"duplicate identifier '{name}'" + f" (first declared at line {_meta_line(first)})", + where, + ) + seen[name] = where + if isinstance(definition.value, _RpcProgram): + _check_rpc_scope_names(definition.value) + + +# RFC 5531 (Section 9) encodes program, version, and procedure numbers +# as unsigned 32-bit integers, so each must fall within [0, 2**32 - 1]. +_RPC_NUMBER_MAX = 2**32 - 1 + + +def _check_rpc_number(kind: str, number: int, scope: str, meta) -> None: + """Reject one RPC number that is negative or wider than 32 bits.""" + if number < 0: + raise XdrSemanticError( + f"negative {kind} number {number} {scope}", + meta, + ) + if number > _RPC_NUMBER_MAX: + raise XdrSemanticError( + f"{kind} number {number} {scope} exceeds {_RPC_NUMBER_MAX}", + meta, + ) + + +def check_rpc_number_range(root: "Specification") -> None: + """Reject an out-of-range program, version, or procedure number. + + RFC 5531 assigns only unsigned constants to program, version, and + procedure numbers (Section 12.3) and encodes each as an unsigned + 32-bit integer (Section 9). RFC 4506 Section 6.2 permits a signed + decimal constant for XDR constants in general and sets no ceiling on + magnitude, so the grammar accepts an out-of-range value; the range + is enforced here instead. The parser retains no per-version or + per-procedure source location, so a violation is reported against the + program definition. + """ + for definition in root.definitions: + program = definition.value + if not isinstance(program, _RpcProgram): + continue + _check_rpc_number( + "program", + program.number, + f"in program '{program.name}'", + definition.meta, + ) + for version in program.versions: + _check_rpc_number( + "version", + version.number, + f"in program '{program.name}'", + definition.meta, + ) + for procedure in version.procedures: + _check_rpc_number( + "procedure", + procedure.number, + f"in version '{version.name}'", + definition.meta, + ) + + +def transform_parse_tree(parse_tree): + """Transform productions into an abstract syntax tree""" + ast = transformer.transform(parse_tree) + ast.definitions = _merge_consecutive_passthru(ast.definitions) + check_duplicate_definitions(ast) + check_rpc_number_range(ast) + return ast + + +def get_header_name() -> str: + """Return header name set by pragma header directive""" + return header_name diff --git a/tools/net/sunrpc/xdrgen/xdr_parse.py b/tools/net/sunrpc/xdrgen/xdr_parse.py new file mode 100644 index 000000000..78298553e --- /dev/null +++ b/tools/net/sunrpc/xdrgen/xdr_parse.py @@ -0,0 +1,204 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Common parsing code for xdrgen""" + +import sys +from typing import Callable + +from lark import Lark +from lark.exceptions import UnexpectedInput, UnexpectedToken, VisitError + + +# Set to True to emit annotation comments in generated source +annotate = False + +# Set to True to emit enum value validation in decoders +enum_validation = True + +# Map internal Lark token names to human-readable names +TOKEN_NAMES = { + "__ANON_0": "identifier", + "__ANON_1": "number", + "SEMICOLON": "';'", + "LBRACE": "'{'", + "RBRACE": "'}'", + "LPAR": "'('", + "RPAR": "')'", + "LSQB": "'['", + "RSQB": "']'", + "LESSTHAN": "'<'", + "MORETHAN": "'>'", + "EQUAL": "'='", + "COLON": "':'", + "COMMA": "','", + "STAR": "'*'", + "$END": "end of file", +} + + +class XdrParseError(Exception): + """Raised when XDR parsing fails""" + + +def set_xdr_annotate(set_it: bool) -> None: + """Set 'annotate' if --annotate was specified on the command line""" + global annotate + annotate = set_it + + +def get_xdr_annotate() -> bool: + """Return True if --annotate was specified on the command line""" + return annotate + + +def set_xdr_enum_validation(set_it: bool) -> None: + """Set 'enum_validation' based on command line options""" + global enum_validation + enum_validation = set_it + + +def get_xdr_enum_validation() -> bool: + """Return True when enum validation is enabled for decoder generation""" + return enum_validation + + +def format_source_caret(line_text: str, column: int) -> list[str]: + """Render an offending source line with a caret beneath a column. + + Args: + line_text: The raw source line containing the error + column: 1-based column of the offending token within line_text + + Returns: + Output lines for the diagnostic: a blank separator, the source + line with tabs expanded, and a caret aligned under the column. + """ + expanded = line_text.expandtabs() + caret = len(line_text[: column - 1].expandtabs()) + return ["", f" {expanded}", f" {' ' * caret}^"] + + +def make_error_handler(source: str, filename: str) -> Callable[[UnexpectedInput], bool]: + """Create an error handler that reports the first parse error and aborts. + + Args: + source: The XDR source text being parsed + filename: The name of the file being parsed + + Returns: + An error handler function for use with Lark's on_error parameter + """ + lines = source.splitlines() + + def handle_parse_error(e: UnexpectedInput) -> bool: + """Report a parse error with context and abort parsing""" + line_num = e.line + column = e.column + line_text = lines[line_num - 1] if 0 < line_num <= len(lines) else "" + + # Build the error message + msg_parts = [f"{filename}:{line_num}:{column}: parse error"] + + # Show what was found vs what was expected + if isinstance(e, UnexpectedToken): + token = e.token + if token.type == "__ANON_0": + found = f"identifier '{token.value}'" + elif token.type == "__ANON_1": + found = f"number '{token.value}'" + else: + found = f"'{token.value}'" + msg_parts.append(f"Unexpected {found}") + + # Provide helpful expected tokens list + expected = e.expected + if expected: + readable = [ + TOKEN_NAMES.get(exp, exp.lower().replace("_", " ")) + for exp in sorted(expected) + ] + if len(readable) == 1: + msg_parts.append(f"Expected {readable[0]}") + elif len(readable) <= 4: + msg_parts.append(f"Expected one of: {', '.join(readable)}") + else: + msg_parts.append(str(e).split("\n")[0]) + + # Show the offending line with a caret pointing to the error + msg_parts.extend(format_source_caret(line_text, column)) + + sys.stderr.write("\n".join(msg_parts) + "\n") + raise XdrParseError() + + return handle_parse_error + + +def handle_transform_error(e: VisitError, source: str, filename: str) -> None: + """Report a transform error with context. + + Args: + e: The VisitError from Lark's transformer + source: The XDR source text being parsed + filename: The name of the file being parsed + """ + lines = source.splitlines() + + # Extract position from the tree node if available + line_num = 0 + column = 0 + if hasattr(e.obj, "meta") and e.obj.meta: + line_num = e.obj.meta.line + column = e.obj.meta.column + + line_text = lines[line_num - 1] if 0 < line_num <= len(lines) else "" + + # Build the error message + msg_parts = [f"{filename}:{line_num}:{column}: semantic error"] + + # The original exception is typically a KeyError for undefined types + if isinstance(e.orig_exc, KeyError): + msg_parts.append(f"Undefined type '{e.orig_exc.args[0]}'") + else: + msg_parts.append(str(e.orig_exc)) + + # Show the offending line with a caret pointing to the error + if line_text: + msg_parts.extend(format_source_caret(line_text, column)) + + sys.stderr.write("\n".join(msg_parts) + "\n") + + +def handle_semantic_error(e, source: str, filename: str) -> None: + """Report a semantic error (e.g., a duplicate name) with context. + + Args: + e: The XdrSemanticError carrying message and source position + source: The XDR source text being parsed + filename: The name of the file being parsed + """ + lines = source.splitlines() + line_num = getattr(e, "line", 0) + column = getattr(e, "column", 0) + line_text = lines[line_num - 1] if 0 < line_num <= len(lines) else "" + + msg_parts = [f"{filename}:{line_num}:{column}: semantic error", e.message] + if line_text: + msg_parts.extend(format_source_caret(line_text, column)) + + sys.stderr.write("\n".join(msg_parts) + "\n") + + +def xdr_parser() -> Lark: + """Return a Lark parser instance configured with the XDR language grammar""" + + return Lark.open( + "grammars/xdr.lark", + rel_to=__file__, + start="specification", + debug=True, + strict=True, + propagate_positions=True, + parser="lalr", + lexer="contextual", + ) diff --git a/tools/net/sunrpc/xdrgen/xdrgen b/tools/net/sunrpc/xdrgen/xdrgen new file mode 100755 index 000000000..b2fb43f4a --- /dev/null +++ b/tools/net/sunrpc/xdrgen/xdrgen @@ -0,0 +1,143 @@ +#!/usr/bin/env python3 +# ex: set filetype=python: + +"""Translate an XDR specification into executable code that +can be compiled for the Linux kernel.""" + +__author__ = "Chuck Lever" +__copyright__ = "Copyright (c) 2024 Oracle and/or its affiliates." +__license__ = "GPL-2.0 only" +__version__ = "0.2" + +import sys +from pathlib import Path +import argparse + +_XDRGEN_DIR = Path(__file__).resolve().parent +if str(_XDRGEN_DIR) not in sys.path: + sys.path.insert(0, str(_XDRGEN_DIR)) + +from subcmds import definitions +from subcmds import declarations +from subcmds import lint +from subcmds import source + + +sys.path.insert(1, "@pythondir@") + + +def main() -> int: + """Parse command-line options""" + parser = argparse.ArgumentParser( + formatter_class=argparse.RawDescriptionHelpFormatter, + description="Convert an XDR specification to Linux kernel source code", + epilog="""\ +Copyright (c) 2024 Oracle and/or its affiliates. + +License GPLv2: <http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt> +This is free software. You are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law.""", + ) + parser.add_argument( + "--version", + help="Display the version of this tool", + action="version", + version=__version__, + ) + + subcommands = parser.add_subparsers(title="Subcommands", required=True) + + definitions_parser = subcommands.add_parser( + "definitions", help="Generate XDR definitions" + ) + definitions_parser.add_argument( + "--annotate", + action="store_true", + default=False, + help="Add annotation comments", + ) + definitions_parser.add_argument( + "--language", + action="store_true", + default="C", + help="Output language", + ) + definitions_parser.add_argument( + "--peer", + choices=["server", "client",], + default="server", + help="Generate header code for client or server side", + type=str, + ) + definitions_parser.add_argument("filename", help="File containing an XDR specification") + definitions_parser.set_defaults(func=definitions.subcmd) + + declarations_parser = subcommands.add_parser( + "declarations", help="Generate function declarations" + ) + declarations_parser.add_argument( + "--annotate", + action="store_true", + default=False, + help="Add annotation comments", + ) + declarations_parser.add_argument( + "--language", + action="store_true", + default="C", + help="Output language", + ) + declarations_parser.add_argument( + "--peer", + choices=["server", "client",], + default="server", + help="Generate code for client or server side", + type=str, + ) + declarations_parser.add_argument("filename", help="File containing an XDR specification") + declarations_parser.set_defaults(func=declarations.subcmd) + + linter_parser = subcommands.add_parser("lint", help="Check an XDR specification") + linter_parser.add_argument("filename", help="File containing an XDR specification") + linter_parser.set_defaults(func=lint.subcmd) + + source_parser = subcommands.add_parser( + "source", help="Generate XDR encoder and decoder source code" + ) + source_parser.add_argument( + "--annotate", + action="store_true", + default=False, + help="Add annotation comments", + ) + source_parser.add_argument( + "--language", + action="store_true", + default="C", + help="Output language", + ) + source_parser.add_argument( + "--peer", + choices=["server", "client",], + default="server", + help="Generate code for client or server side", + type=str, + ) + source_parser.add_argument( + "--no-enum-validation", + action="store_true", + default=False, + help="Disable enum value validation in decoders", + ) + source_parser.add_argument("filename", help="File containing an XDR specification") + source_parser.set_defaults(func=source.subcmd) + + args = parser.parse_args() + return args.func(args) + + +try: + if __name__ == "__main__": + sys.exit(main()) +except (KeyboardInterrupt, BrokenPipeError): + sys.exit(1) diff --git a/tools/net/ynl/Makefile b/tools/net/ynl/Makefile new file mode 100644 index 000000000..3cefe4ed9 --- /dev/null +++ b/tools/net/ynl/Makefile @@ -0,0 +1,77 @@ +# SPDX-License-Identifier: GPL-2.0 + +include ../../scripts/Makefile.arch + +INSTALL ?= install +prefix ?= /usr +ifeq ($(LP64), 1) + libdir_relative = lib64 +else + libdir_relative = lib +endif +libdir ?= $(prefix)/$(libdir_relative) +includedir ?= $(prefix)/include + +SPECDIR=../../../Documentation/netlink/specs + +SUBDIRS = lib generated ynltool tests + +all: $(SUBDIRS) libynl.a + +tests: | lib generated libynl.a +ynltool: | lib generated libynl.a +libynl.a: | lib generated + @echo -e "\tAR $@" + @$(AR) rcs $@ lib/ynl.o generated/*-user.o + +$(SUBDIRS): + @if [ -f "$@/Makefile" ] ; then \ + $(MAKE) -C $@ ; \ + fi + +clean distclean: + @for dir in $(SUBDIRS) ; do \ + if [ -f "$$dir/Makefile" ] ; then \ + $(MAKE) -C $$dir $@; \ + fi \ + done + rm -f libynl.a + rm -rf pyynl/__pycache__ + rm -rf pyynl/lib/__pycache__ + rm -rf pyynl.egg-info + rm -rf build + +install: libynl.a lib/*.h ynltool + @echo -e "\tINSTALL libynl.a" + @$(INSTALL) -d $(DESTDIR)$(libdir) + @$(INSTALL) -m 0644 libynl.a $(DESTDIR)$(libdir)/libynl.a + @echo -e "\tINSTALL libynl headers" + @$(INSTALL) -d $(DESTDIR)$(includedir)/ynl + @$(INSTALL) -m 0644 lib/*.h $(DESTDIR)$(includedir)/ynl/ + @echo -e "\tINSTALL pyynl" + @pip install --prefix=$(DESTDIR)$(prefix) . + @make -C generated install + @make -C ynltool install + +run_tests: + @$(MAKE) -C tests run_tests + +lint: + yamllint $(SPECDIR) + +schema_check: + @N=1; \ + for spec in $(SPECDIR)/*.yaml ; do \ + NAME=$$(basename $$spec) ; \ + OUTPUT=$$(./pyynl/cli.py --spec $$spec --validate) ; \ + if [ $$? -eq 0 ] ; then \ + echo "ok $$N $$NAME schema validation" ; \ + else \ + echo "not ok $$N $$NAME schema validation" ; \ + echo "$$OUTPUT" ; \ + echo ; \ + fi ; \ + N=$$((N+1)) ; \ + done + +.PHONY: all clean distclean install run_tests lint schema_check $(SUBDIRS) diff --git a/tools/net/ynl/Makefile.deps b/tools/net/ynl/Makefile.deps new file mode 100644 index 000000000..277137533 --- /dev/null +++ b/tools/net/ynl/Makefile.deps @@ -0,0 +1,57 @@ +# SPDX-License-Identifier: GPL-2.0 + +# Try to include uAPI headers from the kernel uapi/ path. +# Most code under tools/ requires the respective kernel uAPI headers +# to be copied to tools/include. The duplication is annoying. +# All the family headers should be self-contained. We avoid the copying +# by selectively including just the uAPI header of the family directly +# from the kernel sources. + +UAPI_PATH:=../../../../include/uapi/ + +# scripts/headers_install.sh strips "_UAPI" from header guards so we +# need the explicit -D matching what's in /usr, to avoid multiple definitions. + +get_hdr_inc=-D$(1) -include $(UAPI_PATH)/linux/$(2) +get_hdr_inc2=-D$(1) -D$(2) -include $(UAPI_PATH)/linux/$(3) +get_hdr_inc_drm=-D$(1) -include $(UAPI_PATH)/drm/$(2) + +CFLAGS_dev-energymodel:=$(call get_hdr_inc,_LINUX_DEV_ENERGYMODEL_H,dev_energymodel.h) +CFLAGS_devlink:=$(call get_hdr_inc,_LINUX_DEVLINK_H_,devlink.h) +CFLAGS_dpll:=$(call get_hdr_inc,_LINUX_DPLL_H,dpll.h) +CFLAGS_drm_ras:=$(call get_hdr_inc_drm,_LINUX_DRM_RAS_H,drm_ras.h) +CFLAGS_ethtool:=$(call get_hdr_inc,_LINUX_TYPELIMITS_H,typelimits.h) \ + $(call get_hdr_inc,_LINUX_ETHTOOL_H,ethtool.h) \ + $(call get_hdr_inc,_LINUX_ETHTOOL_NETLINK_H_,ethtool_netlink.h) \ + $(call get_hdr_inc,_LINUX_ETHTOOL_NETLINK_GENERATED_H,ethtool_netlink_generated.h) +CFLAGS_handshake:=$(call get_hdr_inc,_LINUX_HANDSHAKE_H,handshake.h) +CFLAGS_lockd_netlink:=$(call get_hdr_inc,_LINUX_LOCKD_NETLINK_H,lockd_netlink.h) +CFLAGS_mptcp_pm:=$(call get_hdr_inc,_LINUX_MPTCP_PM_H,mptcp_pm.h) +CFLAGS_net_shaper:=$(call get_hdr_inc,_LINUX_NET_SHAPER_H,net_shaper.h) +CFLAGS_netdev:=$(call get_hdr_inc,_LINUX_NETDEV_H,netdev.h) +CFLAGS_nl80211:=$(call get_hdr_inc,__LINUX_NL802121_H,nl80211.h) +CFLAGS_nlctrl:=$(call get_hdr_inc,__LINUX_GENERIC_NETLINK_H,genetlink.h) +CFLAGS_nfsd:=$(call get_hdr_inc,_LINUX_NFSD_NETLINK_H,nfsd_netlink.h) +CFLAGS_ovpn:=$(call get_hdr_inc,_LINUX_OVPN_H,ovpn.h) +CFLAGS_ovs_datapath:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h) +CFLAGS_ovs_flow:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h) +CFLAGS_ovs_packet:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h) +CFLAGS_ovs_vport:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h) +CFLAGS_psp:=$(call get_hdr_inc,_LINUX_PSP_H,psp.h) +CFLAGS_rt-addr:=$(call get_hdr_inc,__LINUX_RTNETLINK_H,rtnetlink.h) \ + $(call get_hdr_inc,__LINUX_IF_ADDR_H,if_addr.h) +CFLAGS_rt-link:=$(call get_hdr_inc,__LINUX_RTNETLINK_H,rtnetlink.h) \ + $(call get_hdr_inc,_LINUX_IF_LINK_H,if_link.h) +CFLAGS_rt-neigh:=$(call get_hdr_inc,__LINUX_RTNETLINK_H,rtnetlink.h) \ + $(call get_hdr_inc,__LINUX_NEIGHBOUR_H,neighbour.h) +CFLAGS_rt-route:=$(call get_hdr_inc,__LINUX_RTNETLINK_H,rtnetlink.h) +CFLAGS_rt-rule:=$(call get_hdr_inc,__LINUX_FIB_RULES_H,fib_rules.h) +CFLAGS_tc:= $(call get_hdr_inc,__LINUX_RTNETLINK_H,rtnetlink.h) \ + $(call get_hdr_inc,__LINUX_PKT_SCHED_H,pkt_sched.h) \ + $(call get_hdr_inc,__LINUX_PKT_CLS_H,pkt_cls.h) \ + $(call get_hdr_inc,_TC_CT_H,tc_act/tc_ct.h) \ + $(call get_hdr_inc,_TC_MIRRED_H,tc_act/tc_mirred.h) \ + $(call get_hdr_inc,_TC_SKBEDIT_H,tc_act/tc_skbedit.h) \ + $(call get_hdr_inc,_TC_TUNNEL_KEY_H,tc_act/tc_tunnel_key.h) +CFLAGS_tcp_metrics:=$(call get_hdr_inc,_LINUX_TCP_METRICS_H,tcp_metrics.h) +CFLAGS_wireguard:=$(call get_hdr_inc2,_LINUX_WIREGUARD_H,_WG_UAPI_WIREGUARD_H,wireguard.h) diff --git a/tools/net/ynl/generated/.gitignore b/tools/net/ynl/generated/.gitignore new file mode 100644 index 000000000..859a6fb44 --- /dev/null +++ b/tools/net/ynl/generated/.gitignore @@ -0,0 +1,3 @@ +*-user.c +*-user.h +*.rst diff --git a/tools/net/ynl/generated/Makefile b/tools/net/ynl/generated/Makefile new file mode 100644 index 000000000..ea4128f61 --- /dev/null +++ b/tools/net/ynl/generated/Makefile @@ -0,0 +1,87 @@ +# SPDX-License-Identifier: GPL-2.0 + +CC=gcc +CFLAGS += -std=gnu11 -O2 -W -Wall -Wextra -Wno-unused-parameter -Wshadow \ + -I../lib/ -idirafter $(UAPI_PATH) +ifeq ("$(DEBUG)","1") + CFLAGS += -g -fsanitize=address -fsanitize=leak -static-libasan +endif + +INSTALL ?= install +prefix ?= /usr +datarootdir ?= $(prefix)/share +docdir ?= $(datarootdir)/doc +includedir ?= $(prefix)/include + +include ../Makefile.deps + +YNL_GEN_ARG_ethtool:=--user-header linux/ethtool_netlink.h \ + --exclude-op stats-get + +TOOL:=../pyynl/ynl_gen_c.py +TOOL_RST:=../pyynl/ynl_gen_rst.py + +SPECS_DIR:=../../../../Documentation/netlink/specs +SPECS_PATHS=$(wildcard $(SPECS_DIR)/*.yaml) +GENS_UNSUP=conntrack nftables +GENS=$(filter-out ${GENS_UNSUP},$(patsubst $(SPECS_DIR)/%.yaml,%,${SPECS_PATHS})) +SRCS=$(patsubst %,%-user.c,${GENS}) +HDRS=$(patsubst %,%-user.h,${GENS}) +OBJS=$(patsubst %,%-user.o,${GENS}) + +SPECS_PATHS=$(wildcard $(SPECS_DIR)/*.yaml) +SPECS=$(patsubst $(SPECS_DIR)/%.yaml,%,${SPECS_PATHS}) +RSTS=$(patsubst %,%.rst,${SPECS}) + +all: protos.a $(HDRS) $(SRCS) $(KHDRS) $(KSRCS) $(UAPI) $(RSTS) + +protos.a: $(OBJS) + @echo -e "\tAR $@" + @$(AR) rcs $@ $(OBJS) + +%-user.h: $(SPECS_DIR)/%.yaml $(TOOL) + @echo -e "\tGEN $@" + @$(TOOL) --mode user --header --spec $< -o $@ $(YNL_GEN_ARG_$*) + +%-user.c: $(SPECS_DIR)/%.yaml $(TOOL) + @echo -e "\tGEN $@" + @$(TOOL) --mode user --source --spec $< -o $@ $(YNL_GEN_ARG_$*) + +%-user.o: %-user.c %-user.h + @echo -e "\tCC $@" + @$(COMPILE.c) $(CFLAGS_$*) -o $@ $< + +%.rst: $(SPECS_DIR)/%.yaml $(TOOL_RST) + @echo -e "\tGEN_RST $@" + @$(TOOL_RST) -o $@ -i $< + +clean: + rm -f *.o + +distclean: clean + rm -f *.c *.h *.a *.rst + +regen: + @../ynl-regen.sh + +install-headers: $(HDRS) + @echo -e "\tINSTALL generated headers" + @$(INSTALL) -d $(DESTDIR)$(includedir)/ynl + @$(INSTALL) -m 0644 *.h $(DESTDIR)$(includedir)/ynl/ + +install-rsts: $(RSTS) + @echo -e "\tINSTALL generated docs" + @$(INSTALL) -d $(DESTDIR)$(docdir)/ynl + @$(INSTALL) -m 0644 $(RSTS) $(DESTDIR)$(docdir)/ynl/ + +install-specs: + @echo -e "\tINSTALL specs" + @$(INSTALL) -d $(DESTDIR)$(datarootdir)/ynl + @$(INSTALL) -m 0644 ../../../../Documentation/netlink/*.yaml $(DESTDIR)$(datarootdir)/ynl/ + @$(INSTALL) -d $(DESTDIR)$(datarootdir)/ynl/specs + @$(INSTALL) -m 0644 $(SPECS_DIR)/*.yaml $(DESTDIR)$(datarootdir)/ynl/specs/ + +install: install-headers install-rsts install-specs + +.PHONY: all clean distclean regen install install-headers install-rsts install-specs +.DEFAULT_GOAL: all diff --git a/tools/net/ynl/lib/.gitignore b/tools/net/ynl/lib/.gitignore new file mode 100644 index 000000000..a4383358e --- /dev/null +++ b/tools/net/ynl/lib/.gitignore @@ -0,0 +1 @@ +*.d diff --git a/tools/net/ynl/lib/Makefile b/tools/net/ynl/lib/Makefile new file mode 100644 index 000000000..9b98c0599 --- /dev/null +++ b/tools/net/ynl/lib/Makefile @@ -0,0 +1,30 @@ +# SPDX-License-Identifier: GPL-2.0 + +CC=gcc +CFLAGS += -std=gnu11 -O2 -W -Wall -Wextra -Wno-unused-parameter -Wshadow +ifeq ("$(DEBUG)","1") + CFLAGS += -g -fsanitize=address -fsanitize=leak -static-libasan +endif + +SRCS=$(wildcard *.c) +OBJS=$(patsubst %.c,%.o,${SRCS}) + +include $(wildcard *.d) + +all: ynl.a + +ynl.a: $(OBJS) + @echo -e "\tAR $@" + @$(AR) rcs $@ $(OBJS) + +clean: + rm -f *.o *.d *~ + +distclean: clean + rm -f *.a + +%.o: %.c + $(COMPILE.c) -MMD -c -o $@ $< + +.PHONY: all clean distclean +.DEFAULT_GOAL=all diff --git a/tools/net/ynl/lib/ynl-priv.h b/tools/net/ynl/lib/ynl-priv.h new file mode 100644 index 000000000..ced7dce44 --- /dev/null +++ b/tools/net/ynl/lib/ynl-priv.h @@ -0,0 +1,478 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ +#ifndef __YNL_C_PRIV_H +#define __YNL_C_PRIV_H 1 + +#include <stdbool.h> +#include <stddef.h> +#include <linux/types.h> + +struct ynl_parse_arg; + +/* + * YNL internals / low level stuff + */ + +enum ynl_policy_type { + YNL_PT_REJECT = 1, + YNL_PT_IGNORE, + YNL_PT_NEST, + YNL_PT_FLAG, + YNL_PT_BINARY, + YNL_PT_U8, + YNL_PT_U16, + YNL_PT_U32, + YNL_PT_U64, + YNL_PT_UINT, + YNL_PT_NUL_STR, + YNL_PT_BITFIELD32, + YNL_PT_SUBMSG, +}; + +enum ynl_parse_result { + YNL_PARSE_CB_ERROR = -1, + YNL_PARSE_CB_STOP = 0, + YNL_PARSE_CB_OK = 1, +}; + +#define YNL_SOCKET_BUFFER_SIZE (1 << 17) + +#define YNL_ARRAY_SIZE(array) (sizeof(array) ? \ + sizeof(array) / sizeof(array[0]) : 0) + +typedef int (*ynl_parse_cb_t)(const struct nlmsghdr *nlh, + struct ynl_parse_arg *yarg); + +struct ynl_policy_attr { + enum ynl_policy_type type:8; + __u8 is_submsg:1; + __u8 is_selector:1; + __u16 selector_type; + unsigned int len; + const char *name; + const struct ynl_policy_nest *nest; +}; + +struct ynl_policy_nest { + unsigned int max_attr; + const struct ynl_policy_attr *table; +}; + +struct ynl_parse_arg { + struct ynl_sock *ys; + const struct ynl_policy_nest *rsp_policy; + void *data; +}; + +struct ynl_dump_list_type { + struct ynl_dump_list_type *next; + unsigned char data[] __attribute__((aligned(8))); +}; +extern struct ynl_dump_list_type *YNL_LIST_END; + +static inline bool ynl_dump_obj_is_last(void *obj) +{ + unsigned long uptr = (unsigned long)obj; + + uptr -= offsetof(struct ynl_dump_list_type, data); + return uptr == (unsigned long)YNL_LIST_END; +} + +static inline void *ynl_dump_obj_next(void *obj) +{ + unsigned long uptr = (unsigned long)obj; + struct ynl_dump_list_type *list; + + uptr -= offsetof(struct ynl_dump_list_type, data); + list = (struct ynl_dump_list_type *)uptr; + uptr = (unsigned long)list->next; + uptr += offsetof(struct ynl_dump_list_type, data); + + return (void *)uptr; +} + +struct ynl_ntf_base_type { + __u16 family; + __u8 cmd; + struct ynl_ntf_base_type *next; + void (*free)(struct ynl_ntf_base_type *ntf); + unsigned char data[] __attribute__((aligned(8))); +}; + +struct nlmsghdr *ynl_msg_start_req(struct ynl_sock *ys, __u32 id, __u16 flags); +struct nlmsghdr *ynl_msg_start_dump(struct ynl_sock *ys, __u32 id); + +struct nlmsghdr * +ynl_gemsg_start_req(struct ynl_sock *ys, __u32 id, __u8 cmd, __u8 version); +struct nlmsghdr * +ynl_gemsg_start_dump(struct ynl_sock *ys, __u32 id, __u8 cmd, __u8 version); + +int ynl_submsg_failed(struct ynl_parse_arg *yarg, const char *field_name, + const char *sel_name); + +/* YNL specific helpers used by the auto-generated code */ + +struct ynl_req_state { + struct ynl_parse_arg yarg; + ynl_parse_cb_t cb; + __u32 rsp_cmd; +}; + +struct ynl_dump_state { + struct ynl_parse_arg yarg; + void *first; + struct ynl_dump_list_type *last; + size_t alloc_sz; + ynl_parse_cb_t cb; + __u32 rsp_cmd; +}; + +struct ynl_ntf_info { + const struct ynl_policy_nest *policy; + ynl_parse_cb_t cb; + size_t alloc_sz; + void (*free)(struct ynl_ntf_base_type *ntf); +}; + +int ynl_exec(struct ynl_sock *ys, struct nlmsghdr *req_nlh, + struct ynl_req_state *yrs); +int ynl_exec_dump(struct ynl_sock *ys, struct nlmsghdr *req_nlh, + struct ynl_dump_state *yds); + +void ynl_error_unknown_notification(struct ynl_sock *ys, __u8 cmd); +int ynl_error_parse(struct ynl_parse_arg *yarg, const char *msg); + +/* Netlink message handling helpers */ + +#define YNL_MSG_OVERFLOW 1 + +static inline struct nlmsghdr *ynl_nlmsg_put_header(void *buf) +{ + struct nlmsghdr *nlh = (struct nlmsghdr *)buf; + + memset(nlh, 0, sizeof(*nlh)); + nlh->nlmsg_len = NLMSG_HDRLEN; + + return nlh; +} + +static inline unsigned int ynl_nlmsg_data_len(const struct nlmsghdr *nlh) +{ + return nlh->nlmsg_len - NLMSG_HDRLEN; +} + +static inline void *ynl_nlmsg_data(const struct nlmsghdr *nlh) +{ + return (unsigned char *)nlh + NLMSG_HDRLEN; +} + +static inline void * +ynl_nlmsg_data_offset(const struct nlmsghdr *nlh, unsigned int offset) +{ + return (unsigned char *)nlh + NLMSG_HDRLEN + offset; +} + +static inline void *ynl_nlmsg_end_addr(const struct nlmsghdr *nlh) +{ + return (char *)nlh + nlh->nlmsg_len; +} + +static inline void * +ynl_nlmsg_put_extra_header(struct nlmsghdr *nlh, unsigned int size) +{ + void *tail = ynl_nlmsg_end_addr(nlh); + + nlh->nlmsg_len += NLMSG_ALIGN(size); + return tail; +} + +/* Netlink attribute helpers */ + +static inline unsigned int ynl_attr_type(const struct nlattr *attr) +{ + return attr->nla_type & NLA_TYPE_MASK; +} + +static inline unsigned int ynl_attr_data_len(const struct nlattr *attr) +{ + return attr->nla_len - NLA_HDRLEN; +} + +static inline void *ynl_attr_data(const struct nlattr *attr) +{ + return (unsigned char *)attr + NLA_HDRLEN; +} + +static inline void *ynl_attr_data_end(const struct nlattr *attr) +{ + return (char *)ynl_attr_data(attr) + ynl_attr_data_len(attr); +} + +#define ynl_attr_for_each(attr, nlh, fixed_hdr_sz) \ + for ((attr) = ynl_attr_first(nlh, (nlh)->nlmsg_len, \ + NLMSG_HDRLEN + fixed_hdr_sz); attr; \ + (attr) = ynl_attr_next(ynl_nlmsg_end_addr(nlh), attr)) + +#define ynl_attr_for_each_nested_off(attr, outer, offset) \ + for ((attr) = ynl_attr_first(outer, outer->nla_len, \ + sizeof(struct nlattr) + offset); \ + attr; \ + (attr) = ynl_attr_next(ynl_attr_data_end(outer), attr)) + +#define ynl_attr_for_each_nested(attr, outer) \ + ynl_attr_for_each_nested_off(attr, outer, 0) + +#define ynl_attr_for_each_payload(start, len, attr) \ + for ((attr) = ynl_attr_first(start, len, 0); attr; \ + (attr) = ynl_attr_next(start + len, attr)) + +static inline struct nlattr * +ynl_attr_if_good(const void *end, struct nlattr *attr) +{ + if (attr + 1 > (const struct nlattr *)end) + return NULL; + if (ynl_attr_data_end(attr) > end) + return NULL; + return attr; +} + +static inline struct nlattr * +ynl_attr_next(const void *end, const struct nlattr *prev) +{ + struct nlattr *attr; + + attr = (struct nlattr *)((char *)prev + NLA_ALIGN(prev->nla_len)); + return ynl_attr_if_good(end, attr); +} + +static inline struct nlattr * +ynl_attr_first(const void *start, size_t len, size_t skip) +{ + struct nlattr *attr; + + attr = (struct nlattr *)((char *)start + NLMSG_ALIGN(skip)); + return ynl_attr_if_good((char *)start + len, attr); +} + +static inline bool +__ynl_attr_put_overflow(struct nlmsghdr *nlh, size_t size) +{ + bool o; + + /* ynl_msg_start() stashed buffer length in nlmsg_pid. */ + o = nlh->nlmsg_len + NLA_HDRLEN + NLMSG_ALIGN(size) > nlh->nlmsg_pid; + if (o) + /* YNL_MSG_OVERFLOW is < NLMSG_HDRLEN, all subsequent checks + * are guaranteed to fail. + */ + nlh->nlmsg_pid = YNL_MSG_OVERFLOW; + return o; +} + +static inline struct nlattr * +ynl_attr_nest_start(struct nlmsghdr *nlh, unsigned int attr_type) +{ + struct nlattr *attr; + + if (__ynl_attr_put_overflow(nlh, 0)) + return (struct nlattr *)ynl_nlmsg_end_addr(nlh) - 1; + + attr = (struct nlattr *)ynl_nlmsg_end_addr(nlh); + attr->nla_type = attr_type | NLA_F_NESTED; + nlh->nlmsg_len += NLA_HDRLEN; + + return attr; +} + +static inline void +ynl_attr_nest_end(struct nlmsghdr *nlh, struct nlattr *attr) +{ + attr->nla_len = (char *)ynl_nlmsg_end_addr(nlh) - (char *)attr; +} + +static inline void +ynl_attr_put(struct nlmsghdr *nlh, unsigned int attr_type, + const void *value, size_t size) +{ + struct nlattr *attr; + + if (__ynl_attr_put_overflow(nlh, size)) + return; + + attr = (struct nlattr *)ynl_nlmsg_end_addr(nlh); + attr->nla_type = attr_type; + attr->nla_len = NLA_HDRLEN + size; + + memcpy(ynl_attr_data(attr), value, size); + + nlh->nlmsg_len += NLMSG_ALIGN(attr->nla_len); +} + +static inline void +ynl_attr_put_str(struct nlmsghdr *nlh, unsigned int attr_type, const char *str) +{ + struct nlattr *attr; + size_t len; + + len = strlen(str) + 1; + if (__ynl_attr_put_overflow(nlh, len)) + return; + + attr = (struct nlattr *)ynl_nlmsg_end_addr(nlh); + attr->nla_type = attr_type; + + strcpy((char *)ynl_attr_data(attr), str); + attr->nla_len = NLA_HDRLEN + len; + + nlh->nlmsg_len += NLMSG_ALIGN(attr->nla_len); +} + +static inline const char *ynl_attr_get_str(const struct nlattr *attr) +{ + return (const char *)ynl_attr_data(attr); +} + +static inline __s8 ynl_attr_get_s8(const struct nlattr *attr) +{ + return *(__s8 *)ynl_attr_data(attr); +} + +static inline __s16 ynl_attr_get_s16(const struct nlattr *attr) +{ + return *(__s16 *)ynl_attr_data(attr); +} + +static inline __s32 ynl_attr_get_s32(const struct nlattr *attr) +{ + return *(__s32 *)ynl_attr_data(attr); +} + +static inline __s64 ynl_attr_get_s64(const struct nlattr *attr) +{ + __s64 tmp; + + memcpy(&tmp, (unsigned char *)(attr + 1), sizeof(tmp)); + return tmp; +} + +static inline __u8 ynl_attr_get_u8(const struct nlattr *attr) +{ + return *(__u8 *)ynl_attr_data(attr); +} + +static inline __u16 ynl_attr_get_u16(const struct nlattr *attr) +{ + return *(__u16 *)ynl_attr_data(attr); +} + +static inline __u32 ynl_attr_get_u32(const struct nlattr *attr) +{ + return *(__u32 *)ynl_attr_data(attr); +} + +static inline __u64 ynl_attr_get_u64(const struct nlattr *attr) +{ + __u64 tmp; + + memcpy(&tmp, (unsigned char *)(attr + 1), sizeof(tmp)); + return tmp; +} + +static inline void +ynl_attr_put_s8(struct nlmsghdr *nlh, unsigned int attr_type, __s8 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline void +ynl_attr_put_s16(struct nlmsghdr *nlh, unsigned int attr_type, __s16 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline void +ynl_attr_put_s32(struct nlmsghdr *nlh, unsigned int attr_type, __s32 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline void +ynl_attr_put_s64(struct nlmsghdr *nlh, unsigned int attr_type, __s64 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline void +ynl_attr_put_u8(struct nlmsghdr *nlh, unsigned int attr_type, __u8 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline void +ynl_attr_put_u16(struct nlmsghdr *nlh, unsigned int attr_type, __u16 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline void +ynl_attr_put_u32(struct nlmsghdr *nlh, unsigned int attr_type, __u32 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline void +ynl_attr_put_u64(struct nlmsghdr *nlh, unsigned int attr_type, __u64 value) +{ + ynl_attr_put(nlh, attr_type, &value, sizeof(value)); +} + +static inline __u64 ynl_attr_get_uint(const struct nlattr *attr) +{ + switch (ynl_attr_data_len(attr)) { + case 4: + return ynl_attr_get_u32(attr); + case 8: + return ynl_attr_get_u64(attr); + default: + return 0; + } +} + +static inline __s64 ynl_attr_get_sint(const struct nlattr *attr) +{ + switch (ynl_attr_data_len(attr)) { + case 4: + return ynl_attr_get_s32(attr); + case 8: + return ynl_attr_get_s64(attr); + default: + return 0; + } +} + +static inline void +ynl_attr_put_uint(struct nlmsghdr *nlh, __u16 type, __u64 data) +{ + if ((__u32)data == (__u64)data) + ynl_attr_put_u32(nlh, type, data); + else + ynl_attr_put_u64(nlh, type, data); +} + +static inline void +ynl_attr_put_sint(struct nlmsghdr *nlh, __u16 type, __s64 data) +{ + if ((__s32)data == (__s64)data) + ynl_attr_put_s32(nlh, type, data); + else + ynl_attr_put_s64(nlh, type, data); +} + +int __ynl_attr_validate(struct ynl_parse_arg *yarg, const struct nlattr *attr, + unsigned int type); + +static inline int ynl_attr_validate(struct ynl_parse_arg *yarg, + const struct nlattr *attr) +{ + return __ynl_attr_validate(yarg, attr, ynl_attr_type(attr)); +} +#endif diff --git a/tools/net/ynl/lib/ynl.c b/tools/net/ynl/lib/ynl.c new file mode 100644 index 000000000..af101544c --- /dev/null +++ b/tools/net/ynl/lib/ynl.c @@ -0,0 +1,1071 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +#include <errno.h> +#include <poll.h> +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <linux/types.h> +#include <linux/genetlink.h> +#include <sys/socket.h> + +#include "ynl.h" + +#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof(*arr)) + +#define __yerr_msg(yse, _msg...) \ + ({ \ + struct ynl_error *_yse = (yse); \ + \ + if (_yse) { \ + snprintf(_yse->msg, sizeof(_yse->msg) - 1, _msg); \ + _yse->msg[sizeof(_yse->msg) - 1] = 0; \ + } \ + }) + +#define __yerr_code(yse, _code...) \ + ({ \ + struct ynl_error *_yse = (yse); \ + \ + if (_yse) { \ + _yse->code = _code; \ + } \ + }) + +#define __yerr(yse, _code, _msg...) \ + ({ \ + __yerr_msg(yse, _msg); \ + __yerr_code(yse, _code); \ + }) + +#define __perr(yse, _msg) __yerr(yse, errno, _msg) + +#define yerr_msg(_ys, _msg...) __yerr_msg(&(_ys)->err, _msg) +#define yerr(_ys, _code, _msg...) __yerr(&(_ys)->err, _code, _msg) +#define perr(_ys, _msg) __yerr(&(_ys)->err, errno, _msg) + +/* -- Netlink boiler plate */ +static bool +ynl_err_walk_is_sel(const struct ynl_policy_nest *policy, + const struct nlattr *attr) +{ + unsigned int type = ynl_attr_type(attr); + + return policy && type <= policy->max_attr && + policy->table[type].is_selector; +} + +static const struct ynl_policy_nest * +ynl_err_walk_sel_policy(const struct ynl_policy_attr *policy_attr, + const struct nlattr *selector) +{ + const struct ynl_policy_nest *policy = policy_attr->nest; + const char *sel; + unsigned int i; + + if (!policy_attr->is_submsg) + return policy; + + sel = ynl_attr_get_str(selector); + for (i = 0; i <= policy->max_attr; i++) { + if (!strcmp(sel, policy->table[i].name)) + return policy->table[i].nest; + } + + return NULL; +} + +static int +ynl_err_walk_report_one(const struct ynl_policy_nest *policy, + const struct nlattr *selector, unsigned int type, + char *str, int str_sz, int *n) +{ + if (!policy) { + if (*n < str_sz) + *n += snprintf(str, str_sz, "!policy"); + return 1; + } + + if (type > policy->max_attr) { + if (*n < str_sz) + *n += snprintf(str, str_sz, "!oob"); + return 1; + } + + if (!policy->table[type].name) { + if (*n < str_sz) + *n += snprintf(str, str_sz, "!name"); + return 1; + } + + if (*n < str_sz) { + int sz; + + sz = snprintf(str, str_sz - *n, + ".%s", policy->table[type].name); + *n += sz; + str += sz; + } + + if (policy->table[type].is_submsg) { + if (!selector) { + if (*n < str_sz) + *n += snprintf(str, str_sz, "(!selector)"); + return 1; + } + + if (ynl_attr_type(selector) != + policy->table[type].selector_type) { + if (*n < str_sz) + *n += snprintf(str, str_sz, "(!=selector)"); + return 1; + } + + if (*n < str_sz) + *n += snprintf(str, str_sz - *n, "(%s)", + ynl_attr_get_str(selector)); + } + + return 0; +} + +static int +ynl_err_walk(struct ynl_sock *ys, void *start, void *end, unsigned int off, + const struct ynl_policy_nest *policy, char *str, int str_sz, + const struct ynl_policy_nest **nest_pol) +{ + const struct ynl_policy_nest *next_pol; + const struct nlattr *selector = NULL; + unsigned int astart_off, aend_off; + const struct nlattr *attr; + unsigned int data_len; + unsigned int type; + bool found = false; + int n = 0; + + if (!policy) { + if (n < str_sz) + n += snprintf(str, str_sz, "!policy"); + return n; + } + + data_len = end - start; + + ynl_attr_for_each_payload(start, data_len, attr) { + astart_off = (char *)attr - (char *)start; + aend_off = (char *)ynl_attr_data_end(attr) - (char *)start; + + if (ynl_err_walk_is_sel(policy, attr)) + selector = attr; + + if (aend_off <= off) + continue; + + found = true; + break; + } + if (!found) + return 0; + + off -= astart_off; + + type = ynl_attr_type(attr); + + if (ynl_err_walk_report_one(policy, selector, type, str, str_sz, &n)) + return n; + + next_pol = ynl_err_walk_sel_policy(&policy->table[type], selector); + if (!next_pol) + return n; + + if (!off) { + if (nest_pol) + *nest_pol = next_pol; + return n; + } + + if (!next_pol) { + if (n < str_sz) + n += snprintf(str, str_sz, "!nest"); + return n; + } + + off -= sizeof(struct nlattr); + start = ynl_attr_data(attr); + end = start + ynl_attr_data_len(attr); + + return n + ynl_err_walk(ys, start, end, off, next_pol, + &str[n], str_sz - n, nest_pol); +} + +#define NLMSGERR_ATTR_MISS_TYPE (NLMSGERR_ATTR_POLICY + 1) +#define NLMSGERR_ATTR_MISS_NEST (NLMSGERR_ATTR_POLICY + 2) +#define NLMSGERR_ATTR_MAX (NLMSGERR_ATTR_MAX + 2) + +static int +ynl_ext_ack_check(struct ynl_sock *ys, const struct nlmsghdr *nlh, + unsigned int hlen) +{ + const struct nlattr *tb[NLMSGERR_ATTR_MAX + 1] = {}; + char miss_attr[sizeof(ys->err.msg)]; + char bad_attr[sizeof(ys->err.msg)]; + const struct nlattr *attr; + const char *str = NULL; + + if (!(nlh->nlmsg_flags & NLM_F_ACK_TLVS)) { + yerr_msg(ys, "%s", strerror(ys->err.code)); + return YNL_PARSE_CB_OK; + } + + ynl_attr_for_each(attr, nlh, hlen) { + unsigned int len, type; + + len = ynl_attr_data_len(attr); + type = ynl_attr_type(attr); + + if (type > NLMSGERR_ATTR_MAX) + continue; + + tb[type] = attr; + + switch (type) { + case NLMSGERR_ATTR_OFFS: + case NLMSGERR_ATTR_MISS_TYPE: + case NLMSGERR_ATTR_MISS_NEST: + if (len != sizeof(__u32)) + return YNL_PARSE_CB_ERROR; + break; + case NLMSGERR_ATTR_MSG: + str = ynl_attr_get_str(attr); + if (str[len - 1]) + return YNL_PARSE_CB_ERROR; + break; + default: + break; + } + } + + bad_attr[0] = '\0'; + miss_attr[0] = '\0'; + + if (tb[NLMSGERR_ATTR_OFFS]) { + unsigned int n, off; + void *start, *end; + + ys->err.attr_offs = ynl_attr_get_u32(tb[NLMSGERR_ATTR_OFFS]); + + n = snprintf(bad_attr, sizeof(bad_attr), "%sbad attribute: ", + str ? " (" : ""); + + start = ynl_nlmsg_data_offset(ys->nlh, ys->req_hdr_len); + end = ynl_nlmsg_end_addr(ys->nlh); + + off = ys->err.attr_offs; + off -= sizeof(struct nlmsghdr); + off -= ys->req_hdr_len; + + n += ynl_err_walk(ys, start, end, off, ys->req_policy, + &bad_attr[n], sizeof(bad_attr) - n, NULL); + + if (n >= sizeof(bad_attr)) + n = sizeof(bad_attr) - 1; + bad_attr[n] = '\0'; + } + if (tb[NLMSGERR_ATTR_MISS_TYPE]) { + const struct ynl_policy_nest *nest_pol = NULL; + unsigned int n, off, type; + void *start, *end; + int n2; + + type = ynl_attr_get_u32(tb[NLMSGERR_ATTR_MISS_TYPE]); + + n = snprintf(miss_attr, sizeof(miss_attr), "%smissing attribute: ", + bad_attr[0] ? ", " : (str ? " (" : "")); + + start = ynl_nlmsg_data_offset(ys->nlh, ys->req_hdr_len); + end = ynl_nlmsg_end_addr(ys->nlh); + + nest_pol = ys->req_policy; + if (tb[NLMSGERR_ATTR_MISS_NEST]) { + off = ynl_attr_get_u32(tb[NLMSGERR_ATTR_MISS_NEST]); + off -= sizeof(struct nlmsghdr); + off -= ys->req_hdr_len; + + n += ynl_err_walk(ys, start, end, off, ys->req_policy, + &miss_attr[n], sizeof(miss_attr) - n, + &nest_pol); + } + + n2 = 0; + ynl_err_walk_report_one(nest_pol, NULL, type, &miss_attr[n], + sizeof(miss_attr) - n, &n2); + n += n2; + + if (n >= sizeof(miss_attr)) + n = sizeof(miss_attr) - 1; + miss_attr[n] = '\0'; + } + + /* Implicitly depend on ys->err.code already set */ + if (str) + yerr_msg(ys, "Kernel %s: '%s'%s%s%s", + ys->err.code ? "error" : "warning", + str, bad_attr, miss_attr, + bad_attr[0] || miss_attr[0] ? ")" : ""); + else if (bad_attr[0] || miss_attr[0]) + yerr_msg(ys, "Kernel %s: %s%s", + ys->err.code ? "error" : "warning", + bad_attr, miss_attr); + else + yerr_msg(ys, "%s", strerror(ys->err.code)); + + return YNL_PARSE_CB_OK; +} + +static int +ynl_cb_error(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg) +{ + const struct nlmsgerr *err = ynl_nlmsg_data(nlh); + unsigned int hlen; + int code; + + code = err->error >= 0 ? err->error : -err->error; + yarg->ys->err.code = code; + errno = code; + + hlen = sizeof(*err); + if (!(nlh->nlmsg_flags & NLM_F_CAPPED)) + hlen += ynl_nlmsg_data_len(&err->msg); + + ynl_ext_ack_check(yarg->ys, nlh, hlen); + + return code ? YNL_PARSE_CB_ERROR : YNL_PARSE_CB_STOP; +} + +static int ynl_cb_done(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg) +{ + int err; + + err = *(int *)NLMSG_DATA(nlh); + if (err < 0) { + yarg->ys->err.code = -err; + errno = -err; + + ynl_ext_ack_check(yarg->ys, nlh, sizeof(int)); + + return YNL_PARSE_CB_ERROR; + } + return YNL_PARSE_CB_STOP; +} + +/* Attribute validation */ + +int __ynl_attr_validate(struct ynl_parse_arg *yarg, const struct nlattr *attr, + unsigned int type) +{ + const struct ynl_policy_attr *policy; + unsigned char *data; + unsigned int len; + + data = ynl_attr_data(attr); + len = ynl_attr_data_len(attr); + if (type > yarg->rsp_policy->max_attr) { + yerr(yarg->ys, YNL_ERROR_INTERNAL, + "Internal error, validating unknown attribute"); + return -1; + } + + policy = &yarg->rsp_policy->table[type]; + + switch (policy->type) { + case YNL_PT_REJECT: + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Rejected attribute (%s)", policy->name); + return -1; + case YNL_PT_IGNORE: + break; + case YNL_PT_U8: + if (len == sizeof(__u8)) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (u8 %s)", policy->name); + return -1; + case YNL_PT_U16: + if (len == sizeof(__u16)) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (u16 %s)", policy->name); + return -1; + case YNL_PT_U32: + if (len == sizeof(__u32)) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (u32 %s)", policy->name); + return -1; + case YNL_PT_U64: + if (len == sizeof(__u64)) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (u64 %s)", policy->name); + return -1; + case YNL_PT_UINT: + if (len == sizeof(__u32) || len == sizeof(__u64)) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (uint %s)", policy->name); + return -1; + case YNL_PT_FLAG: + /* Let flags grow into real attrs, why not.. */ + break; + case YNL_PT_NEST: + if (!len || len >= sizeof(*attr)) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (nest %s)", policy->name); + return -1; + case YNL_PT_BINARY: + if (!policy->len || len == policy->len) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (binary %s)", policy->name); + return -1; + case YNL_PT_NUL_STR: + if (len && (!policy->len || len <= policy->len) && !data[len - 1]) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (string %s)", policy->name); + return -1; + case YNL_PT_BITFIELD32: + if (len == sizeof(struct nla_bitfield32)) + break; + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (bitfield32 %s)", policy->name); + return -1; + default: + yerr(yarg->ys, YNL_ERROR_ATTR_INVALID, + "Invalid attribute (unknown %s)", policy->name); + return -1; + } + + return 0; +} + +int ynl_submsg_failed(struct ynl_parse_arg *yarg, const char *field_name, + const char *sel_name) +{ + yerr(yarg->ys, YNL_ERROR_SUBMSG_KEY, + "Parsing error: Sub-message key not set (msg %s, key %s)", + field_name, sel_name); + return YNL_PARSE_CB_ERROR; +} + +/* Generic code */ + +static void ynl_err_reset(struct ynl_sock *ys) +{ + ys->err.code = 0; + ys->err.attr_offs = 0; + ys->err.msg[0] = 0; +} + +struct nlmsghdr *ynl_msg_start(struct ynl_sock *ys, __u32 id, __u16 flags) +{ + struct nlmsghdr *nlh; + + ynl_err_reset(ys); + + nlh = ys->nlh = ynl_nlmsg_put_header(ys->tx_buf); + nlh->nlmsg_type = id; + nlh->nlmsg_flags = flags; + nlh->nlmsg_seq = ++ys->seq; + + /* This is a local YNL hack for length checking, we put the buffer + * length in nlmsg_pid, since messages sent to the kernel always use + * PID 0. Message needs to be terminated with ynl_msg_end(). + */ + nlh->nlmsg_pid = YNL_SOCKET_BUFFER_SIZE; + + return nlh; +} + +static int ynl_msg_end(struct ynl_sock *ys, struct nlmsghdr *nlh) +{ + /* We stash buffer length in nlmsg_pid. */ + if (nlh->nlmsg_pid == 0) { + yerr(ys, YNL_ERROR_INPUT_INVALID, + "Unknown input buffer length"); + return -EINVAL; + } + if (nlh->nlmsg_pid == YNL_MSG_OVERFLOW) { + yerr(ys, YNL_ERROR_INPUT_TOO_BIG, + "Constructed message longer than internal buffer"); + return -EMSGSIZE; + } + + nlh->nlmsg_pid = 0; + return 0; +} + +struct nlmsghdr * +ynl_gemsg_start(struct ynl_sock *ys, __u32 id, __u16 flags, + __u8 cmd, __u8 version) +{ + struct genlmsghdr gehdr; + struct nlmsghdr *nlh; + void *data; + + nlh = ynl_msg_start(ys, id, flags); + + memset(&gehdr, 0, sizeof(gehdr)); + gehdr.cmd = cmd; + gehdr.version = version; + + data = ynl_nlmsg_put_extra_header(nlh, sizeof(gehdr)); + memcpy(data, &gehdr, sizeof(gehdr)); + + return nlh; +} + +struct nlmsghdr *ynl_msg_start_req(struct ynl_sock *ys, __u32 id, __u16 flags) +{ + return ynl_msg_start(ys, id, NLM_F_REQUEST | NLM_F_ACK | flags); +} + +struct nlmsghdr *ynl_msg_start_dump(struct ynl_sock *ys, __u32 id) +{ + return ynl_msg_start(ys, id, NLM_F_REQUEST | NLM_F_ACK | NLM_F_DUMP); +} + +struct nlmsghdr * +ynl_gemsg_start_req(struct ynl_sock *ys, __u32 id, __u8 cmd, __u8 version) +{ + return ynl_gemsg_start(ys, id, NLM_F_REQUEST | NLM_F_ACK, cmd, version); +} + +struct nlmsghdr * +ynl_gemsg_start_dump(struct ynl_sock *ys, __u32 id, __u8 cmd, __u8 version) +{ + return ynl_gemsg_start(ys, id, NLM_F_REQUEST | NLM_F_ACK | NLM_F_DUMP, + cmd, version); +} + +static int ynl_cb_null(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg) +{ + yerr(yarg->ys, YNL_ERROR_UNEXPECT_MSG, + "Received a message when none were expected"); + + return YNL_PARSE_CB_ERROR; +} + +static int +__ynl_sock_read_msgs(struct ynl_parse_arg *yarg, ynl_parse_cb_t cb, int flags) +{ + struct ynl_sock *ys = yarg->ys; + const struct nlmsghdr *nlh; + ssize_t len, rem; + int ret; + + len = recv(ys->socket, ys->rx_buf, YNL_SOCKET_BUFFER_SIZE, flags); + if (len < 0) { + if (flags & MSG_DONTWAIT && errno == EAGAIN) + return YNL_PARSE_CB_STOP; + return len; + } + + ret = YNL_PARSE_CB_STOP; + for (rem = len; rem > 0; NLMSG_NEXT(nlh, rem)) { + nlh = (struct nlmsghdr *)&ys->rx_buf[len - rem]; + if (!NLMSG_OK(nlh, rem)) { + yerr(yarg->ys, YNL_ERROR_INV_RESP, + "Invalid message or trailing data in the response."); + return YNL_PARSE_CB_ERROR; + } + + if (nlh->nlmsg_flags & NLM_F_DUMP_INTR) { + /* TODO: handle this better */ + yerr(yarg->ys, YNL_ERROR_DUMP_INTER, + "Dump interrupted / inconsistent, please retry."); + return YNL_PARSE_CB_ERROR; + } + + switch (nlh->nlmsg_type) { + case 0: + yerr(yarg->ys, YNL_ERROR_INV_RESP, + "Invalid message type in the response."); + return YNL_PARSE_CB_ERROR; + case NLMSG_NOOP: + case NLMSG_OVERRUN ... NLMSG_MIN_TYPE - 1: + ret = YNL_PARSE_CB_OK; + break; + case NLMSG_ERROR: + ret = ynl_cb_error(nlh, yarg); + break; + case NLMSG_DONE: + ret = ynl_cb_done(nlh, yarg); + break; + default: + ret = cb(nlh, yarg); + break; + } + } + + return ret; +} + +static int ynl_sock_read_msgs(struct ynl_parse_arg *yarg, ynl_parse_cb_t cb) +{ + return __ynl_sock_read_msgs(yarg, cb, 0); +} + +static int ynl_recv_ack(struct ynl_sock *ys, int ret) +{ + struct ynl_parse_arg yarg = { .ys = ys, }; + + if (!ret) { + yerr(ys, YNL_ERROR_EXPECT_ACK, + "Expecting an ACK but nothing received"); + return -1; + } + + return ynl_sock_read_msgs(&yarg, ynl_cb_null); +} + +/* Init/fini and genetlink boiler plate */ +static int +ynl_get_family_info_mcast(struct ynl_sock *ys, const struct nlattr *mcasts) +{ + const struct nlattr *entry, *attr; + unsigned int i; + + ynl_attr_for_each_nested(attr, mcasts) + ys->n_mcast_groups++; + + if (!ys->n_mcast_groups) + return 0; + + ys->mcast_groups = calloc(ys->n_mcast_groups, + sizeof(*ys->mcast_groups)); + if (!ys->mcast_groups) + return YNL_PARSE_CB_ERROR; + + i = 0; + ynl_attr_for_each_nested(entry, mcasts) { + ynl_attr_for_each_nested(attr, entry) { + if (ynl_attr_type(attr) == CTRL_ATTR_MCAST_GRP_ID) + ys->mcast_groups[i].id = ynl_attr_get_u32(attr); + if (ynl_attr_type(attr) == CTRL_ATTR_MCAST_GRP_NAME) { + strncpy(ys->mcast_groups[i].name, + ynl_attr_get_str(attr), + GENL_NAMSIZ - 1); + ys->mcast_groups[i].name[GENL_NAMSIZ - 1] = 0; + } + } + i++; + } + + return 0; +} + +static int +ynl_get_family_info_cb(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg) +{ + struct ynl_sock *ys = yarg->ys; + const struct nlattr *attr; + bool found_id = true; + + ynl_attr_for_each(attr, nlh, sizeof(struct genlmsghdr)) { + if (ynl_attr_type(attr) == CTRL_ATTR_MCAST_GROUPS) + if (ynl_get_family_info_mcast(ys, attr)) + return YNL_PARSE_CB_ERROR; + + if (ynl_attr_type(attr) != CTRL_ATTR_FAMILY_ID) + continue; + + if (ynl_attr_data_len(attr) != sizeof(__u16)) { + yerr(ys, YNL_ERROR_ATTR_INVALID, "Invalid family ID"); + return YNL_PARSE_CB_ERROR; + } + + ys->family_id = ynl_attr_get_u16(attr); + found_id = true; + } + + if (!found_id) { + yerr(ys, YNL_ERROR_ATTR_MISSING, "Family ID missing"); + return YNL_PARSE_CB_ERROR; + } + return YNL_PARSE_CB_OK; +} + +static int ynl_sock_read_family(struct ynl_sock *ys, const char *family_name) +{ + struct ynl_parse_arg yarg = { .ys = ys, }; + struct nlmsghdr *nlh; + int err; + + nlh = ynl_gemsg_start_req(ys, GENL_ID_CTRL, CTRL_CMD_GETFAMILY, 1); + ynl_attr_put_str(nlh, CTRL_ATTR_FAMILY_NAME, family_name); + + err = ynl_msg_end(ys, nlh); + if (err < 0) + return err; + + err = send(ys->socket, nlh, nlh->nlmsg_len, 0); + if (err < 0) { + perr(ys, "failed to request socket family info"); + return err; + } + + err = ynl_sock_read_msgs(&yarg, ynl_get_family_info_cb); + if (err < 0) { + free(ys->mcast_groups); + perr(ys, "failed to receive the socket family info - no such family?"); + return err; + } + + err = ynl_recv_ack(ys, err); + if (err < 0) { + free(ys->mcast_groups); + return err; + } + + return 0; +} + +struct ynl_sock * +ynl_sock_create(const struct ynl_family *yf, struct ynl_error *yse) +{ + struct sockaddr_nl addr; + struct ynl_sock *ys; + socklen_t addrlen; + int sock_type; + int one = 1; + + ys = malloc(sizeof(*ys) + 2 * YNL_SOCKET_BUFFER_SIZE); + if (!ys) + return NULL; + memset(ys, 0, sizeof(*ys)); + + ys->family = yf; + ys->tx_buf = &ys->raw_buf[0]; + ys->rx_buf = &ys->raw_buf[YNL_SOCKET_BUFFER_SIZE]; + ys->ntf_last_next = &ys->ntf_first; + + sock_type = yf->is_classic ? yf->classic_id : NETLINK_GENERIC; + + ys->socket = socket(AF_NETLINK, SOCK_RAW, sock_type); + if (ys->socket < 0) { + __perr(yse, "failed to create a netlink socket"); + goto err_free_sock; + } + + if (setsockopt(ys->socket, SOL_NETLINK, NETLINK_CAP_ACK, + &one, sizeof(one))) { + __perr(yse, "failed to enable netlink ACK"); + goto err_close_sock; + } + if (setsockopt(ys->socket, SOL_NETLINK, NETLINK_EXT_ACK, + &one, sizeof(one))) { + __perr(yse, "failed to enable netlink ext ACK"); + goto err_close_sock; + } + + memset(&addr, 0, sizeof(addr)); + addr.nl_family = AF_NETLINK; + if (bind(ys->socket, (struct sockaddr *)&addr, sizeof(addr)) < 0) { + __perr(yse, "unable to bind to a socket address"); + goto err_close_sock; + } + + memset(&addr, 0, sizeof(addr)); + addrlen = sizeof(addr); + if (getsockname(ys->socket, (struct sockaddr *)&addr, &addrlen) < 0) { + __perr(yse, "unable to read socket address"); + goto err_close_sock; + } + ys->portid = addr.nl_pid; + ys->seq = random(); + + if (yf->is_classic) { + ys->family_id = yf->classic_id; + } else if (ynl_sock_read_family(ys, yf->name)) { + if (yse) + memcpy(yse, &ys->err, sizeof(*yse)); + goto err_close_sock; + } + + return ys; + +err_close_sock: + close(ys->socket); +err_free_sock: + free(ys); + return NULL; +} + +void ynl_sock_destroy(struct ynl_sock *ys) +{ + struct ynl_ntf_base_type *ntf; + + close(ys->socket); + while ((ntf = ynl_ntf_dequeue(ys))) + ynl_ntf_free(ntf); + free(ys->mcast_groups); + free(ys); +} + +/* YNL multicast handling */ + +void ynl_ntf_free(struct ynl_ntf_base_type *ntf) +{ + ntf->free(ntf); +} + +int ynl_subscribe(struct ynl_sock *ys, const char *grp_name) +{ + unsigned int i; + int err; + + for (i = 0; i < ys->n_mcast_groups; i++) + if (!strcmp(ys->mcast_groups[i].name, grp_name)) + break; + if (i == ys->n_mcast_groups) { + yerr(ys, ENOENT, "Multicast group '%s' not found", grp_name); + return -1; + } + + err = setsockopt(ys->socket, SOL_NETLINK, NETLINK_ADD_MEMBERSHIP, + &ys->mcast_groups[i].id, + sizeof(ys->mcast_groups[i].id)); + if (err < 0) { + perr(ys, "Subscribing to multicast group failed"); + return -1; + } + + return 0; +} + +int ynl_socket_get_fd(struct ynl_sock *ys) +{ + return ys->socket; +} + +struct ynl_ntf_base_type *ynl_ntf_dequeue(struct ynl_sock *ys) +{ + struct ynl_ntf_base_type *ntf; + + if (!ynl_has_ntf(ys)) + return NULL; + + ntf = ys->ntf_first; + ys->ntf_first = ntf->next; + if (ys->ntf_last_next == &ntf->next) + ys->ntf_last_next = &ys->ntf_first; + + return ntf; +} + +static int ynl_ntf_parse(struct ynl_sock *ys, const struct nlmsghdr *nlh) +{ + struct ynl_parse_arg yarg = { .ys = ys, }; + const struct ynl_ntf_info *info; + struct ynl_ntf_base_type *rsp; + __u32 cmd; + int ret; + + if (ys->family->is_classic) { + cmd = nlh->nlmsg_type; + } else { + struct genlmsghdr *gehdr; + + gehdr = ynl_nlmsg_data(nlh); + cmd = gehdr->cmd; + } + + if (cmd >= ys->family->ntf_info_size) + return YNL_PARSE_CB_ERROR; + info = &ys->family->ntf_info[cmd]; + if (!info->cb) + return YNL_PARSE_CB_ERROR; + + rsp = calloc(1, info->alloc_sz); + if (!rsp) + return YNL_PARSE_CB_ERROR; + + rsp->free = info->free; + yarg.data = rsp->data; + yarg.rsp_policy = info->policy; + + ret = info->cb(nlh, &yarg); + if (ret <= YNL_PARSE_CB_STOP) + goto err_free; + + rsp->family = nlh->nlmsg_type; + rsp->cmd = cmd; + + *ys->ntf_last_next = rsp; + ys->ntf_last_next = &rsp->next; + + return YNL_PARSE_CB_OK; + +err_free: + info->free(rsp); + return YNL_PARSE_CB_ERROR; +} + +static int +ynl_ntf_trampoline(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg) +{ + return ynl_ntf_parse(yarg->ys, nlh); +} + +int ynl_ntf_check(struct ynl_sock *ys) +{ + struct ynl_parse_arg yarg = { .ys = ys, }; + int err; + + do { + err = __ynl_sock_read_msgs(&yarg, ynl_ntf_trampoline, + MSG_DONTWAIT); + if (err < 0) + return err; + } while (err > 0); + + return 0; +} + +/* YNL specific helpers used by the auto-generated code */ + +struct ynl_dump_list_type *YNL_LIST_END = (void *)(0xb4d123); + +void ynl_error_unknown_notification(struct ynl_sock *ys, __u8 cmd) +{ + yerr(ys, YNL_ERROR_UNKNOWN_NTF, + "Unknown notification message type '%d'", cmd); +} + +int ynl_error_parse(struct ynl_parse_arg *yarg, const char *msg) +{ + yerr(yarg->ys, YNL_ERROR_INV_RESP, "Error parsing response: %s", msg); + return YNL_PARSE_CB_ERROR; +} + +static int +ynl_check_alien(struct ynl_sock *ys, const struct nlmsghdr *nlh, __u32 rsp_cmd) +{ + if (ys->family->is_classic) { + if (nlh->nlmsg_type != rsp_cmd) + return ynl_ntf_parse(ys, nlh); + } else { + struct genlmsghdr *gehdr; + + if (ynl_nlmsg_data_len(nlh) < sizeof(*gehdr)) { + yerr(ys, YNL_ERROR_INV_RESP, + "Kernel responded with truncated message"); + return -1; + } + + gehdr = ynl_nlmsg_data(nlh); + if (gehdr->cmd != rsp_cmd) + return ynl_ntf_parse(ys, nlh); + } + + return 0; +} + +static +int ynl_req_trampoline(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg) +{ + struct ynl_req_state *yrs = (void *)yarg; + int ret; + + ret = ynl_check_alien(yrs->yarg.ys, nlh, yrs->rsp_cmd); + if (ret) + return ret < 0 ? YNL_PARSE_CB_ERROR : YNL_PARSE_CB_OK; + + return yrs->cb(nlh, &yrs->yarg); +} + +int ynl_exec(struct ynl_sock *ys, struct nlmsghdr *req_nlh, + struct ynl_req_state *yrs) +{ + int err; + + err = ynl_msg_end(ys, req_nlh); + if (err < 0) + return err; + + err = send(ys->socket, req_nlh, req_nlh->nlmsg_len, 0); + if (err < 0) + return err; + + do { + err = ynl_sock_read_msgs(&yrs->yarg, ynl_req_trampoline); + } while (err > 0); + + return err; +} + +static int +ynl_dump_trampoline(const struct nlmsghdr *nlh, struct ynl_parse_arg *data) +{ + struct ynl_dump_state *ds = (void *)data; + struct ynl_dump_list_type *obj; + struct ynl_parse_arg yarg = {}; + int ret; + + ret = ynl_check_alien(ds->yarg.ys, nlh, ds->rsp_cmd); + if (ret) + return ret < 0 ? YNL_PARSE_CB_ERROR : YNL_PARSE_CB_OK; + + obj = calloc(1, ds->alloc_sz); + if (!obj) + return YNL_PARSE_CB_ERROR; + + if (!ds->first) + ds->first = obj; + if (ds->last) + ds->last->next = obj; + ds->last = obj; + + yarg = ds->yarg; + yarg.data = &obj->data; + + return ds->cb(nlh, &yarg); +} + +static void *ynl_dump_end(struct ynl_dump_state *ds) +{ + if (!ds->first) + return YNL_LIST_END; + + ds->last->next = YNL_LIST_END; + return ds->first; +} + +int ynl_exec_dump(struct ynl_sock *ys, struct nlmsghdr *req_nlh, + struct ynl_dump_state *yds) +{ + int err; + + err = ynl_msg_end(ys, req_nlh); + if (err < 0) + return err; + + err = send(ys->socket, req_nlh, req_nlh->nlmsg_len, 0); + if (err < 0) + return err; + + do { + err = ynl_sock_read_msgs(&yds->yarg, ynl_dump_trampoline); + if (err < 0) + goto err_close_list; + } while (err > 0); + + yds->first = ynl_dump_end(yds); + return 0; + +err_close_list: + yds->first = ynl_dump_end(yds); + return -1; +} diff --git a/tools/net/ynl/lib/ynl.h b/tools/net/ynl/lib/ynl.h new file mode 100644 index 000000000..db7c0591a --- /dev/null +++ b/tools/net/ynl/lib/ynl.h @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +#ifndef __YNL_C_H +#define __YNL_C_H 1 + +#include <stdbool.h> +#include <stddef.h> +#include <linux/genetlink.h> +#include <linux/types.h> + +#include "ynl-priv.h" + +enum ynl_error_code { + YNL_ERROR_NONE = 0, + __YNL_ERRNO_END = 4096, + YNL_ERROR_INTERNAL, + YNL_ERROR_DUMP_INTER, + YNL_ERROR_EXPECT_ACK, + YNL_ERROR_EXPECT_MSG, + YNL_ERROR_UNEXPECT_MSG, + YNL_ERROR_ATTR_MISSING, + YNL_ERROR_ATTR_INVALID, + YNL_ERROR_UNKNOWN_NTF, + YNL_ERROR_INV_RESP, + YNL_ERROR_INPUT_INVALID, + YNL_ERROR_INPUT_TOO_BIG, + YNL_ERROR_SUBMSG_KEY, +}; + +/** + * struct ynl_error - error encountered by YNL + * @code: errno (low values) or YNL error code (enum ynl_error_code) + * @attr_offs: offset of bad attribute (for very advanced users) + * @msg: error message + * + * Error information for when YNL operations fail. + * Users should interact with the err member of struct ynl_sock directly. + * The main exception to that rule is ynl_sock_create(). + */ +struct ynl_error { + enum ynl_error_code code; + unsigned int attr_offs; + char msg[512]; +}; + +/** + * struct ynl_family - YNL family info + * Family description generated by codegen. Pass to ynl_sock_create(). + */ +struct ynl_family { +/* private: */ + const char *name; + size_t hdr_len; + bool is_classic; + __u16 classic_id; + const struct ynl_ntf_info *ntf_info; + unsigned int ntf_info_size; +}; + +/** + * struct ynl_sock - YNL wrapped netlink socket + * @err: YNL error descriptor, cleared on every request. + */ +struct ynl_sock { + struct ynl_error err; + +/* private: */ + const struct ynl_family *family; + int socket; + __u32 seq; + __u32 portid; + __u16 family_id; + + unsigned int n_mcast_groups; + struct { + unsigned int id; + char name[GENL_NAMSIZ]; + } *mcast_groups; + + struct ynl_ntf_base_type *ntf_first; + struct ynl_ntf_base_type **ntf_last_next; + + struct nlmsghdr *nlh; + const struct ynl_policy_nest *req_policy; + size_t req_hdr_len; + unsigned char *tx_buf; + unsigned char *rx_buf; + unsigned char raw_buf[]; +}; + +/** + * struct ynl_string - parsed individual string + * @len: length of the string (excluding terminating character) + * @str: value of the string + * + * Parsed and nul-terminated string. This struct is only used for arrays of + * strings. Non-array string members are placed directly in respective types. + */ +struct ynl_string { + unsigned int len; + char str[]; +}; + +struct ynl_sock * +ynl_sock_create(const struct ynl_family *yf, struct ynl_error *e); +void ynl_sock_destroy(struct ynl_sock *ys); + +#define ynl_dump_foreach(dump, iter) \ + for (typeof(dump->obj) *iter = &dump->obj; \ + !ynl_dump_obj_is_last(iter); \ + iter = ynl_dump_obj_next(iter)) + +/** + * ynl_dump_empty() - does the dump have no entries + * @dump: pointer to the dump list, as returned by a dump call + * + * Check if the dump is empty, i.e. contains no objects. + * Dump calls return NULL on error, and terminator element if empty. + */ +static inline bool ynl_dump_empty(void *dump) +{ + return dump == (void *)YNL_LIST_END; +} + +int ynl_subscribe(struct ynl_sock *ys, const char *grp_name); +int ynl_socket_get_fd(struct ynl_sock *ys); +int ynl_ntf_check(struct ynl_sock *ys); + +/** + * ynl_has_ntf() - check if socket has *parsed* notifications + * @ys: active YNL socket + * + * Note that this does not take into account notifications sitting + * in netlink socket, just the notifications which have already been + * read and parsed (e.g. during a ynl_ntf_check() call). + */ +static inline bool ynl_has_ntf(struct ynl_sock *ys) +{ + return ys->ntf_last_next != &ys->ntf_first; +} +struct ynl_ntf_base_type *ynl_ntf_dequeue(struct ynl_sock *ys); + +void ynl_ntf_free(struct ynl_ntf_base_type *ntf); +#endif diff --git a/tools/net/ynl/pyproject.toml b/tools/net/ynl/pyproject.toml new file mode 100644 index 000000000..a81d8779b --- /dev/null +++ b/tools/net/ynl/pyproject.toml @@ -0,0 +1,24 @@ +[build-system] +requires = ["setuptools>=61.0"] +build-backend = "setuptools.build_meta" + +[project] +name = "pyynl" +authors = [ + {name = "Donald Hunter", email = "donald.hunter@gmail.com"}, + {name = "Jakub Kicinski", email = "kuba@kernel.org"}, +] +description = "yaml netlink (ynl)" +version = "0.0.1" +requires-python = ">=3.9" +dependencies = [ + "pyyaml==6.*", + "jsonschema==4.*" +] + +[tool.setuptools.packages.find] +include = ["pyynl", "pyynl.lib"] + +[project.scripts] +ynl = "pyynl.cli:main" +ynl-ethtool = "pyynl.ethtool:main" diff --git a/tools/net/ynl/pyynl/.gitignore b/tools/net/ynl/pyynl/.gitignore new file mode 100644 index 000000000..b801cd2d0 --- /dev/null +++ b/tools/net/ynl/pyynl/.gitignore @@ -0,0 +1,2 @@ +__pycache__/ +lib/__pycache__/ diff --git a/tools/net/ynl/pyynl/__init__.py b/tools/net/ynl/pyynl/__init__.py new file mode 100644 index 000000000..d8f59c132 --- /dev/null +++ b/tools/net/ynl/pyynl/__init__.py @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause + +""" Python YNL (YAML Netlink) library. """ + +# Re-export the public library API so it can be imported straight from the +# package, e.g. `from pyynl import YnlFamily`. +# pylint: disable=wildcard-import,unused-wildcard-import +from .lib import * +from .lib import __all__ diff --git a/tools/net/ynl/pyynl/cli.py b/tools/net/ynl/pyynl/cli.py new file mode 100755 index 000000000..b6a6ce12b --- /dev/null +++ b/tools/net/ynl/pyynl/cli.py @@ -0,0 +1,334 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause + +""" +YNL cli tool +""" + +import argparse +import json +import os +import pathlib +import pprint +import shutil +import sys +import textwrap + +# pylint: disable=no-name-in-module,wrong-import-position +sys.path.append(pathlib.Path(__file__).resolve().parent.as_posix()) +from lib import YnlFamily, Netlink, NlError, SpecFamily, SpecException, YnlException +from lib import list_families + +# pylint: disable=too-few-public-methods,too-many-locals +class Colors: + """ANSI color and font modifier codes""" + RESET = '\033[0m' + + BOLD = '\033[1m' + ITALICS = '\033[3m' + UNDERLINE = '\033[4m' + INVERT = '\033[7m' + + +def color(text, modifiers): + """Add color to text if output is a TTY + + Returns: + Colored text if stdout is a TTY, otherwise plain text + """ + if sys.stdout.isatty(): + # Join the colors if they are a list, if it's a string this a noop + modifiers = "".join(modifiers) + return f"{modifiers}{text}{Colors.RESET}" + return text + +def term_width(): + """ Get terminal width in columns (80 if stdout is not a terminal) """ + return shutil.get_terminal_size().columns + +class YnlEncoder(json.JSONEncoder): + """A custom encoder for emitting JSON with ynl-specific instance types""" + def default(self, o): + if isinstance(o, bytes): + return bytes.hex(o) + if isinstance(o, set): + return sorted(o) + return json.JSONEncoder.default(self, o) + + +def print_attr_list(ynl, attr_names, attr_set, indent=2): + """Print a list of attributes with their types and documentation.""" + prefix = ' ' * indent + for attr_name in attr_names: + if attr_name in attr_set.attrs: + attr = attr_set.attrs[attr_name] + attr_info = f'{prefix}- {color(attr_name, Colors.BOLD)}: {attr.type}' + if 'enum' in attr.yaml: + enum_name = attr.yaml['enum'] + attr_info += f" (enum: {enum_name})" + # Print enum values if available + if enum_name in ynl.consts: + const = ynl.consts[enum_name] + enum_values = list(const.entries.keys()) + type_fmted = color(const.type.capitalize(), Colors.ITALICS) + attr_info += f"\n{prefix} {type_fmted}: {', '.join(enum_values)}" + + # Show nested attributes reference and recursively display them + nested_set_name = None + if attr.type == 'nest' and 'nested-attributes' in attr.yaml: + nested_set_name = attr.yaml['nested-attributes'] + attr_info += f" -> {nested_set_name}" + + if attr.yaml.get('doc'): + doc_prefix = prefix + ' ' * 4 + doc_text = textwrap.fill(attr.yaml['doc'], width=term_width(), + initial_indent=doc_prefix, + subsequent_indent=doc_prefix) + attr_info += f"\n{doc_text}" + print(attr_info) + + # Recursively show nested attributes + if nested_set_name in ynl.attr_sets: + nested_set = ynl.attr_sets[nested_set_name] + # Filter out 'unspec' and other unused attrs + nested_names = [n for n in nested_set.attrs.keys() + if nested_set.attrs[n].type != 'unused'] + if nested_names: + print_attr_list(ynl, nested_names, nested_set, indent + 4) + + +def print_mode_attrs(ynl, mode, mode_spec, attr_set, consistent_dd_reply=None): + """Print a given mode (do/dump/event/notify).""" + mode_title = mode.capitalize() + + if 'request' in mode_spec and 'attributes' in mode_spec['request']: + print(f'\n{mode_title} request attributes:') + print_attr_list(ynl, mode_spec['request']['attributes'], attr_set) + + if 'reply' in mode_spec and 'attributes' in mode_spec['reply']: + if consistent_dd_reply and mode == "do": + title = None # Dump handling will print in combined format + elif consistent_dd_reply and mode == "dump": + title = 'Do and Dump' + else: + title = f'{mode_title}' + if title: + print(f'\n{title} reply attributes:') + print_attr_list(ynl, mode_spec['reply']['attributes'], attr_set) + + +def do_doc(ynl, op): + """Handle --list-attrs $op, print the attr information to stdout""" + print(f'Operation: {color(op.name, Colors.BOLD)}') + print(op.yaml['doc']) + + consistent_dd_reply = False + if 'do' in op.yaml and 'dump' in op.yaml and 'reply' in op.yaml['do'] and \ + op.yaml['do']['reply'] == op.yaml['dump'].get('reply'): + consistent_dd_reply = True + + for mode in ['do', 'dump']: + if mode in op.yaml: + print_mode_attrs(ynl, mode, op.yaml[mode], op.attr_set, + consistent_dd_reply=consistent_dd_reply) + + if 'attributes' in op.yaml.get('event', {}): + print('\nEvent attributes:') + print_attr_list(ynl, op.yaml['event']['attributes'], op.attr_set) + + if 'notify' in op.yaml: + mode_spec = op.yaml['notify'] + ref_spec = ynl.msgs.get(mode_spec).yaml.get('do') + if not ref_spec: + ref_spec = ynl.msgs.get(mode_spec).yaml.get('dump') + if ref_spec: + print('\nNotification attributes:') + print_attr_list(ynl, ref_spec['reply']['attributes'], op.attr_set) + + if 'mcgrp' in op.yaml: + print(f"\nMulticast group: {op.yaml['mcgrp']}") + + +# pylint: disable=too-many-locals,too-many-branches,too-many-statements +def main(): + """YNL cli tool""" + + description = """ + YNL CLI utility - a general purpose netlink utility that uses YAML + specs to drive protocol encoding and decoding. + """ + epilog = """ + The --multi option can be repeated to include several do operations + in the same netlink payload. + """ + + parser = argparse.ArgumentParser(description=description, + epilog=epilog, add_help=False) + + gen_group = parser.add_argument_group('General options') + gen_group.add_argument('-h', '--help', action='help', + help='show this help message and exit') + + spec_group = parser.add_argument_group('Netlink family selection') + spec_sel = spec_group.add_mutually_exclusive_group(required=True) + spec_sel.add_argument('--list-families', action='store_true', + help=('list Netlink families supported by YNL ' + '(which have a spec available in the standard ' + 'system path)')) + spec_sel.add_argument('--family', dest='family', type=str, + help='name of the Netlink FAMILY to use') + spec_sel.add_argument('--spec', dest='spec', type=str, + help='full file path to the YAML spec file') + + ops_group = parser.add_argument_group('Operations') + ops = ops_group.add_mutually_exclusive_group() + ops.add_argument('--do', dest='do', metavar='DO-OPERATION', type=str) + ops.add_argument('--dump', dest='dump', metavar='DUMP-OPERATION', type=str) + ops.add_argument('--multi', dest='multi', nargs=2, action='append', + metavar=('DO-OPERATION', 'JSON_TEXT'), type=str, + help="Multi-message operation sequence (for nftables)") + ops.add_argument('--list-ops', action='store_true', + help="List available --do and --dump operations") + ops.add_argument('--list-msgs', action='store_true', + help="List all messages of the family (incl. notifications)") + ops.add_argument('--list-attrs', '--doc', dest='list_attrs', metavar='MSG', + type=str, help='List attributes for a message / operation') + ops.add_argument('--validate', action='store_true', + help="Validate the spec against schema and exit") + + io_group = parser.add_argument_group('Input / Output') + io_group.add_argument('--json', dest='json_text', type=str, + help=('Specify attributes of the message to send ' + 'to the kernel in JSON format. Can be left out ' + 'if the message is expected to be empty.')) + io_group.add_argument('--output-json', action='store_true', + help='Format output as JSON') + + ntf_group = parser.add_argument_group('Notifications') + ntf_group.add_argument('--subscribe', dest='ntf', type=str) + ntf_group.add_argument('--duration', dest='duration', type=int, + help='when subscribed, watch for DURATION seconds') + ntf_group.add_argument('--sleep', dest='duration', type=int, + help='alias for duration') + + nlflags = parser.add_argument_group('Netlink message flags (NLM_F_*)', + ('Extra flags to set in nlmsg_flags of ' + 'the request, used mostly by older ' + 'Classic Netlink families.')) + nlflags.add_argument('--replace', dest='flags', action='append_const', + const=Netlink.NLM_F_REPLACE) + nlflags.add_argument('--excl', dest='flags', action='append_const', + const=Netlink.NLM_F_EXCL) + nlflags.add_argument('--create', dest='flags', action='append_const', + const=Netlink.NLM_F_CREATE) + nlflags.add_argument('--append', dest='flags', action='append_const', + const=Netlink.NLM_F_APPEND) + + schema_group = parser.add_argument_group('Development options') + schema_group.add_argument('--schema', dest='schema', type=str, + help="JSON schema to validate the spec") + schema_group.add_argument('--no-schema', action='store_true') + + dbg_group = parser.add_argument_group('Debug options') + io_group.add_argument('--policy', action='store_true', + help='Query kernel policy for the operation instead of executing it') + dbg_group.add_argument('--dbg-small-recv', default=0, const=4000, + action='store', nargs='?', type=int, metavar='INT', + help="Length of buffers used for recv()") + dbg_group.add_argument('--process-unknown', action=argparse.BooleanOptionalAction) + + args = parser.parse_args() + + def output(msg): + if args.output_json: + print(json.dumps(msg, cls=YnlEncoder)) + else: + pprint.pprint(msg, width=term_width(), compact=True) + + if args.list_families: + for family in list_families(): + print(family) + return + + if args.no_schema: + args.schema = '' + + attrs = {} + if args.json_text: + attrs = json.loads(args.json_text) + + if args.spec and not os.path.isfile(args.spec): + raise YnlException(f"Spec file {args.spec} does not exist") + + # Spec/YnlFamily will raise if both or neither spec and family are given + if args.validate: + # Force validation even for installed specs (schema=True), unless the + # user explicitly picked a schema or opted out with --no-schema. + schema = True if args.schema is None else args.schema + try: + SpecFamily(args.spec, schema_path=schema, family=args.family) + except SpecException as error: + print(error) + sys.exit(1) + return + + ynl = YnlFamily(args.spec, schema=args.schema, family=args.family, + process_unknown=args.process_unknown, + recv_size=args.dbg_small_recv) + if args.dbg_small_recv: + ynl.set_recv_dbg(True) + + if args.policy: + if args.do: + pol = ynl.get_policy(args.do, 'do') + output(pol.to_dict() if pol else None) + args.do = None + if args.dump: + pol = ynl.get_policy(args.dump, 'dump') + output(pol.to_dict() if pol else None) + args.dump = None + + if args.ntf: + ynl.ntf_subscribe(args.ntf) + + if args.list_ops: + for op_name, op in ynl.ops.items(): + print(op_name, " [", ", ".join(op.modes), "]") + if args.list_msgs: + for op_name, op in ynl.msgs.items(): + print(op_name, " [", ", ".join(op.modes), "]") + + if args.list_attrs: + op = ynl.msgs.get(args.list_attrs) + if not op: + print(f'Operation {args.list_attrs} not found') + sys.exit(1) + + do_doc(ynl, op) + + try: + if args.do: + reply = ynl.do(args.do, attrs, args.flags) + output(reply) + if args.dump: + reply = ynl.dump(args.dump, attrs) + output(reply) + if args.multi: + ops = [ (item[0], json.loads(item[1]), args.flags or []) for item in args.multi ] + reply = ynl.do_multi(ops) + output(reply) + + if args.ntf: + for msg in ynl.poll_ntf(duration=args.duration): + output(msg) + except NlError as e: + print(e) + sys.exit(1) + except KeyboardInterrupt: + pass + except BrokenPipeError: + pass + + +if __name__ == "__main__": + main() diff --git a/tools/net/ynl/pyynl/lib/__init__.py b/tools/net/ynl/pyynl/lib/__init__.py new file mode 100644 index 000000000..aa4263c8c --- /dev/null +++ b/tools/net/ynl/pyynl/lib/__init__.py @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause + +""" YNL library """ + +from .nlspec import SpecAttr, SpecAttrSet, SpecEnumEntry, SpecEnumSet, \ + SpecFamily, SpecOperation, SpecSubMessage, SpecSubMessageFormat, \ + SpecException +from .specdir import list_families +from .ynl import YnlFamily, Netlink, NlError, NlPolicy, YnlException + +from .doc_generator import YnlDocGenerator + +__all__ = ["SpecAttr", "SpecAttrSet", "SpecEnumEntry", "SpecEnumSet", + "SpecFamily", "SpecOperation", "SpecSubMessage", "SpecSubMessageFormat", + "SpecException", "list_families", + "YnlFamily", "Netlink", "NlError", "NlPolicy", "YnlException", + "YnlDocGenerator"] diff --git a/tools/net/ynl/pyynl/lib/doc_generator.py b/tools/net/ynl/pyynl/lib/doc_generator.py new file mode 100644 index 000000000..74f5d408e --- /dev/null +++ b/tools/net/ynl/pyynl/lib/doc_generator.py @@ -0,0 +1,404 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# -*- coding: utf-8; mode: python -*- + +""" + Class to auto generate the documentation for Netlink specifications. + + :copyright: Copyright (C) 2023 Breno Leitao <leitao@debian.org> + :license: GPL Version 2, June 1991 see linux/COPYING for details. + + This class performs extensive parsing to the Linux kernel's netlink YAML + spec files, in an effort to avoid needing to heavily mark up the original + YAML file. + + This code is split in two classes: + 1) RST formatters: Use to convert a string to a RST output + 2) YAML Netlink (YNL) doc generator: Generate docs from YAML data +""" + +from typing import Any, Dict, List +import yaml + +LINE_STR = '__lineno__' + +class NumberedSafeLoader(yaml.SafeLoader): # pylint: disable=R0901 + """Override the SafeLoader class to add line number to parsed data""" + + def construct_mapping(self, node, *args, **kwargs): + mapping = super().construct_mapping(node, *args, **kwargs) + mapping[LINE_STR] = node.start_mark.line + + return mapping + +class RstFormatters: + """RST Formatters""" + + SPACE_PER_LEVEL = 4 + + @staticmethod + def headroom(level: int) -> str: + """Return space to format""" + return " " * (level * RstFormatters.SPACE_PER_LEVEL) + + @staticmethod + def bold(text: str) -> str: + """Format bold text""" + return f"**{text}**" + + @staticmethod + def inline(text: str) -> str: + """Format inline text""" + return f"``{text}``" + + @staticmethod + def sanitize(text: str) -> str: + """Remove newlines and multiple spaces""" + # This is useful for some fields that are spread across multiple lines + return str(text).replace("\n", " ").strip() + + def rst_fields(self, key: str, value: str, level: int = 0) -> str: + """Return a RST formatted field""" + return self.headroom(level) + f":{key}: {value}" + + def rst_definition(self, key: str, value: Any, level: int = 0) -> str: + """Format a single rst definition""" + return self.headroom(level) + key + "\n" + self.headroom(level + 1) + str(value) + + def rst_paragraph(self, paragraph: str, level: int = 0) -> str: + """Return a formatted paragraph""" + return self.headroom(level) + paragraph + + def rst_bullet(self, item: str, level: int = 0) -> str: + """Return a formatted a bullet""" + return self.headroom(level) + f"- {item}" + + @staticmethod + def rst_subsection(title: str) -> str: + """Add a sub-section to the document""" + return f"{title}\n" + "-" * len(title) + + @staticmethod + def rst_subsubsection(title: str) -> str: + """Add a sub-sub-section to the document""" + return f"{title}\n" + "~" * len(title) + + @staticmethod + def rst_section(namespace: str, prefix: str, title: str) -> str: + """Add a section to the document""" + return f".. _{namespace}-{prefix}-{title}:\n\n{title}\n" + "=" * len(title) + + @staticmethod + def rst_subtitle(title: str) -> str: + """Add a subtitle to the document""" + return "\n" + "-" * len(title) + f"\n{title}\n" + "-" * len(title) + "\n\n" + + @staticmethod + def rst_title(title: str) -> str: + """Add a title to the document""" + return "=" * len(title) + f"\n{title}\n" + "=" * len(title) + "\n\n" + + def rst_list_inline(self, list_: List[str], level: int = 0) -> str: + """Format a list using inlines""" + return self.headroom(level) + "[" + ", ".join(self.inline(i) for i in list_) + "]" + + @staticmethod + def rst_ref(namespace: str, prefix: str, name: str) -> str: + """Add a hyperlink to the document""" + mappings = {'enum': 'definition', + 'fixed-header': 'definition', + 'nested-attributes': 'attribute-set', + 'struct': 'definition'} + prefix = mappings.get(prefix, prefix) + return f":ref:`{namespace}-{prefix}-{name}`" + + def rst_header(self) -> str: + """The headers for all the auto generated RST files""" + lines = [] + + lines.append(self.rst_paragraph(".. SPDX-License-Identifier: GPL-2.0")) + lines.append(self.rst_paragraph(".. NOTE: This document was auto-generated.\n\n")) + + return "\n".join(lines) + + @staticmethod + def rst_toctree(maxdepth: int = 2) -> str: + """Generate a toctree RST primitive""" + lines = [] + + lines.append(".. toctree::") + lines.append(f" :maxdepth: {maxdepth}\n\n") + + return "\n".join(lines) + + @staticmethod + def rst_label(title: str) -> str: + """Return a formatted label""" + return f".. _{title}:\n\n" + + @staticmethod + def rst_lineno(lineno: int) -> str: + """Return a lineno comment""" + return f".. LINENO {lineno}\n" + +class YnlDocGenerator: + """YAML Netlink specs Parser""" + + fmt = RstFormatters() + + def parse_mcast_group(self, mcast_group: List[Dict[str, Any]]) -> str: + """Parse 'multicast' group list and return a formatted string""" + lines = [] + for group in mcast_group: + lines.append(self.fmt.rst_bullet(group["name"])) + + return "\n".join(lines) + + def parse_do(self, do_dict: Dict[str, Any], level: int = 0) -> str: + """Parse 'do' section and return a formatted string""" + lines = [] + if LINE_STR in do_dict: + lines.append(self.fmt.rst_lineno(do_dict[LINE_STR])) + + for key in do_dict.keys(): + if key == LINE_STR: + continue + lines.append(self.fmt.rst_paragraph(self.fmt.bold(key), level + 1)) + if key in ['request', 'reply']: + lines.append(self.parse_op_attributes(do_dict[key], level + 1) + "\n") + else: + lines.append(self.fmt.headroom(level + 2) + do_dict[key] + "\n") + + return "\n".join(lines) + + def parse_op_attributes(self, attrs: Dict[str, Any], level: int = 0) -> str: + """Parse 'attributes' section""" + if "attributes" not in attrs: + return "" + lines = [self.fmt.rst_fields("attributes", + self.fmt.rst_list_inline(attrs["attributes"]), + level + 1)] + + return "\n".join(lines) + + def parse_operations(self, operations: List[Dict[str, Any]], namespace: str) -> str: + """Parse operations block""" + preprocessed = ["name", "doc", "title", "do", "dump", "flags", "event"] + linkable = ["fixed-header", "attribute-set"] + lines = [] + + for operation in operations: + if LINE_STR in operation: + lines.append(self.fmt.rst_lineno(operation[LINE_STR])) + + lines.append(self.fmt.rst_section(namespace, 'operation', + operation["name"])) + lines.append(self.fmt.rst_paragraph(operation["doc"]) + "\n") + + for key in operation.keys(): + if key == LINE_STR: + continue + + if key in preprocessed: + # Skip the special fields + continue + value = operation[key] + if key in linkable: + value = self.fmt.rst_ref(namespace, key, value) + lines.append(self.fmt.rst_fields(key, value, 0)) + if 'flags' in operation: + lines.append(self.fmt.rst_fields('flags', + self.fmt.rst_list_inline(operation['flags']))) + + if "do" in operation: + lines.append(self.fmt.rst_paragraph(":do:", 0)) + lines.append(self.parse_do(operation["do"], 0)) + if "dump" in operation: + lines.append(self.fmt.rst_paragraph(":dump:", 0)) + lines.append(self.parse_do(operation["dump"], 0)) + if "event" in operation: + lines.append(self.fmt.rst_paragraph(":event:", 0)) + lines.append(self.parse_op_attributes(operation["event"], 0)) + + # New line after fields + lines.append("\n") + + return "\n".join(lines) + + def parse_entries(self, entries: List[Dict[str, Any]], level: int) -> str: + """Parse a list of entries""" + ignored = ["pad"] + lines = [] + for entry in entries: + if isinstance(entry, dict): + # entries could be a list or a dictionary + field_name = entry.get("name", "") + if field_name in ignored: + continue + type_ = entry.get("type") + if type_: + field_name += f" ({self.fmt.inline(type_)})" + lines.append( + self.fmt.rst_fields(field_name, + self.fmt.sanitize(entry.get("doc", "")), + level) + ) + elif isinstance(entry, list): + lines.append(self.fmt.rst_list_inline(entry, level)) + else: + lines.append(self.fmt.rst_bullet(self.fmt.inline(self.fmt.sanitize(entry)), + level)) + + lines.append("\n") + return "\n".join(lines) + + def parse_definitions(self, defs: Dict[str, Any], namespace: str) -> str: + """Parse definitions section""" + preprocessed = ["name", "entries", "members"] + ignored = ["render-max"] # This is not printed + lines = [] + + for definition in defs: + if LINE_STR in definition: + lines.append(self.fmt.rst_lineno(definition[LINE_STR])) + + lines.append(self.fmt.rst_section(namespace, 'definition', definition["name"])) + for k in definition.keys(): + if k == LINE_STR: + continue + if k in preprocessed + ignored: + continue + lines.append(self.fmt.rst_fields(k, self.fmt.sanitize(definition[k]), 0)) + + # Field list needs to finish with a new line + lines.append("\n") + if "entries" in definition: + lines.append(self.fmt.rst_paragraph(":entries:", 0)) + lines.append(self.parse_entries(definition["entries"], 1)) + if "members" in definition: + lines.append(self.fmt.rst_paragraph(":members:", 0)) + lines.append(self.parse_entries(definition["members"], 1)) + + return "\n".join(lines) + + def parse_attr_sets(self, entries: List[Dict[str, Any]], namespace: str) -> str: + """Parse attribute from attribute-set""" + preprocessed = ["name", "type"] + linkable = ["enum", "nested-attributes", "struct", "sub-message"] + ignored = ["checks"] + lines = [] + + for entry in entries: + lines.append(self.fmt.rst_section(namespace, 'attribute-set', + entry["name"])) + + if "doc" in entry: + lines.append(self.fmt.rst_paragraph(entry["doc"], 0) + "\n") + + for attr in entry["attributes"]: + if LINE_STR in attr: + lines.append(self.fmt.rst_lineno(attr[LINE_STR])) + + type_ = attr.get("type") + attr_line = attr["name"] + if type_: + # Add the attribute type in the same line + attr_line += f" ({self.fmt.inline(type_)})" + + lines.append(self.fmt.rst_subsubsection(attr_line)) + + for k in attr.keys(): + if k == LINE_STR: + continue + if k in preprocessed + ignored: + continue + if k in linkable: + value = self.fmt.rst_ref(namespace, k, attr[k]) + else: + value = self.fmt.sanitize(attr[k]) + lines.append(self.fmt.rst_fields(k, value, 0)) + lines.append("\n") + + return "\n".join(lines) + + def parse_sub_messages(self, entries: List[Dict[str, Any]], namespace: str) -> str: + """Parse sub-message definitions""" + lines = [] + + for entry in entries: + lines.append(self.fmt.rst_section(namespace, 'sub-message', + entry["name"])) + for fmt in entry["formats"]: + value = fmt["value"] + + lines.append(self.fmt.rst_bullet(self.fmt.bold(value))) + for attr in ['fixed-header', 'attribute-set']: + if attr in fmt: + lines.append(self.fmt.rst_fields(attr, + self.fmt.rst_ref(namespace, + attr, + fmt[attr]), + 1)) + lines.append("\n") + + return "\n".join(lines) + + def parse_yaml(self, obj: Dict[str, Any]) -> str: + """Format the whole YAML into a RST string""" + lines = [] + + # Main header + lineno = obj.get('__lineno__', 0) + lines.append(self.fmt.rst_lineno(lineno)) + + family = obj['name'] + + lines.append(self.fmt.rst_header()) + lines.append(self.fmt.rst_label("netlink-" + family)) + + title = f"Family ``{family}`` netlink specification" + lines.append(self.fmt.rst_title(title)) + lines.append(self.fmt.rst_paragraph(".. contents:: :depth: 3\n")) + + if "doc" in obj: + lines.append(self.fmt.rst_subtitle("Summary")) + lines.append(self.fmt.rst_paragraph(obj["doc"], 0)) + + # Operations + if "operations" in obj: + lines.append(self.fmt.rst_subtitle("Operations")) + lines.append(self.parse_operations(obj["operations"]["list"], + family)) + + # Multicast groups + if "mcast-groups" in obj: + lines.append(self.fmt.rst_subtitle("Multicast groups")) + lines.append(self.parse_mcast_group(obj["mcast-groups"]["list"])) + + # Definitions + if "definitions" in obj: + lines.append(self.fmt.rst_subtitle("Definitions")) + lines.append(self.parse_definitions(obj["definitions"], family)) + + # Attributes set + if "attribute-sets" in obj: + lines.append(self.fmt.rst_subtitle("Attribute sets")) + lines.append(self.parse_attr_sets(obj["attribute-sets"], family)) + + # Sub-messages + if "sub-messages" in obj: + lines.append(self.fmt.rst_subtitle("Sub-messages")) + lines.append(self.parse_sub_messages(obj["sub-messages"], family)) + + return "\n".join(lines) + + # Main functions + # ============== + + def parse_yaml_file(self, filename: str) -> str: + """Transform the YAML specified by filename into an RST-formatted string""" + with open(filename, "r", encoding="utf-8") as spec_file: + numbered_yaml = yaml.load(spec_file, Loader=NumberedSafeLoader) + content = self.parse_yaml(numbered_yaml) + + return content diff --git a/tools/net/ynl/pyynl/lib/nlspec.py b/tools/net/ynl/pyynl/lib/nlspec.py new file mode 100644 index 000000000..b4ec59814 --- /dev/null +++ b/tools/net/ynl/pyynl/lib/nlspec.py @@ -0,0 +1,649 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +# +# pylint: disable=missing-function-docstring, too-many-instance-attributes, too-many-branches + +""" +The nlspec is a python library for parsing and using YNL netlink +specifications. +""" + +import collections +import importlib +import os +import yaml as pyyaml + +from .specdir import find_spec, SYS_SCHEMA_DIR + + +class SpecException(Exception): + """Netlink spec exception. + """ + + +class SpecElement: + """Netlink spec element. + + Abstract element of the Netlink spec. Implements the dictionary interface + for access to the raw spec. Supports iterative resolution of dependencies + across elements and class inheritance levels. The elements of the spec + may refer to each other, and although loops should be very rare, having + to maintain correct ordering of instantiation is painful, so the resolve() + method should be used to perform parts of init which require access to + other parts of the spec. + + Attributes: + yaml raw spec as loaded from the spec file + family back reference to the full family + + name name of the entity as listed in the spec (optional) + ident_name name which can be safely used as identifier in code (optional) + """ + def __init__(self, family, yaml): + self.yaml = yaml + self.family = family + + if 'name' in self.yaml: + self.name = self.yaml['name'] + self.ident_name = self.name.replace('-', '_') + + self._super_resolved = False + family.add_unresolved(self) + + def __getitem__(self, key): + return self.yaml[key] + + def __contains__(self, key): + return key in self.yaml + + def get(self, key, default=None): + return self.yaml.get(key, default) + + def resolve_up(self, up): + if not self._super_resolved: + up.resolve() + self._super_resolved = True + + def resolve(self): + pass + + +class SpecEnumEntry(SpecElement): + """ Entry within an enum declared in the Netlink spec. + + Attributes: + doc documentation string + enum_set back reference to the enum + value numerical value of this enum (use accessors in most situations!) + + Methods: + raw_value raw value, i.e. the id in the enum, unlike user value which is a mask for flags + user_value user value, same as raw value for enums, for flags it's the mask + """ + def __init__(self, enum_set, yaml, prev, value_start): + if isinstance(yaml, str): + yaml = {'name': yaml} + super().__init__(enum_set.family, yaml) + + self.doc = yaml.get('doc', '') + self.enum_set = enum_set + + if 'value' in yaml: + self.value = yaml['value'] + elif prev: + self.value = prev.value + 1 + else: + self.value = value_start + + def has_doc(self): + return bool(self.doc) + + def raw_value(self): + return self.value + + def user_value(self, as_flags=None): + if self.enum_set['type'] == 'flags' or as_flags: + return 1 << self.value + return self.value + + +class SpecEnumSet(SpecElement): + """ Enum type + + Represents an enumeration (list of numerical constants) + as declared in the "definitions" section of the spec. + + Attributes: + type enum or flags + entries entries by name + entries_by_val entries by value + Methods: + get_mask for flags compute the mask of all defined values + """ + def __init__(self, family, yaml): + super().__init__(family, yaml) + + self.type = yaml['type'] + + prev_entry = None + value_start = self.yaml.get('value-start', 0) + self.entries = {} + self.entries_by_val = {} + for entry in self.yaml['entries']: + e = self.new_entry(entry, prev_entry, value_start) + self.entries[e.name] = e + self.entries_by_val[e.raw_value()] = e + prev_entry = e + + def new_entry(self, entry, prev_entry, value_start): + return SpecEnumEntry(self, entry, prev_entry, value_start) + + def has_doc(self): + if 'doc' in self.yaml: + return True + return self.has_entry_doc() + + def has_entry_doc(self): + for entry in self.entries.values(): + if entry.has_doc(): + return True + return False + + def get_mask(self, as_flags=None): + mask = 0 + for e in self.entries.values(): + mask += e.user_value(as_flags) + return mask + + +class SpecAttr(SpecElement): + """ Single Netlink attribute type + + Represents a single attribute type within an attr space. + + Attributes: + type string, attribute type + value numerical ID when serialized + attr_set Attribute Set containing this attr + is_multi bool, attr may repeat multiple times + struct_name string, name of struct definition + sub_type string, name of sub type + len integer, optional byte length of binary types + display_hint string, hint to help choose format specifier + when displaying the value + sub_message string, name of sub message type + selector string, name of attribute used to select + sub-message type + + is_auto_scalar bool, attr is a variable-size scalar + """ + def __init__(self, family, attr_set, yaml, value): + super().__init__(family, yaml) + + self.type = yaml['type'] + self.value = value + self.attr_set = attr_set + self.is_multi = yaml.get('multi-attr', False) + self.struct_name = yaml.get('struct') + self.sub_type = yaml.get('sub-type') + self.byte_order = yaml.get('byte-order') + self.len = yaml.get('len') + self.display_hint = yaml.get('display-hint') + self.sub_message = yaml.get('sub-message') + self.selector = yaml.get('selector') + + self.is_auto_scalar = self.type in ("sint", "uint") + + +class SpecAttrSet(SpecElement): + """ Netlink Attribute Set class. + + Represents a ID space of attributes within Netlink. + + Note that unlike other elements, which expose contents of the raw spec + via the dictionary interface Attribute Set exposes attributes by name. + + Attributes: + attrs ordered dict of all attributes (indexed by name) + attrs_by_val ordered dict of all attributes (indexed by value) + subset_of parent set if this is a subset, otherwise None + """ + def __init__(self, family, yaml): + super().__init__(family, yaml) + + self.subset_of = self.yaml.get('subset-of', None) + + self.attrs = collections.OrderedDict() + self.attrs_by_val = collections.OrderedDict() + + if self.subset_of is None: + val = 1 + for elem in self.yaml['attributes']: + if 'value' in elem: + val = elem['value'] + + attr = self.new_attr(elem, val) + self.attrs[attr.name] = attr + self.attrs_by_val[attr.value] = attr + val += 1 + else: + real_set = family.attr_sets[self.subset_of] + for elem in self.yaml['attributes']: + real_attr = real_set[elem['name']] + combined_elem = real_attr.yaml | elem + attr = self.new_attr(combined_elem, real_attr.value) + + self.attrs[attr.name] = attr + self.attrs_by_val[attr.value] = attr + + def new_attr(self, elem, value): + return SpecAttr(self.family, self, elem, value) + + def __getitem__(self, key): + return self.attrs[key] + + def __contains__(self, key): + return key in self.attrs + + def __iter__(self): + yield from self.attrs + + def items(self): + return self.attrs.items() + + +class SpecStructMember(SpecElement): + """Struct member attribute + + Represents a single struct member attribute. + + Attributes: + type string, type of the member attribute + byte_order string or None for native byte order + enum string, name of the enum definition + len integer, optional byte length of binary types + display_hint string, hint to help choose format specifier + when displaying the value + struct string, name of nested struct type + """ + def __init__(self, family, yaml): + super().__init__(family, yaml) + self.type = yaml['type'] + self.byte_order = yaml.get('byte-order') + self.enum = yaml.get('enum') + self.len = yaml.get('len') + self.display_hint = yaml.get('display-hint') + self.struct = yaml.get('struct') + + +class SpecStruct(SpecElement): + """Netlink struct type + + Represents a C struct definition. + + Attributes: + members ordered list of struct members + """ + def __init__(self, family, yaml): + super().__init__(family, yaml) + + self.members = [] + for member in yaml.get('members', []): + self.members.append(self.new_member(family, member)) + + def new_member(self, family, elem): + return SpecStructMember(family, elem) + + def __iter__(self): + yield from self.members + + def items(self): + return self.members + + +class SpecSubMessage(SpecElement): + """ Netlink sub-message definition + + Represents a set of sub-message formats for polymorphic nlattrs + that contain type-specific sub messages. + + Attributes: + name string, name of sub-message definition + formats dict of sub-message formats indexed by match value + """ + def __init__(self, family, yaml): + super().__init__(family, yaml) + + self.formats = collections.OrderedDict() + for elem in self.yaml['formats']: + msg_format = self.new_format(family, elem) + self.formats[msg_format.value] = msg_format + + def new_format(self, family, msg_format): + return SpecSubMessageFormat(family, msg_format) + + +class SpecSubMessageFormat(SpecElement): + """ Netlink sub-message format definition + + Represents a single format for a sub-message. + + Attributes: + value attribute value to match against type selector + fixed_header string, name of fixed header, or None + attr_set string, name of attribute set, or None + """ + def __init__(self, family, yaml): + super().__init__(family, yaml) + + self.value = yaml.get('value') + self.fixed_header = yaml.get('fixed-header') + self.attr_set = yaml.get('attribute-set') + + +class SpecOperation(SpecElement): + """Netlink Operation + + Information about a single Netlink operation. + + Attributes: + value numerical ID when serialized, None if req/rsp values differ + + req_value numerical ID when serialized, user -> kernel + rsp_value numerical ID when serialized, user <- kernel + modes supported operation modes (do, dump, event etc.) + is_call bool, whether the operation is a call + is_async bool, whether the operation is a notification + is_resv bool, whether the operation does not exist (it's just a reserved ID) + attr_set attribute set name + fixed_header string, optional name of fixed header struct + + yaml raw spec as loaded from the spec file + """ + def __init__(self, family, yaml, req_value, rsp_value): + super().__init__(family, yaml) + + self.value = req_value if req_value == rsp_value else None + self.req_value = req_value + self.rsp_value = rsp_value + + self.modes = yaml.keys() & {'do', 'dump', 'event', 'notify'} + self.is_call = 'do' in yaml or 'dump' in yaml + self.is_async = 'notify' in yaml or 'event' in yaml + self.is_resv = not self.is_async and not self.is_call + self.fixed_header = self.yaml.get('fixed-header', family.fixed_header) + + # Added by resolve: + self.attr_set = None + delattr(self, "attr_set") + + def resolve(self): + self.resolve_up(super()) + + if 'attribute-set' in self.yaml: + attr_set_name = self.yaml['attribute-set'] + elif 'notify' in self.yaml: + msg = self.family.msgs[self.yaml['notify']] + attr_set_name = msg['attribute-set'] + elif self.is_resv: + attr_set_name = '' + else: + raise SpecException(f"Can't resolve attribute set for op '{self.name}'") + if attr_set_name: + self.attr_set = self.family.attr_sets[attr_set_name] + + +class SpecMcastGroup(SpecElement): + """Netlink Multicast Group + + Information about a multicast group. + + Value is only used for classic netlink families that use the + netlink-raw schema. Genetlink families use dynamic ID allocation + where the ids of multicast groups get resolved at runtime. Value + will be None for genetlink families. + + Attributes: + name name of the mulitcast group + value integer id of this multicast group for netlink-raw or None + yaml raw spec as loaded from the spec file + """ + def __init__(self, family, yaml): + super().__init__(family, yaml) + self.value = self.yaml.get('value') + + +class SpecFamily(SpecElement): + """ Netlink Family Spec class. + + Netlink family information loaded from a spec (e.g. in YAML). + Takes care of unfolding implicit information which can be skipped + in the spec itself for brevity. + + The class can be used like a dictionary to access the raw spec + elements but that's usually a bad idea. + + Attributes: + proto protocol type (e.g. genetlink) + msg_id_model enum-model for operations (unified, directional etc.) + license spec license (loaded from an SPDX tag on the spec) + + attr_sets dict of attribute sets + msgs dict of all messages (index by name) + sub_msgs dict of all sub messages (index by name) + ops dict of all valid requests / responses + ntfs dict of all async events + consts dict of all constants/enums + fixed_header string, optional name of family default fixed header struct + mcast_groups dict of all multicast groups (index by name) + kernel_family dict of kernel family attributes + """ + + # To be loaded dynamically as needed + jsonschema = None + + try: + _yaml_loader = pyyaml.CSafeLoader + except AttributeError: + _yaml_loader = pyyaml.SafeLoader + + def __init__(self, spec_path=None, schema_path=None, exclude_ops=None, + family=None): + # schema_path selects how the spec is validated: + # None -- no preference: validate against the default schema, + # but trust (skip) installed specs selected by family= + # True -- always validate against the default schema + # path -- validate against this schema + # '' -- do not validate + if (spec_path is None) == (family is None): + raise ValueError("Specify exactly one of spec path or family name") + if family is not None: + spec_path = find_spec(family) + # Installed specs are assumed correct, so skip schema validation + # to save cycles unless the caller asked to validate. + if schema_path is None and spec_path.startswith(SYS_SCHEMA_DIR): + schema_path = '' + + with open(spec_path, "r", encoding='utf-8') as stream: + prefix = '# SPDX-License-Identifier: ' + first = stream.readline().strip() + if not first.startswith(prefix): + raise SpecException('SPDX license tag required in the spec') + self.license = first[len(prefix):] + + stream.seek(0) + spec = pyyaml.load(stream, Loader=self._yaml_loader) + + self.fixed_header = None + self._resolution_list = [] + + super().__init__(self, spec) + + self._exclude_ops = exclude_ops if exclude_ops else [] + + self.proto = self.yaml.get('protocol', 'genetlink') + self.msg_id_model = self.yaml['operations'].get('enum-model', 'unified') + + if schema_path is None or schema_path is True: + schema_path = os.path.dirname(os.path.dirname(spec_path)) + f'/{self.proto}.yaml' + if schema_path: + with open(schema_path, "r", encoding='utf-8') as stream: + schema = pyyaml.load(stream, Loader=self._yaml_loader) + + if SpecFamily.jsonschema is None: + SpecFamily.jsonschema = importlib.import_module("jsonschema") + + SpecFamily.jsonschema.validate(self.yaml, schema) + + self.attr_sets = collections.OrderedDict() + self.sub_msgs = collections.OrderedDict() + self.msgs = collections.OrderedDict() + self.req_by_value = collections.OrderedDict() + self.rsp_by_value = collections.OrderedDict() + self.ops = collections.OrderedDict() + self.ntfs = collections.OrderedDict() + self.consts = collections.OrderedDict() + self.mcast_groups = collections.OrderedDict() + self.kernel_family = collections.OrderedDict(self.yaml.get('kernel-family', {})) + + last_exception = None + while len(self._resolution_list) > 0: + resolved = [] + unresolved = self._resolution_list + self._resolution_list = [] + + for elem in unresolved: + try: + elem.resolve() + except (KeyError, AttributeError) as e: + self._resolution_list.append(elem) + last_exception = e + continue + + resolved.append(elem) + + if len(resolved) == 0: + raise last_exception + + def new_enum(self, elem): + return SpecEnumSet(self, elem) + + def new_attr_set(self, elem): + return SpecAttrSet(self, elem) + + def new_struct(self, elem): + return SpecStruct(self, elem) + + def new_sub_message(self, elem): + return SpecSubMessage(self, elem) + + def new_operation(self, elem, req_val, rsp_val): + return SpecOperation(self, elem, req_val, rsp_val) + + def new_mcast_group(self, elem): + return SpecMcastGroup(self, elem) + + def add_unresolved(self, elem): + self._resolution_list.append(elem) + + def _dictify_ops_unified(self): + self.fixed_header = self.yaml['operations'].get('fixed-header') + val = 1 + for elem in self.yaml['operations']['list']: + if 'value' in elem: + val = elem['value'] + + op = self.new_operation(elem, val, val) + val += 1 + + self.msgs[op.name] = op + + def _dictify_ops_directional(self): + self.fixed_header = self.yaml['operations'].get('fixed-header') + req_val = rsp_val = 1 + for elem in self.yaml['operations']['list']: + if 'notify' in elem or 'event' in elem: + if 'value' in elem: + rsp_val = elem['value'] + req_val_next = req_val + rsp_val_next = rsp_val + 1 + req_val = None + elif 'do' in elem or 'dump' in elem: + mode = elem['do'] if 'do' in elem else elem['dump'] + + v = mode.get('request', {}).get('value', None) + if v: + req_val = v + v = mode.get('reply', {}).get('value', None) + if v: + rsp_val = v + + rsp_inc = 1 if 'reply' in mode else 0 + req_val_next = req_val + 1 + rsp_val_next = rsp_val + rsp_inc + else: + raise SpecException("Can't parse directional ops") + + if req_val == req_val_next: + req_val = None + if rsp_val == rsp_val_next: + rsp_val = None + + skip = False + for exclude in self._exclude_ops: + skip |= bool(exclude.match(elem['name'])) + if not skip: + op = self.new_operation(elem, req_val, rsp_val) + self.msgs[op.name] = op + + req_val = req_val_next + rsp_val = rsp_val_next + + def find_operation(self, name): + """ + For a given operation name, find and return operation spec. + """ + for op in self.yaml['operations']['list']: + if name == op['name']: + return op + return None + + def resolve(self): + self.resolve_up(super()) + + definitions = self.yaml.get('definitions', []) + for elem in definitions: + if elem['type'] == 'enum' or elem['type'] == 'flags': + self.consts[elem['name']] = self.new_enum(elem) + elif elem['type'] == 'struct': + self.consts[elem['name']] = self.new_struct(elem) + else: + self.consts[elem['name']] = elem + + for elem in self.yaml['attribute-sets']: + attr_set = self.new_attr_set(elem) + self.attr_sets[elem['name']] = attr_set + + for elem in self.yaml.get('sub-messages', []): + sub_message = self.new_sub_message(elem) + self.sub_msgs[sub_message.name] = sub_message + + if self.msg_id_model == 'unified': + self._dictify_ops_unified() + elif self.msg_id_model == 'directional': + self._dictify_ops_directional() + + for op in self.msgs.values(): + if op.req_value is not None: + self.req_by_value[op.req_value] = op + if op.rsp_value is not None: + self.rsp_by_value[op.rsp_value] = op + if not op.is_async and 'attribute-set' in op: + self.ops[op.name] = op + elif op.is_async: + self.ntfs[op.name] = op + + mcgs = self.yaml.get('mcast-groups') + if mcgs: + for elem in mcgs['list']: + mcg = self.new_mcast_group(elem) + self.mcast_groups[elem['name']] = mcg diff --git a/tools/net/ynl/pyynl/lib/specdir.py b/tools/net/ynl/pyynl/lib/specdir.py new file mode 100644 index 000000000..fcea9b9fb --- /dev/null +++ b/tools/net/ynl/pyynl/lib/specdir.py @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause + +""" +Locating YNL spec and schema files on disk. + +Resolves the directory holding the YAML specs (preferring an in-tree copy +over the installed system path) and maps family names to spec files. +""" + +import os + +SYS_SCHEMA_DIR='/usr/share/ynl' +RELATIVE_SCHEMA_DIR='../../../../../Documentation/netlink' + + +def schema_dir(): + """ + Return the effective schema directory, preferring in-tree before + system schema directory. + """ + script_dir = os.path.dirname(os.path.abspath(__file__)) + schema_dir_ = os.path.abspath(f"{script_dir}/{RELATIVE_SCHEMA_DIR}") + if not os.path.isdir(schema_dir_): + schema_dir_ = SYS_SCHEMA_DIR + if not os.path.isdir(schema_dir_): + raise FileNotFoundError(f"Schema directory {schema_dir_} does not exist") + return schema_dir_ + +def spec_dir(): + """ + Return the effective spec directory, relative to the effective + schema directory. + """ + spec_dir_ = schema_dir() + '/specs' + if not os.path.isdir(spec_dir_): + raise FileNotFoundError(f"Spec directory {spec_dir_} does not exist") + return spec_dir_ + + +def find_spec(family): + """ Return the path to the YAML spec file for a family by name. """ + spec = f"{spec_dir()}/{family}.yaml" + if not os.path.isfile(spec): + raise FileNotFoundError(f"Spec for family '{family}' not found at {spec}") + return spec + + +def list_families(): + """ Return the sorted names of all families with an installed spec. """ + return sorted(f.removesuffix('.yaml') + for f in os.listdir(spec_dir()) if f.endswith('.yaml')) diff --git a/tools/net/ynl/pyynl/lib/ynl.py b/tools/net/ynl/pyynl/lib/ynl.py new file mode 100644 index 000000000..8682bf588 --- /dev/null +++ b/tools/net/ynl/pyynl/lib/ynl.py @@ -0,0 +1,1474 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +# +# pylint: disable=missing-class-docstring, missing-function-docstring +# pylint: disable=too-many-branches, too-many-locals, too-many-instance-attributes +# pylint: disable=too-many-lines + +""" +YAML Netlink Library + +An implementation of the genetlink and raw netlink protocols. +""" + +from collections import namedtuple +from enum import Enum +import functools +import os +import random +import socket +import struct +from struct import Struct +import sys +import ipaddress +import uuid +import queue +import selectors +import time + +from .nlspec import SpecFamily + +# +# Generic Netlink code which should really be in some library, but I can't quickly find one. +# + + +class YnlException(Exception): + pass + + +# pylint: disable=too-few-public-methods +class Netlink: + # Netlink socket + SOL_NETLINK = 270 + + NETLINK_ADD_MEMBERSHIP = 1 + NETLINK_LISTEN_ALL_NSID = 8 + NETLINK_CAP_ACK = 10 + NETLINK_EXT_ACK = 11 + NETLINK_GET_STRICT_CHK = 12 + + # Netlink message + NLMSG_ERROR = 2 + NLMSG_DONE = 3 + + NLM_F_REQUEST = 1 + NLM_F_ACK = 4 + NLM_F_ROOT = 0x100 + NLM_F_MATCH = 0x200 + + NLM_F_REPLACE = 0x100 + NLM_F_EXCL = 0x200 + NLM_F_CREATE = 0x400 + NLM_F_APPEND = 0x800 + + NLM_F_CAPPED = 0x100 + NLM_F_ACK_TLVS = 0x200 + + NLM_F_DUMP = NLM_F_ROOT | NLM_F_MATCH + + NLA_F_NESTED = 0x8000 + NLA_F_NET_BYTEORDER = 0x4000 + + NLA_TYPE_MASK = NLA_F_NESTED | NLA_F_NET_BYTEORDER + + # Genetlink defines + NETLINK_GENERIC = 16 + + GENL_ID_CTRL = 0x10 + + # nlctrl + CTRL_CMD_GETFAMILY = 3 + CTRL_CMD_GETPOLICY = 10 + + CTRL_ATTR_FAMILY_ID = 1 + CTRL_ATTR_FAMILY_NAME = 2 + CTRL_ATTR_MAXATTR = 5 + CTRL_ATTR_MCAST_GROUPS = 7 + CTRL_ATTR_POLICY = 8 + CTRL_ATTR_OP_POLICY = 9 + CTRL_ATTR_OP = 10 + + CTRL_ATTR_MCAST_GRP_NAME = 1 + CTRL_ATTR_MCAST_GRP_ID = 2 + + CTRL_ATTR_POLICY_DO = 1 + CTRL_ATTR_POLICY_DUMP = 2 + + # Extack types + NLMSGERR_ATTR_MSG = 1 + NLMSGERR_ATTR_OFFS = 2 + NLMSGERR_ATTR_COOKIE = 3 + NLMSGERR_ATTR_POLICY = 4 + NLMSGERR_ATTR_MISS_TYPE = 5 + NLMSGERR_ATTR_MISS_NEST = 6 + + # Policy types + NL_POLICY_TYPE_ATTR_TYPE = 1 + NL_POLICY_TYPE_ATTR_MIN_VALUE_S = 2 + NL_POLICY_TYPE_ATTR_MAX_VALUE_S = 3 + NL_POLICY_TYPE_ATTR_MIN_VALUE_U = 4 + NL_POLICY_TYPE_ATTR_MAX_VALUE_U = 5 + NL_POLICY_TYPE_ATTR_MIN_LENGTH = 6 + NL_POLICY_TYPE_ATTR_MAX_LENGTH = 7 + NL_POLICY_TYPE_ATTR_POLICY_IDX = 8 + NL_POLICY_TYPE_ATTR_POLICY_MAXTYPE = 9 + NL_POLICY_TYPE_ATTR_BITFIELD32_MASK = 10 + NL_POLICY_TYPE_ATTR_PAD = 11 + NL_POLICY_TYPE_ATTR_MASK = 12 + + AttrType = Enum('AttrType', ['flag', 'u8', 'u16', 'u32', 'u64', + 's8', 's16', 's32', 's64', + 'binary', 'string', 'nul-string', + 'nested', 'nested-array', + 'bitfield32', 'sint', 'uint']) + +class NlError(Exception): + def __init__(self, nl_msg): + self.nl_msg = nl_msg + self.error = -nl_msg.error + + def __str__(self): + msg = "Netlink error: " + + extack = self.nl_msg.extack.copy() if self.nl_msg.extack else {} + if 'msg' in extack: + msg += extack['msg'] + ': ' + del extack['msg'] + msg += os.strerror(self.error) + if extack: + msg += ' ' + str(extack) + return msg + + +class ConfigError(Exception): + pass + + +class NlPolicy: + """Kernel policy for one mode (do or dump) of one operation. + + Returned by YnlFamily.get_policy(). Attributes of the policy + are accessible as attributes of the object. Nested policies + can be accessed indexing the object like a dictionary:: + + pol = ynl.get_policy('page-pool-stats-get', 'do') + pol['info'].type # 'nested' + pol['info']['id'].type # 'uint' + pol['info']['id'].min_value # 1 + + Each policy entry always has a 'type' attribute (e.g. u32, string, + nested). Optional attributes depending on the 'type': min-value, + max-value, min-length, max-length, mask. + + Policies can form infinite nesting loops. These loops are trimmed + when policy is converted to a dict with pol.to_dict(). + """ + def __init__(self, ynl, policy_idx, policy_table, attr_set, props=None): + self._policy_idx = policy_idx + self._policy_table = policy_table + self._ynl = ynl + self._props = props or {} + self._entries = {} + self._cache = {} + if policy_idx is not None and policy_idx in policy_table: + for attr_id, decoded in policy_table[policy_idx].items(): + if attr_set and attr_id in attr_set.attrs_by_val: + spec = attr_set.attrs_by_val[attr_id] + name = spec['name'] + else: + spec = None + name = f'attr-{attr_id}' + self._entries[name] = (spec, decoded) + + def __getitem__(self, name): + """Descend into a nested policy by attribute name.""" + if name not in self._cache: + spec, decoded = self._entries[name] + props = dict(decoded) + child_idx = None + child_set = None + if 'policy-idx' in props: + child_idx = props.pop('policy-idx') + if spec and 'nested-attributes' in spec.yaml: + child_set = self._ynl.attr_sets[spec.yaml['nested-attributes']] + self._cache[name] = NlPolicy(self._ynl, child_idx, + self._policy_table, + child_set, props) + return self._cache[name] + + def __getattr__(self, name): + """Access this policy entry's own properties (type, min-value, etc.). + + Underscores in the name are converted to dashes, so that + pol.min_value looks up "min-value". + """ + key = name.replace('_', '-') + try: + # Hack for level-0 which we still want to have .type but we don't + # want type to pointlessly show up in the dict / JSON form. + if not self._props and name == "type": + return "nested" + return self._props[key] + except KeyError: + raise AttributeError(name) + + def get(self, name, default=None): + """Look up a child policy entry by attribute name, with a default.""" + try: + return self[name] + except KeyError: + return default + + def __contains__(self, name): + return name in self._entries + + def __len__(self): + return len(self._entries) + + def __iter__(self): + return iter(self._entries) + + def keys(self): + """Return attribute names accepted by this policy.""" + return self._entries.keys() + + def to_dict(self, seen=None): + """Convert to a plain dict, suitable for JSON serialization. + + Nested NlPolicy objects are expanded recursively. Cyclic + references are trimmed (resolved to just {"type": "nested"}). + """ + if seen is None: + seen = set() + result = dict(self._props) + if self._policy_idx is not None: + if self._policy_idx not in seen: + seen = seen | {self._policy_idx} + children = {} + for name in self: + children[name] = self[name].to_dict(seen) + if self._props: + result['policy'] = children + else: + result = children + return result + + def __repr__(self): + return repr(self.to_dict()) + + +class NlAttr: + ScalarFormat = namedtuple('ScalarFormat', ['native', 'big', 'little']) + type_formats = { + 'u8' : ScalarFormat(Struct('B'), Struct("B"), Struct("B")), + 's8' : ScalarFormat(Struct('b'), Struct("b"), Struct("b")), + 'u16': ScalarFormat(Struct('H'), Struct(">H"), Struct("<H")), + 's16': ScalarFormat(Struct('h'), Struct(">h"), Struct("<h")), + 'u32': ScalarFormat(Struct('I'), Struct(">I"), Struct("<I")), + 's32': ScalarFormat(Struct('i'), Struct(">i"), Struct("<i")), + 'u64': ScalarFormat(Struct('Q'), Struct(">Q"), Struct("<Q")), + 's64': ScalarFormat(Struct('q'), Struct(">q"), Struct("<q")) + } + + def __init__(self, raw, offset): + self._len, self._type = struct.unpack("HH", raw[offset : offset + 4]) + self.type = self._type & ~Netlink.NLA_TYPE_MASK + self.is_nest = self._type & Netlink.NLA_F_NESTED + self.payload_len = self._len + self.full_len = (self.payload_len + 3) & ~3 + self.raw = raw[offset + 4 : offset + self.payload_len] + + @classmethod + def get_format(cls, attr_type, byte_order=None): + format_ = cls.type_formats[attr_type] + if byte_order: + return format_.big if byte_order == "big-endian" \ + else format_.little + return format_.native + + def as_scalar(self, attr_type, byte_order=None): + format_ = self.get_format(attr_type, byte_order) + return format_.unpack(self.raw)[0] + + def as_auto_scalar(self, attr_type, byte_order=None): + if len(self.raw) != 4 and len(self.raw) != 8: + raise YnlException(f"Auto-scalar len payload be 4 or 8 bytes, got {len(self.raw)}") + real_type = attr_type[0] + str(len(self.raw) * 8) + format_ = self.get_format(real_type, byte_order) + return format_.unpack(self.raw)[0] + + def as_strz(self): + return self.raw.decode('ascii')[:-1] + + def as_bin(self): + return self.raw + + def as_c_array(self, c_type): + format_ = self.get_format(c_type) + return [ x[0] for x in format_.iter_unpack(self.raw) ] + + def __repr__(self): + return f"[type:{self.type} len:{self._len}] {self.raw}" + + +class NlAttrs: + def __init__(self, msg, offset=0): + self.attrs = [] + + while offset < len(msg): + attr = NlAttr(msg, offset) + offset += attr.full_len + self.attrs.append(attr) + + def __iter__(self): + yield from self.attrs + + def __repr__(self): + msg = '' + for a in self.attrs: + if msg: + msg += '\n' + msg += repr(a) + return msg + + +class NlMsg: + def __init__(self, msg, offset, attr_space=None): + self.hdr = msg[offset : offset + 16] + + self.nl_len, self.nl_type, self.nl_flags, self.nl_seq, self.nl_portid = \ + struct.unpack("IHHII", self.hdr) + + self.raw = msg[offset + 16 : offset + self.nl_len] + + self.error = 0 + self.done = 0 + + extack_off = None + if self.nl_type == Netlink.NLMSG_ERROR: + self.error = struct.unpack("i", self.raw[0:4])[0] + self.done = 1 + extack_off = 20 + elif self.nl_type == Netlink.NLMSG_DONE: + self.error = struct.unpack("i", self.raw[0:4])[0] + self.done = 1 + extack_off = 4 + + self.extack = None + if self.nl_flags & Netlink.NLM_F_ACK_TLVS and extack_off: + self.extack = {} + extack_attrs = NlAttrs(self.raw[extack_off:]) + for extack in extack_attrs: + if extack.type == Netlink.NLMSGERR_ATTR_MSG: + self.extack['msg'] = extack.as_strz() + elif extack.type == Netlink.NLMSGERR_ATTR_MISS_TYPE: + self.extack['miss-type'] = extack.as_scalar('u32') + elif extack.type == Netlink.NLMSGERR_ATTR_MISS_NEST: + self.extack['miss-nest'] = extack.as_scalar('u32') + elif extack.type == Netlink.NLMSGERR_ATTR_OFFS: + self.extack['bad-attr-offs'] = extack.as_scalar('u32') + elif extack.type == Netlink.NLMSGERR_ATTR_POLICY: + self.extack['policy'] = _genl_decode_policy(extack.raw) + else: + if 'unknown' not in self.extack: + self.extack['unknown'] = [] + self.extack['unknown'].append(extack) + + if attr_space: + self.annotate_extack(attr_space) + + def annotate_extack(self, attr_space): + """ Make extack more human friendly with attribute information """ + + # We don't have the ability to parse nests yet, so only do global + if 'miss-type' in self.extack and 'miss-nest' not in self.extack: + miss_type = self.extack['miss-type'] + if miss_type in attr_space.attrs_by_val: + spec = attr_space.attrs_by_val[miss_type] + self.extack['miss-type'] = spec['name'] + if 'doc' in spec: + self.extack['miss-type-doc'] = spec['doc'] + + def cmd(self): + return self.nl_type + + def __repr__(self): + msg = (f"nl_len = {self.nl_len} ({len(self.raw)}) " + f"nl_flags = 0x{self.nl_flags:x} nl_type = {self.nl_type}") + if self.error: + msg += '\n\terror: ' + str(self.error) + if self.extack: + msg += '\n\textack: ' + repr(self.extack) + return msg + + +# pylint: disable=too-few-public-methods +class NlMsgs: + def __init__(self, data): + self.msgs = [] + + offset = 0 + while offset < len(data): + msg = NlMsg(data, offset) + offset += msg.nl_len + self.msgs.append(msg) + + def __iter__(self): + yield from self.msgs + + +def _genl_msg(nl_type, nl_flags, genl_cmd, genl_version, seq=None): + # we prepend length in _genl_msg_finalize() + if seq is None: + seq = random.randint(1, 1024) + nlmsg = struct.pack("HHII", nl_type, nl_flags, seq, 0) + genlmsg = struct.pack("BBH", genl_cmd, genl_version, 0) + return nlmsg + genlmsg + + +def _genl_msg_finalize(msg): + return struct.pack("I", len(msg) + 4) + msg + + +def _genl_decode_policy(raw): + policy = {} + for attr in NlAttrs(raw): + if attr.type == Netlink.NL_POLICY_TYPE_ATTR_TYPE: + type_ = attr.as_scalar('u32') + policy['type'] = Netlink.AttrType(type_).name + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_MIN_VALUE_S: + policy['min-value'] = attr.as_scalar('s64') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_MAX_VALUE_S: + policy['max-value'] = attr.as_scalar('s64') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_MIN_VALUE_U: + policy['min-value'] = attr.as_scalar('u64') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_MAX_VALUE_U: + policy['max-value'] = attr.as_scalar('u64') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_MIN_LENGTH: + policy['min-length'] = attr.as_scalar('u32') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_MAX_LENGTH: + policy['max-length'] = attr.as_scalar('u32') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_POLICY_IDX: + policy['policy-idx'] = attr.as_scalar('u32') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_BITFIELD32_MASK: + policy['bitfield32-mask'] = attr.as_scalar('u32') + elif attr.type == Netlink.NL_POLICY_TYPE_ATTR_MASK: + policy['mask'] = attr.as_scalar('u64') + return policy + + +# pylint: disable=too-many-nested-blocks +def _genl_load_families(): + genl_family_name_to_id = {} + + with socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, Netlink.NETLINK_GENERIC) as sock: + sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_CAP_ACK, 1) + + msg = _genl_msg(Netlink.GENL_ID_CTRL, + Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK | Netlink.NLM_F_DUMP, + Netlink.CTRL_CMD_GETFAMILY, 1) + msg = _genl_msg_finalize(msg) + + sock.send(msg, 0) + + while True: + reply = sock.recv(128 * 1024) + nms = NlMsgs(reply) + for nl_msg in nms: + if nl_msg.error: + raise YnlException(f"Netlink error: {nl_msg.error}") + if nl_msg.done: + return genl_family_name_to_id + + gm = GenlMsg(nl_msg) + fam = {} + for attr in NlAttrs(gm.raw): + if attr.type == Netlink.CTRL_ATTR_FAMILY_ID: + fam['id'] = attr.as_scalar('u16') + elif attr.type == Netlink.CTRL_ATTR_FAMILY_NAME: + fam['name'] = attr.as_strz() + elif attr.type == Netlink.CTRL_ATTR_MAXATTR: + fam['maxattr'] = attr.as_scalar('u32') + elif attr.type == Netlink.CTRL_ATTR_MCAST_GROUPS: + fam['mcast'] = {} + for entry in NlAttrs(attr.raw): + mcast_name = None + mcast_id = None + for entry_attr in NlAttrs(entry.raw): + if entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_NAME: + mcast_name = entry_attr.as_strz() + elif entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_ID: + mcast_id = entry_attr.as_scalar('u32') + if mcast_name and mcast_id is not None: + fam['mcast'][mcast_name] = mcast_id + if 'name' in fam and 'id' in fam: + genl_family_name_to_id[fam['name']] = fam + + +# pylint: disable=too-many-nested-blocks +def _genl_policy_dump(family_id, op): + op_policy = {} + policy_table = {} + + with socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, Netlink.NETLINK_GENERIC) as sock: + sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_CAP_ACK, 1) + + msg = _genl_msg(Netlink.GENL_ID_CTRL, + Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK | Netlink.NLM_F_DUMP, + Netlink.CTRL_CMD_GETPOLICY, 1) + msg += struct.pack('HHHxx', 6, Netlink.CTRL_ATTR_FAMILY_ID, family_id) + msg += struct.pack('HHI', 8, Netlink.CTRL_ATTR_OP, op) + msg = _genl_msg_finalize(msg) + + sock.send(msg, 0) + + while True: + reply = sock.recv(128 * 1024) + nms = NlMsgs(reply) + for nl_msg in nms: + if nl_msg.error: + raise YnlException(f"Netlink error: {nl_msg.error}") + if nl_msg.done: + return op_policy, policy_table + + gm = GenlMsg(nl_msg) + for attr in NlAttrs(gm.raw): + if attr.type == Netlink.CTRL_ATTR_OP_POLICY: + for op_attr in NlAttrs(attr.raw): + for method_attr in NlAttrs(op_attr.raw): + if method_attr.type == Netlink.CTRL_ATTR_POLICY_DO: + op_policy['do'] = method_attr.as_scalar('u32') + elif method_attr.type == Netlink.CTRL_ATTR_POLICY_DUMP: + op_policy['dump'] = method_attr.as_scalar('u32') + elif attr.type == Netlink.CTRL_ATTR_POLICY: + for pidx_attr in NlAttrs(attr.raw): + policy_idx = pidx_attr.type + for aid_attr in NlAttrs(pidx_attr.raw): + attr_id = aid_attr.type + decoded = _genl_decode_policy(aid_attr.raw) + if policy_idx not in policy_table: + policy_table[policy_idx] = {} + policy_table[policy_idx][attr_id] = decoded + + +class GenlMsg: + def __init__(self, nl_msg): + self.nl = nl_msg + self.genl_cmd, self.genl_version, _ = struct.unpack_from("BBH", nl_msg.raw, 0) + self.raw = nl_msg.raw[4:] + self.raw_attrs = [] + + def cmd(self): + return self.genl_cmd + + def __repr__(self): + msg = repr(self.nl) + msg += f"\tgenl_cmd = {self.genl_cmd} genl_ver = {self.genl_version}\n" + for a in self.raw_attrs: + msg += '\t\t' + repr(a) + '\n' + return msg + + +class NetlinkProtocol: + def __init__(self, family_name, proto_num): + self.family_name = family_name + self.proto_num = proto_num + + def _message(self, nl_type, nl_flags, seq=None): + if seq is None: + seq = random.randint(1, 1024) + nlmsg = struct.pack("HHII", nl_type, nl_flags, seq, 0) + return nlmsg + + def message(self, flags, command, _version, seq=None): + return self._message(command, flags, seq) + + def _decode(self, nl_msg): + return nl_msg + + def decode(self, ynl, nl_msg, op): + msg = self._decode(nl_msg) + if op is None: + op = ynl.rsp_by_value[msg.cmd()] + fixed_header_size = ynl.struct_size(op.fixed_header) + msg.raw_attrs = NlAttrs(msg.raw, fixed_header_size) + return msg + + def get_mcast_id(self, mcast_name, mcast_groups): + if mcast_name not in mcast_groups: + raise YnlException(f'Multicast group "{mcast_name}" not present in the spec') + return mcast_groups[mcast_name].value + + def msghdr_size(self): + return 16 + + +class GenlProtocol(NetlinkProtocol): + genl_family_name_to_id = {} + + def __init__(self, family_name): + super().__init__(family_name, Netlink.NETLINK_GENERIC) + + if not GenlProtocol.genl_family_name_to_id: + GenlProtocol.genl_family_name_to_id = _genl_load_families() + + self.genl_family = GenlProtocol.genl_family_name_to_id[family_name] + self.family_id = GenlProtocol.genl_family_name_to_id[family_name]['id'] + + def message(self, flags, command, version, seq=None): + nlmsg = self._message(self.family_id, flags, seq) + genlmsg = struct.pack("BBH", command, version, 0) + return nlmsg + genlmsg + + def _decode(self, nl_msg): + return GenlMsg(nl_msg) + + def get_mcast_id(self, mcast_name, mcast_groups): + if mcast_name not in self.genl_family['mcast']: + raise YnlException(f'Multicast group "{mcast_name}" not present in the family') + return self.genl_family['mcast'][mcast_name] + + def msghdr_size(self): + return super().msghdr_size() + 4 + + +# pylint: disable=too-few-public-methods +class SpaceAttrs: + SpecValuesPair = namedtuple('SpecValuesPair', ['spec', 'values']) + + def __init__(self, attr_space, attrs, outer = None): + outer_scopes = outer.scopes if outer else [] + inner_scope = self.SpecValuesPair(attr_space, attrs) + self.scopes = [inner_scope] + outer_scopes + + def lookup(self, name): + for scope in self.scopes: + if name in scope.spec: + if name in scope.values: + return scope.values[name] + spec_name = scope.spec.yaml['name'] + raise YnlException( + f"No value for '{name}' in attribute space '{spec_name}'") + raise YnlException(f"Attribute '{name}' not defined in any attribute-set") + + +# +# YNL implementation details. +# + + +class YnlFamily(SpecFamily): + """ + YNL family -- a Netlink interface built from a YAML spec. + + The spec can be selected either by file path (def_path=) or, when it + ships in a well-known location, by family name (family="xyz"); exactly + one of the two must be given. For example: + + from pyynl import YnlFamily + + ynl = YnlFamily(family="netdev") + + Primary use of the class is to execute Netlink commands: + + ynl.<op_name>(attrs, ...) + + By default this will execute the <op_name> as "do", pass dump=True + to perform a dump operation. + + ynl.<op_name> is a shorthand / convenience wrapper for the following + methods which take the op_name as a string: + + ynl.do(op_name, attrs, flags=None) -- execute a do operation + ynl.dump(op_name, attrs) -- execute a dump operation + ynl.do_multi(ops) -- batch multiple do operations + + The flags argument in ynl.do() allows passing in extra NLM_F_* flags + which may be necessary for old families. + + Notification API: + + ynl.ntf_subscribe(mcast_name) -- join a multicast group + ynl.ntf_listen_all_nsid() -- listen on all netns + ynl.ntf_bind(addr=(0, 0)) -- bind socket for unicast notifications + ynl.check_ntf() -- drain pending notifications + ynl.poll_ntf(duration=None) -- yield notifications + + Policy introspection allows querying validation criteria from the running + kernel. Allows checking whether kernel supports a given attribute or value. + + ynl.get_policy(op_name, mode) -- query kernel policy for an op + """ + def __init__(self, def_path=None, schema=None, process_unknown=None, + recv_size=0, family=None): + super().__init__(def_path, schema, family=family) + + self.include_raw = False + # Specs from /usr (selected by family=) have a higher chance of being + # stale, default to ignoring unknown attrs. In-tree users, and users + # who bundle the spec need to make a conscious decision. + if process_unknown is None: + process_unknown = family is not None + self.process_unknown = process_unknown + + try: + if self.proto == "netlink-raw": + self.nlproto = NetlinkProtocol(self.yaml['name'], + self.yaml['protonum']) + else: + self.nlproto = GenlProtocol(self.yaml['name']) + except KeyError as err: + raise YnlException(f"Family '{self.yaml['name']}' not supported by the kernel") from err + + self._recv_dbg = False + # Note that netlink will use conservative (min) message size for + # the first dump recv() on the socket, our setting will only matter + # from the second recv() on. + self._recv_size = recv_size if recv_size else 131072 + # Netlink will always allocate at least PAGE_SIZE - sizeof(skb_shinfo) + # for a message, so smaller receive sizes will lead to truncation. + # Note that the min size for other families may be larger than 4k! + if self._recv_size < 4000: + raise ConfigError() + + self.sock = socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, self.nlproto.proto_num) + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_CAP_ACK, 1) + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_EXT_ACK, 1) + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_GET_STRICT_CHK, 1) + + self.async_msg_ids = set() + self.async_msg_queue = queue.Queue() + + for msg in self.msgs.values(): + if msg.is_async: + self.async_msg_ids.add(msg.rsp_value) + + for op_name, op in self.ops.items(): + bound_f = functools.partial(self._op, op_name) + setattr(self, op.ident_name, bound_f) + + def close(self): + if self.sock is not None: + self.sock.close() + self.sock = None + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + self.close() + + def ntf_subscribe(self, mcast_name): + mcast_id = self.nlproto.get_mcast_id(mcast_name, self.mcast_groups) + self.sock.bind((0, 0)) + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_ADD_MEMBERSHIP, + mcast_id) + + def ntf_listen_all_nsid(self): + """Enable NETLINK_LISTEN_ALL_NSID to receive notifications from all + namespaces that have an nsid mapped in the current one.""" + self.sock.setsockopt(Netlink.SOL_NETLINK, + Netlink.NETLINK_LISTEN_ALL_NSID, 1) + + @staticmethod + def _decode_nsid(ancdata): + for cmsg_level, cmsg_type, cmsg_data in ancdata: + if (cmsg_level == Netlink.SOL_NETLINK and + cmsg_type == Netlink.NETLINK_LISTEN_ALL_NSID): + nsid = struct.unpack('i', cmsg_data)[0] + if nsid >= 0: + return nsid + return None + return None + + def ntf_bind(self, addr=(0, 0)): + """Bind socket for receiving unicast notifications.""" + self.sock.bind(addr) + + def set_recv_dbg(self, enabled): + self._recv_dbg = enabled + + def _recv_dbg_print(self, reply, nl_msgs): + if not self._recv_dbg: + return + print("Recv: read", len(reply), "bytes,", + len(nl_msgs.msgs), "messages", file=sys.stderr) + for nl_msg in nl_msgs: + print(" ", nl_msg, file=sys.stderr) + + def _encode_enum(self, attr_spec, value): + enum = self.consts[attr_spec['enum']] + if enum.type == 'flags' or attr_spec.get('enum-as-flags', False): + scalar = 0 + if isinstance(value, str): + value = [value] + for single_value in value: + scalar += enum.entries[single_value].user_value(as_flags = True) + return scalar + return enum.entries[value].user_value() + + def _get_scalar(self, attr_spec, value): + try: + return int(value) + except (ValueError, TypeError) as e: + if 'enum' in attr_spec: + return self._encode_enum(attr_spec, value) + if attr_spec.display_hint: + return self._from_string(value, attr_spec) + raise e + + # pylint: disable=too-many-statements + def _add_attr(self, space, name, value, search_attrs): + try: + attr = self.attr_sets[space][name] + except KeyError as err: + raise YnlException(f"Space '{space}' has no attribute '{name}'") from err + nl_type = attr.value + + if attr.is_multi and isinstance(value, list): + attr_payload = b'' + for subvalue in value: + attr_payload += self._add_attr(space, name, subvalue, search_attrs) + return attr_payload + + if attr["type"] == 'nest': + nl_type |= Netlink.NLA_F_NESTED + sub_space = attr['nested-attributes'] + attr_payload = self._add_nest_attrs(value, sub_space, search_attrs) + elif attr['type'] == 'indexed-array' and attr['sub-type'] == 'nest': + nl_type |= Netlink.NLA_F_NESTED + sub_space = attr['nested-attributes'] + attr_payload = self._encode_indexed_array(value, sub_space, + search_attrs) + elif attr["type"] == 'flag': + if not value: + # If value is absent or false then skip attribute creation. + return b'' + attr_payload = b'' + elif attr["type"] == 'string': + attr_payload = str(value).encode('ascii') + b'\x00' + elif attr["type"] == 'binary': + if value is None: + attr_payload = b'' + elif isinstance(value, bytes): + attr_payload = value + elif isinstance(value, str): + if attr.display_hint: + attr_payload = self._from_string(value, attr) + else: + attr_payload = bytes.fromhex(value) + elif isinstance(value, dict) and attr.struct_name: + attr_payload = self._encode_struct(attr.struct_name, value) + elif isinstance(value, list) and attr.sub_type in NlAttr.type_formats: + format_ = NlAttr.get_format(attr.sub_type) + attr_payload = b''.join([format_.pack(x) for x in value]) + else: + raise YnlException(f'Unknown type for binary attribute, value: {value}') + elif attr['type'] in NlAttr.type_formats or attr.is_auto_scalar: + scalar = self._get_scalar(attr, value) + if attr.is_auto_scalar: + attr_type = attr["type"][0] + ('32' if scalar.bit_length() <= 32 else '64') + else: + attr_type = attr["type"] + format_ = NlAttr.get_format(attr_type, attr.byte_order) + attr_payload = format_.pack(scalar) + elif attr['type'] in "bitfield32": + scalar_value = self._get_scalar(attr, value["value"]) + scalar_selector = self._get_scalar(attr, value["selector"]) + attr_payload = struct.pack("II", scalar_value, scalar_selector) + elif attr['type'] == 'sub-message': + msg_format, _ = self._resolve_selector(attr, search_attrs) + attr_payload = b'' + if msg_format.fixed_header: + attr_payload += self._encode_struct(msg_format.fixed_header, value) + if msg_format.attr_set: + if msg_format.attr_set in self.attr_sets: + nl_type |= Netlink.NLA_F_NESTED + sub_attrs = SpaceAttrs(msg_format.attr_set, value, search_attrs) + for subname, subvalue in value.items(): + attr_payload += self._add_attr(msg_format.attr_set, + subname, subvalue, sub_attrs) + else: + raise YnlException(f"Unknown attribute-set '{msg_format.attr_set}'") + else: + raise YnlException(f'Unknown type at {space} {name} {value} {attr["type"]}') + + return self._add_attr_raw(nl_type, attr_payload) + + def _add_attr_raw(self, nl_type, attr_payload): + pad = b'\x00' * ((4 - len(attr_payload) % 4) % 4) + return struct.pack('HH', len(attr_payload) + 4, nl_type) + attr_payload + pad + + def _add_nest_attrs(self, value, sub_space, search_attrs): + sub_attrs = SpaceAttrs(self.attr_sets[sub_space], value, search_attrs) + attr_payload = b'' + for subname, subvalue in value.items(): + attr_payload += self._add_attr(sub_space, subname, subvalue, + sub_attrs) + return attr_payload + + def _encode_indexed_array(self, vals, sub_space, search_attrs): + attr_payload = b'' + for i, val in enumerate(vals): + idx = i | Netlink.NLA_F_NESTED + val_payload = self._add_nest_attrs(val, sub_space, search_attrs) + attr_payload += self._add_attr_raw(idx, val_payload) + return attr_payload + + def _get_enum_or_unknown(self, enum, raw): + try: + name = enum.entries_by_val[raw].name + except KeyError as error: + if self.process_unknown: + name = f"Unknown({raw})" + else: + raise error + return name + + def _decode_enum(self, raw, attr_spec): + enum = self.consts[attr_spec['enum']] + if enum.type == 'flags' or attr_spec.get('enum-as-flags', False): + i = 0 + value = set() + while raw: + if raw & 1: + value.add(self._get_enum_or_unknown(enum, i)) + raw >>= 1 + i += 1 + else: + value = self._get_enum_or_unknown(enum, raw) + return value + + def _decode_binary(self, attr, attr_spec): + if attr_spec.struct_name: + decoded = self._decode_struct(attr.raw, attr_spec.struct_name) + elif attr_spec.sub_type: + decoded = attr.as_c_array(attr_spec.sub_type) + if 'enum' in attr_spec: + decoded = [ self._decode_enum(x, attr_spec) for x in decoded ] + elif attr_spec.display_hint: + decoded = [ self._formatted_string(x, attr_spec.display_hint) + for x in decoded ] + else: + decoded = attr.as_bin() + if attr_spec.display_hint: + decoded = self._formatted_string(decoded, attr_spec.display_hint) + return decoded + + def _decode_array_attr(self, attr, attr_spec): + decoded = [] + offset = 0 + while offset < len(attr.raw): + item = NlAttr(attr.raw, offset) + offset += item.full_len + + if attr_spec["sub-type"] == 'nest': + subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) + decoded.append({ item.type: subattrs }) + elif attr_spec["sub-type"] == 'binary': + subattr = item.as_bin() + if attr_spec.display_hint: + subattr = self._formatted_string(subattr, attr_spec.display_hint) + decoded.append(subattr) + elif attr_spec["sub-type"] in NlAttr.type_formats: + subattr = item.as_scalar(attr_spec['sub-type'], attr_spec.byte_order) + if 'enum' in attr_spec: + subattr = self._decode_enum(subattr, attr_spec) + elif attr_spec.display_hint: + subattr = self._formatted_string(subattr, attr_spec.display_hint) + decoded.append(subattr) + else: + raise YnlException(f'Unknown {attr_spec["sub-type"]} with name {attr_spec["name"]}') + return decoded + + def _decode_nest_type_value(self, attr, attr_spec): + decoded = {} + value = attr + for name in attr_spec['type-value']: + value = NlAttr(value.raw, 0) + decoded[name] = value.type + subattrs = self._decode(NlAttrs(value.raw), attr_spec['nested-attributes']) + decoded.update(subattrs) + return decoded + + def _decode_unknown(self, attr): + if attr.is_nest: + return self._decode(NlAttrs(attr.raw), None) + return attr.as_bin() + + def _rsp_add(self, rsp, name, is_multi, decoded): + if is_multi is None: + if name in rsp and not isinstance(rsp[name], list): + rsp[name] = [rsp[name]] + is_multi = True + else: + is_multi = False + + if not is_multi: + rsp[name] = decoded + elif name in rsp: + rsp[name].append(decoded) + else: + rsp[name] = [decoded] + + def _resolve_selector(self, attr_spec, search_attrs): + sub_msg = attr_spec.sub_message + if sub_msg not in self.sub_msgs: + raise YnlException(f"No sub-message spec named {sub_msg} for {attr_spec.name}") + sub_msg_spec = self.sub_msgs[sub_msg] + + selector = attr_spec.selector + value = search_attrs.lookup(selector) + if value not in sub_msg_spec.formats: + raise YnlException(f"No message format for '{value}' in sub-message spec '{sub_msg}'") + + spec = sub_msg_spec.formats[value] + return spec, value + + def _decode_sub_msg(self, attr, attr_spec, search_attrs): + msg_format, _ = self._resolve_selector(attr_spec, search_attrs) + decoded = {} + offset = 0 + if msg_format.fixed_header: + decoded.update(self._decode_struct(attr.raw, msg_format.fixed_header)) + offset = self.struct_size(msg_format.fixed_header) + if msg_format.attr_set: + if msg_format.attr_set in self.attr_sets: + subdict = self._decode(NlAttrs(attr.raw, offset), msg_format.attr_set) + decoded.update(subdict) + else: + raise YnlException(f"Unknown attribute-set '{msg_format.attr_set}' " + f"when decoding '{attr_spec.name}'") + return decoded + + # pylint: disable=too-many-statements + def _decode(self, attrs, space, outer_attrs = None): + rsp = {} + search_attrs = {} + if space: + attr_space = self.attr_sets[space] + search_attrs = SpaceAttrs(attr_space, rsp, outer_attrs) + + for attr in attrs: + try: + attr_spec = attr_space.attrs_by_val[attr.type] + except (KeyError, UnboundLocalError) as err: + if not self.process_unknown: + raise YnlException(f"Space '{space}' has no attribute " + f"with value '{attr.type}'") from err + attr_name = f"UnknownAttr({attr.type})" + self._rsp_add(rsp, attr_name, None, self._decode_unknown(attr)) + continue + + try: + if attr_spec["type"] == 'pad': + continue + elif attr_spec["type"] == 'nest': + subdict = self._decode(NlAttrs(attr.raw), + attr_spec['nested-attributes'], + search_attrs) + decoded = subdict + elif attr_spec["type"] == 'string': + decoded = attr.as_strz() + elif attr_spec["type"] == 'binary': + decoded = self._decode_binary(attr, attr_spec) + elif attr_spec["type"] == 'flag': + decoded = True + elif attr_spec.is_auto_scalar: + decoded = attr.as_auto_scalar(attr_spec['type'], attr_spec.byte_order) + if 'enum' in attr_spec: + decoded = self._decode_enum(decoded, attr_spec) + elif attr_spec["type"] in NlAttr.type_formats: + decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order) + if 'enum' in attr_spec: + decoded = self._decode_enum(decoded, attr_spec) + elif attr_spec.display_hint: + decoded = self._formatted_string(decoded, attr_spec.display_hint) + elif attr_spec["type"] == 'indexed-array': + decoded = self._decode_array_attr(attr, attr_spec) + elif attr_spec["type"] == 'bitfield32': + value, selector = struct.unpack("II", attr.raw) + if 'enum' in attr_spec: + value = self._decode_enum(value, attr_spec) + selector = self._decode_enum(selector, attr_spec) + decoded = {"value": value, "selector": selector} + elif attr_spec["type"] == 'sub-message': + decoded = self._decode_sub_msg(attr, attr_spec, search_attrs) + elif attr_spec["type"] == 'nest-type-value': + decoded = self._decode_nest_type_value(attr, attr_spec) + else: + if not self.process_unknown: + raise YnlException(f'Unknown {attr_spec["type"]} ' + f'with name {attr_spec["name"]}') + decoded = self._decode_unknown(attr) + + self._rsp_add(rsp, attr_spec["name"], attr_spec.is_multi, decoded) + except: + print(f"Error decoding '{attr_spec.name}' from '{space}'") + raise + + return rsp + + # pylint: disable=too-many-arguments, too-many-positional-arguments + def _decode_extack_path(self, attrs, attr_set, offset, target, search_attrs): + for attr in attrs: + try: + attr_spec = attr_set.attrs_by_val[attr.type] + except KeyError as err: + raise YnlException( + f"Space '{attr_set.name}' has no attribute with value '{attr.type}'") from err + if offset > target: + break + if offset == target: + return '.' + attr_spec.name + + if offset + attr.full_len <= target: + offset += attr.full_len + continue + + pathname = attr_spec.name + if attr_spec['type'] == 'nest': + sub_attrs = self.attr_sets[attr_spec['nested-attributes']] + search_attrs = SpaceAttrs(sub_attrs, search_attrs.lookup(attr_spec['name'])) + elif attr_spec['type'] == 'sub-message': + msg_format, value = self._resolve_selector(attr_spec, search_attrs) + if msg_format is None: + raise YnlException(f"Can't resolve sub-message of " + f"{attr_spec['name']} for extack") + sub_attrs = self.attr_sets[msg_format.attr_set] + pathname += f"({value})" + else: + raise YnlException(f"Can't dive into {attr.type} ({attr_spec['name']}) for extack") + offset += 4 + subpath = self._decode_extack_path(NlAttrs(attr.raw), sub_attrs, + offset, target, search_attrs) + if subpath is None: + return None + return '.' + pathname + subpath + + return None + + def _decode_extack(self, request, op, extack, vals): + if 'bad-attr-offs' not in extack: + return + + msg = self.nlproto.decode(self, NlMsg(request, 0, op.attr_set), op) + offset = self.nlproto.msghdr_size() + self.struct_size(op.fixed_header) + search_attrs = SpaceAttrs(op.attr_set, vals) + path = self._decode_extack_path(msg.raw_attrs, op.attr_set, offset, + extack['bad-attr-offs'], search_attrs) + if path: + del extack['bad-attr-offs'] + extack['bad-attr'] = path + + def struct_size(self, name): + if name: + members = self.consts[name].members + size = 0 + for m in members: + if m.type in ['pad', 'binary']: + if m.struct: + size += self.struct_size(m.struct) + else: + size += m.len + else: + format_ = NlAttr.get_format(m.type, m.byte_order) + size += format_.size + return size + return 0 + + def _decode_struct(self, data, name): + members = self.consts[name].members + attrs = {} + offset = 0 + for m in members: + value = None + if m.type == 'pad': + offset += m.len + elif m.type == 'binary': + if m.struct: + len_ = self.struct_size(m.struct) + value = self._decode_struct(data[offset : offset + len_], + m.struct) + offset += len_ + else: + value = data[offset : offset + m.len] + offset += m.len + else: + format_ = NlAttr.get_format(m.type, m.byte_order) + [ value ] = format_.unpack_from(data, offset) + offset += format_.size + if value is not None: + if m.enum: + value = self._decode_enum(value, m) + elif m.display_hint: + value = self._formatted_string(value, m.display_hint) + attrs[m.name] = value + return attrs + + def _encode_struct(self, name, vals): + members = self.consts[name].members + attr_payload = b'' + for m in members: + value = vals.pop(m.name) if m.name in vals else None + if m.type == 'pad': + attr_payload += bytearray(m.len) + elif m.type == 'binary': + if m.struct: + if value is None: + value = {} + attr_payload += self._encode_struct(m.struct, value) + else: + if value is None: + attr_payload += bytearray(m.len) + else: + attr_payload += bytes.fromhex(value) + else: + if value is None: + value = 0 + format_ = NlAttr.get_format(m.type, m.byte_order) + attr_payload += format_.pack(value) + return attr_payload + + def _formatted_string(self, raw, display_hint): + if display_hint == 'mac': + formatted = ':'.join(f'{b:02x}' for b in raw) + elif display_hint == 'hex': + if isinstance(raw, int): + formatted = hex(raw) + else: + formatted = bytes.hex(raw, ' ') + elif display_hint in [ 'ipv4', 'ipv6', 'ipv4-or-v6' ]: + formatted = format(ipaddress.ip_address(raw)) + elif display_hint == 'uuid': + formatted = str(uuid.UUID(bytes=raw)) + else: + formatted = raw + return formatted + + def _from_string(self, string, attr_spec): + if attr_spec.display_hint in ['ipv4', 'ipv6', 'ipv4-or-v6']: + ip = ipaddress.ip_address(string) + if attr_spec['type'] == 'binary': + raw = ip.packed + else: + raw = int(ip) + elif attr_spec.display_hint == 'hex': + if attr_spec['type'] == 'binary': + raw = bytes.fromhex(string) + else: + raw = int(string, 16) + elif attr_spec.display_hint == 'mac': + # Parse MAC address in format "00:11:22:33:44:55" or "001122334455" + if ':' in string: + mac_bytes = [int(x, 16) for x in string.split(':')] + else: + if len(string) % 2 != 0: + raise YnlException(f"Invalid MAC address format: {string}") + mac_bytes = [int(string[i:i+2], 16) for i in range(0, len(string), 2)] + raw = bytes(mac_bytes) + else: + raise YnlException(f"Display hint '{attr_spec.display_hint}' not implemented" + f" when parsing '{attr_spec['name']}'") + return raw + + def handle_ntf(self, decoded, nsid=None): + msg = {} + if self.include_raw: + msg['raw'] = decoded + op = self.rsp_by_value[decoded.cmd()] + attrs = self._decode(decoded.raw_attrs, op.attr_set.name) + if op.fixed_header: + attrs.update(self._decode_struct(decoded.raw, op.fixed_header)) + + msg['name'] = op['name'] + msg['msg'] = attrs + if nsid is not None: + msg['nsid'] = nsid + self.async_msg_queue.put(msg) + + def _recvmsg(self, flags=0): + reply, ancdata, _, _ = self.sock.recvmsg(self._recv_size, 4096, flags) + return reply, ancdata + + def check_ntf(self): + while True: + try: + reply, ancdata = self._recvmsg(socket.MSG_DONTWAIT) + except BlockingIOError: + return + + nsid = self._decode_nsid(ancdata) + nms = NlMsgs(reply) + self._recv_dbg_print(reply, nms) + for nl_msg in nms: + if nl_msg.error: + print("Netlink error in ntf!?", os.strerror(-nl_msg.error)) + print(nl_msg) + continue + if nl_msg.done: + print("Netlink done while checking for ntf!?") + continue + + decoded = self.nlproto.decode(self, nl_msg, None) + if decoded.cmd() not in self.async_msg_ids: + print("Unexpected msg id while checking for ntf", decoded) + continue + + self.handle_ntf(decoded, nsid) + + def poll_ntf(self, duration=None): + start_time = time.time() + selector = selectors.DefaultSelector() + selector.register(self.sock, selectors.EVENT_READ) + + while True: + try: + yield self.async_msg_queue.get_nowait() + except queue.Empty: + if duration is not None: + timeout = start_time + duration - time.time() + if timeout <= 0: + return + else: + timeout = None + events = selector.select(timeout) + if events: + self.check_ntf() + + def operation_do_attributes(self, name): + """ + For a given operation name, find and return a supported + set of attributes (as a dict). + """ + op = self.find_operation(name) + if not op: + return None + + return op['do']['request']['attributes'].copy() + + def _encode_message(self, op, vals, flags, req_seq): + nl_flags = Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK + for flag in flags or []: + nl_flags |= flag + + msg = self.nlproto.message(nl_flags, op.req_value, 1, req_seq) + if op.fixed_header: + msg += self._encode_struct(op.fixed_header, vals) + search_attrs = SpaceAttrs(op.attr_set, vals) + for name, value in vals.items(): + msg += self._add_attr(op.attr_set.name, name, value, search_attrs) + msg = _genl_msg_finalize(msg) + return msg + + # pylint: disable=too-many-statements + def _ops(self, ops): + reqs_by_seq = {} + req_seq = random.randint(1024, 65535) + payload = b'' + for (method, vals, flags) in ops: + op = self.ops[method] + msg = self._encode_message(op, vals, flags, req_seq) + reqs_by_seq[req_seq] = (op, vals, msg, flags) + payload += msg + req_seq += 1 + + self.sock.send(payload, 0) + + done = False + rsp = [] + op_rsp = [] + while not done: + reply, ancdata = self._recvmsg() + nsid = self._decode_nsid(ancdata) + nms = NlMsgs(reply) + self._recv_dbg_print(reply, nms) + for nl_msg in nms: + if nl_msg.nl_seq in reqs_by_seq: + (op, vals, req_msg, req_flags) = reqs_by_seq[nl_msg.nl_seq] + if nl_msg.extack: + nl_msg.annotate_extack(op.attr_set) + self._decode_extack(req_msg, op, nl_msg.extack, vals) + else: + op = None + req_flags = [] + + if nl_msg.error: + raise NlError(nl_msg) + if nl_msg.done: + if nl_msg.extack: + print("Netlink warning:") + print(nl_msg) + + if Netlink.NLM_F_DUMP in req_flags: + rsp.append(op_rsp) + elif not op_rsp: + rsp.append(None) + elif len(op_rsp) == 1: + rsp.append(op_rsp[0]) + else: + rsp.append(op_rsp) + op_rsp = [] + + del reqs_by_seq[nl_msg.nl_seq] + done = len(reqs_by_seq) == 0 + break + + decoded = self.nlproto.decode(self, nl_msg, op) + + # Check if this is a reply to our request + if nl_msg.nl_seq not in reqs_by_seq or decoded.cmd() != op.rsp_value: + if decoded.cmd() in self.async_msg_ids: + self.handle_ntf(decoded, nsid) + continue + print('Unexpected message: ' + repr(decoded)) + continue + + rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name) + if op.fixed_header: + rsp_msg.update(self._decode_struct(decoded.raw, op.fixed_header)) + op_rsp.append(rsp_msg) + + return rsp + + def _op(self, method, vals, flags=None, dump=False): + req_flags = flags or [] + if dump: + req_flags.append(Netlink.NLM_F_DUMP) + + ops = [(method, vals, req_flags)] + return self._ops(ops)[0] + + def do(self, method, vals, flags=None): + return self._op(method, vals, flags) + + def dump(self, method, vals): + return self._op(method, vals, dump=True) + + def do_multi(self, ops): + return self._ops(ops) + + def get_policy(self, op_name, mode): + """Query running kernel for the Netlink policy of an operation. + + Allows checking whether kernel supports a given attribute or value. + This method consults the running kernel, not the YAML spec. + + Args: + op_name: operation name as it appears in the YAML spec + mode: 'do' or 'dump' + + Returns: + NlPolicy acting as a read-only dict mapping attribute names + to their policy properties (type, min/max, nested, etc.), + or None if the operation has no policy for the given mode. + Empty policy usually implies that the operation rejects + all attributes. + """ + op = self.ops[op_name] + op_policy, policy_table = _genl_policy_dump(self.nlproto.family_id, + op.req_value) + if mode not in op_policy: + return None + policy_idx = op_policy[mode] + return NlPolicy(self, policy_idx, policy_table, op.attr_set) diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py new file mode 100755 index 000000000..2b3483db1 --- /dev/null +++ b/tools/net/ynl/pyynl/ynl_gen_c.py @@ -0,0 +1,3780 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) +# +# pylint: disable=line-too-long, missing-class-docstring, missing-function-docstring +# pylint: disable=too-many-positional-arguments, too-many-arguments, too-many-statements +# pylint: disable=too-many-branches, too-many-locals, too-many-instance-attributes +# pylint: disable=too-many-nested-blocks, too-many-lines, too-few-public-methods +# pylint: disable=broad-exception-raised, broad-exception-caught, protected-access + +""" +ynl_gen_c + +A YNL to C code generator for both kernel and userspace protocol stubs. +""" + +import argparse +import filecmp +import pathlib +import os +import re +import shutil +import sys +import tempfile +import yaml as pyyaml + +# pylint: disable=no-name-in-module,wrong-import-position +sys.path.append(pathlib.Path(__file__).resolve().parent.as_posix()) +from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, SpecEnumEntry +from lib import SpecSubMessage + + +def c_upper(name): + return name.upper().replace('-', '_') + + +def c_lower(name): + return name.lower().replace('-', '_') + + +def limit_to_number(name): + """ + Turn a string limit like u32-max or s64-min into its numerical value + """ + if name[0] == 'u' and name.endswith('-min'): + return 0 + width = int(name[1:-4]) + if name[0] == 's': + width -= 1 + value = (1 << width) - 1 + if name[0] == 's' and name.endswith('-min'): + value = -value - 1 + return value + + +class BaseNlLib: + def get_family_id(self): + return 'ys->family_id' + + +class Type(SpecAttr): + def __init__(self, family, attr_set, attr, value): + super().__init__(family, attr_set, attr, value) + + self.attr = attr + self.attr_set = attr_set + self.type = attr['type'] + self.checks = attr.get('checks', {}) + + self.request = False + self.reply = False + + self.is_selector = False + + if 'len' in attr: + self.len = attr['len'] + + if 'nested-attributes' in attr: + nested = attr['nested-attributes'] + elif 'sub-message' in attr: + nested = attr['sub-message'] + else: + nested = None + + if nested: + self.nested_attrs = nested + if self.nested_attrs == family.name: + self.nested_render_name = c_lower(f"{family.ident_name}") + else: + self.nested_render_name = c_lower(f"{family.ident_name}_{self.nested_attrs}") + + if self.nested_attrs in self.family.consts: + self.nested_struct_type = 'struct ' + self.nested_render_name + '_' + else: + self.nested_struct_type = 'struct ' + self.nested_render_name + + self.c_name = c_lower(self.name) + if self.c_name in _C_KW: + self.c_name += '_' + if self.c_name[0].isdigit(): + self.c_name = '_' + self.c_name + + # Added by resolve(): + self.enum_name = None + delattr(self, "enum_name") + + def _get_real_attr(self): + # if the attr is for a subset return the "real" attr (just one down, does not recurse) + return self.family.attr_sets[self.attr_set.subset_of][self.name] + + def set_request(self): + self.request = True + if self.attr_set.subset_of: + self._get_real_attr().set_request() + + def set_reply(self): + self.reply = True + if self.attr_set.subset_of: + self._get_real_attr().set_reply() + + def get_limit(self, limit, default=None): + value = self.checks.get(limit, default) + if value is None: + return value + if isinstance(value, int): + return value + if value in self.family.consts: + return self.family.consts[value]["value"] + return limit_to_number(value) + + def get_limit_str(self, limit, default=None, suffix=''): + value = self.checks.get(limit, default) + if value is None: + return '' + if isinstance(value, int): + return str(value) + suffix + if value in self.family.consts: + const = self.family.consts[value] + if const.get('header'): + return c_upper(value) + return c_upper(f"{self.family['name']}-{value}") + return c_upper(value) + + def resolve(self): + if 'parent-sub-message' in self.attr: + enum_name = self.attr['parent-sub-message'].enum_name + elif 'name-prefix' in self.attr: + enum_name = f"{self.attr['name-prefix']}{self.name}" + else: + enum_name = f"{self.attr_set.name_prefix}{self.name}" + self.enum_name = c_upper(enum_name) + + if self.attr_set.subset_of: + if self.checks != self._get_real_attr().checks: + raise Exception("Overriding checks not supported by codegen, yet") + + def is_multi_val(self): + return None + + def is_scalar(self): + return self.type in {'u8', 'u16', 'u32', 'u64', 's32', 's64'} + + def is_recursive(self): + return False + + def is_recursive_for_op(self, ri): + return self.is_recursive() and not ri.op + + def presence_type(self): + return 'present' + + def presence_member(self, space, type_filter): + if self.presence_type() != type_filter: + return '' + + if self.presence_type() == 'present': + pfx = '__' if space == 'user' else '' + return f"{pfx}u32 {self.c_name}:1;" + + if self.presence_type() in {'len', 'count'}: + pfx = '__' if space == 'user' else '' + return f"{pfx}u32 {self.c_name};" + return '' + + def _complex_member_type(self, _ri): + return None + + def free_needs_iter(self): + return False + + def _free_lines(self, _ri, var, ref): + if self.is_multi_val() or self.presence_type() in {'count', 'len'}: + return [f'free({var}->{ref}{self.c_name});'] + return [] + + def free(self, ri, var, ref): + lines = self._free_lines(ri, var, ref) + for line in lines: + ri.cw.p(line) + + # pylint: disable=assignment-from-none + def arg_member(self, ri): + member = self._complex_member_type(ri) + if member is not None: + spc = ' ' if member[-1] != '*' else '' + arg = [member + spc + '*' + self.c_name] + if self.presence_type() == 'count': + arg += ['unsigned int n_' + self.c_name] + return arg + raise Exception(f"Struct member not implemented for class type {self.type}") + + def struct_member(self, ri): + member = self._complex_member_type(ri) + if member is not None: + ptr = '*' if self.is_multi_val() else '' + if self.is_recursive_for_op(ri): + ptr = '*' + spc = ' ' if member[-1] != '*' else '' + ri.cw.p(f"{member}{spc}{ptr}{self.c_name};") + return + members = self.arg_member(ri) + for one in members: + ri.cw.p(one + ';') + + def _attr_policy(self, policy): + return '{ .type = ' + policy + ', }' + + def attr_policy(self, cw): + policy = f'NLA_{c_upper(self.type)}' + if self.attr.get('byte-order') == 'big-endian': + if self.type in {'u16', 'u32'}: + policy = f'NLA_BE{self.type[1:]}' + + spec = self._attr_policy(policy) + cw.p(f"\t[{self.enum_name}] = {spec},") + + def _attr_typol(self): + raise Exception(f"Type policy not implemented for class type {self.type}") + + def attr_typol(self, cw): + typol = self._attr_typol() + cw.p(f'[{self.enum_name}] = {"{"} .name = "{self.name}", {typol}{"}"},') + + def _attr_put_line(self, ri, var, line): + presence = self.presence_type() + if presence in {'present', 'len'}: + ri.cw.p(f"if ({var}->_{presence}.{self.c_name})") + ri.cw.p(f"{line};") + + def _attr_put_simple(self, ri, var, put_type): + line = f"ynl_attr_put_{put_type}(nlh, {self.enum_name}, {var}->{self.c_name})" + self._attr_put_line(ri, var, line) + + def attr_put(self, ri, var): + raise Exception(f"Put not implemented for class type {self.type}") + + def _attr_get(self, ri, var): + raise Exception(f"Attr get not implemented for class type {self.type}") + + def attr_get(self, ri, var, first): + lines, init_lines, _ = self._attr_get(ri, var) + if isinstance(lines, str): + lines = [lines] + if isinstance(init_lines, str): + init_lines = [init_lines] + + kw = 'if' if first else 'else if' + ri.cw.block_start(line=f"{kw} (type == {self.enum_name})") + + if not self.is_multi_val(): + ri.cw.p("if (ynl_attr_validate(yarg, attr))") + ri.cw.p("return YNL_PARSE_CB_ERROR;") + if self.presence_type() == 'present': + ri.cw.p(f"{var}->_present.{self.c_name} = 1;") + + if init_lines: + ri.cw.nl() + for line in init_lines: + ri.cw.p(line) + + for line in lines: + ri.cw.p(line) + ri.cw.block_end() + return True + + def _setter_lines(self, ri, member, presence): + raise Exception(f"Setter not implemented for class type {self.type}") + + def setter(self, ri, _space, direction, deref=False, ref=None, var="req"): + ref = (ref if ref else []) + [self.c_name] + member = f"{var}->{'.'.join(ref)}" + + local_vars = [] + if self.free_needs_iter(): + local_vars += ['unsigned int i;'] + + code = [] + presence = '' + # pylint: disable=consider-using-enumerate + for i in range(0, len(ref)): + presence = f"{var}->{'.'.join(ref[:i] + [''])}_present.{ref[i]}" + # Every layer below last is a nest, so we know it uses bit presence + # last layer is "self" and may be a complex type + if i == len(ref) - 1 and self.presence_type() != 'present': + presence = f"{var}->{'.'.join(ref[:i] + [''])}_{self.presence_type()}.{ref[i]}" + continue + code.append(presence + ' = 1;') + ref_path = '.'.join(ref[:-1]) + if ref_path: + ref_path += '.' + code += self._free_lines(ri, var, ref_path) + code += self._setter_lines(ri, member, presence) + + func_name = f"{op_prefix(ri, direction, deref=deref)}_set_{'_'.join(ref)}" + free = bool([x for x in code if 'free(' in x]) + alloc = bool([x for x in code if 'alloc(' in x]) + if free and not alloc: + func_name = '__' + func_name + ri.cw.write_func('static inline void', func_name, local_vars=local_vars, + body=code, + args=[f'{type_name(ri, direction, deref=deref)} *{var}'] + self.arg_member(ri)) + + +class TypeUnused(Type): + def presence_type(self): + return '' + + def arg_member(self, ri): + return [] + + def _attr_get(self, ri, var): + return ['return YNL_PARSE_CB_ERROR;'], None, None + + def _attr_typol(self): + return '.type = YNL_PT_REJECT, ' + + def attr_policy(self, cw): + pass + + def attr_put(self, ri, var): + pass + + def attr_get(self, ri, var, first): + pass + + def setter(self, ri, space, direction, deref=False, ref=None, var=None): + pass + + +class TypePad(Type): + def presence_type(self): + return '' + + def arg_member(self, ri): + return [] + + def _attr_typol(self): + return '.type = YNL_PT_IGNORE, ' + + def attr_put(self, ri, var): + pass + + def attr_get(self, ri, var, first): + pass + + def attr_policy(self, cw): + pass + + def setter(self, ri, space, direction, deref=False, ref=None, var=None): + pass + + +class TypeScalar(Type): + def __init__(self, family, attr_set, attr, value): + super().__init__(family, attr_set, attr, value) + + self.byte_order_comment = '' + if 'byte-order' in attr: + self.byte_order_comment = f" /* {attr['byte-order']} */" + + # Classic families have some funny enums, don't bother + # computing checks, since we only need them for kernel policies + if not family.is_classic(): + self._init_checks() + + # Added by resolve(): + self.is_bitfield = None + delattr(self, "is_bitfield") + self.type_name = None + delattr(self, "type_name") + + def resolve(self): + self.resolve_up(super()) + + if 'enum-as-flags' in self.attr and self.attr['enum-as-flags']: + self.is_bitfield = True + elif 'enum' in self.attr: + self.is_bitfield = self.family.consts[self.attr['enum']]['type'] == 'flags' + else: + self.is_bitfield = False + + if not self.is_bitfield and 'enum' in self.attr: + self.type_name = self.family.consts[self.attr['enum']].user_type + elif self.is_auto_scalar: + self.type_name = '__' + self.type[0] + '64' + else: + self.type_name = '__' + self.type + + def _init_checks(self): + if 'enum' in self.attr: + enum = self.family.consts[self.attr['enum']] + low, high = enum.value_range() + if low is None and high is None: + self.checks['sparse'] = True + else: + if 'min' not in self.checks: + if low != 0 or self.type[0] == 's': + self.checks['min'] = low + if 'max' not in self.checks: + self.checks['max'] = high + + if 'min' in self.checks and 'max' in self.checks: + if self.get_limit('min') > self.get_limit('max'): + raise Exception(f'Invalid limit for "{self.name}" min: {self.get_limit("min")} max: {self.get_limit("max")}') + self.checks['range'] = True + + low = min(self.get_limit('min', 0), self.get_limit('max', 0)) + high = max(self.get_limit('min', 0), self.get_limit('max', 0)) + if low < 0 and self.type[0] == 'u': + raise Exception(f'Invalid limit for "{self.name}" negative limit for unsigned type') + if low < -32768 or high > 32767: + self.checks['full-range'] = True + + # pylint: disable=too-many-return-statements + def _attr_policy(self, policy): + if 'flags-mask' in self.checks or self.is_bitfield: + if self.is_bitfield: + enum = self.family.consts[self.attr['enum']] + mask = enum.get_mask(as_flags=True) + else: + flags = self.family.consts[self.checks['flags-mask']] + flag_cnt = len(flags['entries']) + mask = (1 << flag_cnt) - 1 + return f"NLA_POLICY_MASK({policy}, 0x{mask:x})" + if 'full-range' in self.checks: + return f"NLA_POLICY_FULL_RANGE({policy}, &{c_lower(self.enum_name)}_range)" + if 'range' in self.checks: + return f"NLA_POLICY_RANGE({policy}, {self.get_limit_str('min')}, {self.get_limit_str('max')})" + if 'min' in self.checks: + return f"NLA_POLICY_MIN({policy}, {self.get_limit_str('min')})" + if 'max' in self.checks: + return f"NLA_POLICY_MAX({policy}, {self.get_limit_str('max')})" + if 'sparse' in self.checks: + return f"NLA_POLICY_VALIDATE_FN({policy}, &{c_lower(self.enum_name)}_validate)" + return super()._attr_policy(policy) + + def _attr_typol(self): + return f'.type = YNL_PT_U{c_upper(self.type[1:])}, ' + + def arg_member(self, ri): + return [f'{self.type_name} {self.c_name}{self.byte_order_comment}'] + + def attr_put(self, ri, var): + self._attr_put_simple(ri, var, self.type) + + def _attr_get(self, ri, var): + return f"{var}->{self.c_name} = ynl_attr_get_{self.type}(attr);", None, None + + def _setter_lines(self, ri, member, presence): + return [f"{member} = {self.c_name};"] + + +class TypeFlag(Type): + def arg_member(self, ri): + return [] + + def _attr_typol(self): + return '.type = YNL_PT_FLAG, ' + + def attr_put(self, ri, var): + self._attr_put_line(ri, var, f"ynl_attr_put(nlh, {self.enum_name}, NULL, 0)") + + def _attr_get(self, ri, var): + return [], None, None + + def _setter_lines(self, ri, member, presence): + return [] + + +class TypeString(Type): + def arg_member(self, ri): + return [f"const char *{self.c_name}"] + + def presence_type(self): + return 'len' + + def struct_member(self, ri): + ri.cw.p(f"char *{self.c_name};") + + def _attr_typol(self): + typol = '.type = YNL_PT_NUL_STR, ' + if self.is_selector: + typol += '.is_selector = 1, ' + return typol + + def _attr_policy(self, policy): + if 'exact-len' in self.checks: + mem = 'NLA_POLICY_EXACT_LEN(' + self.get_limit_str('exact-len') + ')' + else: + mem = '{ .type = ' + policy + if 'max-len' in self.checks: + mem += ', .len = ' + self.get_limit_str('max-len') + mem += ', }' + return mem + + def attr_policy(self, cw): + if self.checks.get('unterminated-ok', False): + policy = 'NLA_STRING' + else: + policy = 'NLA_NUL_STRING' + + spec = self._attr_policy(policy) + cw.p(f"\t[{self.enum_name}] = {spec},") + + def attr_put(self, ri, var): + self._attr_put_simple(ri, var, 'str') + + def _attr_get(self, ri, var): + len_mem = var + '->_len.' + self.c_name + return [f"{var}->{self.c_name} = malloc(len + 1);", + f"if (!{var}->{self.c_name})", + "return YNL_PARSE_CB_ERROR;", + f"{len_mem} = len;", + f"memcpy({var}->{self.c_name}, ynl_attr_get_str(attr), len);", + f"{var}->{self.c_name}[len] = 0;"], \ + ['len = strnlen(ynl_attr_get_str(attr), ynl_attr_data_len(attr));'], \ + ['unsigned int len;'] + + def _setter_lines(self, ri, member, presence): + return [f"{presence} = strlen({self.c_name});", + f"{member} = malloc({presence} + 1);", + f'memcpy({member}, {self.c_name}, {presence});', + f'{member}[{presence}] = 0;'] + + +class TypeBinary(Type): + def arg_member(self, ri): + return [f"const void *{self.c_name}", 'size_t len'] + + def presence_type(self): + return 'len' + + def struct_member(self, ri): + ri.cw.p(f"void *{self.c_name};") + + def _attr_typol(self): + return '.type = YNL_PT_BINARY,' + + def _attr_policy(self, policy): + if len(self.checks) == 0: + pass + elif len(self.checks) == 1: + check_name = list(self.checks)[0] + if check_name not in {'exact-len', 'min-len', 'max-len'}: + raise Exception('Unsupported check for binary type: ' + check_name) + else: + raise Exception('More than one check for binary type not implemented, yet') + + if len(self.checks) == 0: + mem = '{ .type = NLA_BINARY, }' + elif 'exact-len' in self.checks: + mem = 'NLA_POLICY_EXACT_LEN(' + self.get_limit_str('exact-len') + ')' + elif 'min-len' in self.checks: + mem = 'NLA_POLICY_MIN_LEN(' + self.get_limit_str('min-len') + ')' + elif 'max-len' in self.checks: + mem = 'NLA_POLICY_MAX_LEN(' + self.get_limit_str('max-len') + ')' + else: + raise Exception('Failed to process policy check for binary type') + + return mem + + def attr_put(self, ri, var): + self._attr_put_line(ri, var, f"ynl_attr_put(nlh, {self.enum_name}, " + + f"{var}->{self.c_name}, {var}->_len.{self.c_name})") + + def _attr_get(self, ri, var): + len_mem = var + '->_len.' + self.c_name + return [f"{var}->{self.c_name} = malloc(len);", + f"if (!{var}->{self.c_name})", + "return YNL_PARSE_CB_ERROR;", + f"{len_mem} = len;", + f"memcpy({var}->{self.c_name}, ynl_attr_data(attr), len);"], \ + ['len = ynl_attr_data_len(attr);'], \ + ['unsigned int len;'] + + def _setter_lines(self, ri, member, presence): + return [f"{presence} = len;", + f"{member} = malloc({presence});", + f'memcpy({member}, {self.c_name}, {presence});'] + + +class TypeBinaryStruct(TypeBinary): + def struct_member(self, ri): + ri.cw.p(f'struct {c_lower(self.get("struct"))} *{self.c_name};') + + def _attr_get(self, ri, var): + struct_sz = 'sizeof(struct ' + c_lower(self.get("struct")) + ')' + len_mem = var + '->_' + self.presence_type() + '.' + self.c_name + return [f"if (len < {struct_sz})", + f"{var}->{self.c_name} = calloc(1, {struct_sz});", + "else", + f"{var}->{self.c_name} = malloc(len);", + f"if (!{var}->{self.c_name})", + "return YNL_PARSE_CB_ERROR;", + f"{len_mem} = len;", + f"memcpy({var}->{self.c_name}, ynl_attr_data(attr), len);"], \ + ['len = ynl_attr_data_len(attr);'], \ + ['unsigned int len;'] + + +class TypeBinaryScalarArray(TypeBinary): + def arg_member(self, ri): + return [f'__{self.get("sub-type")} *{self.c_name}', 'size_t count'] + + def presence_type(self): + return 'count' + + def struct_member(self, ri): + ri.cw.p(f'__{self.get("sub-type")} *{self.c_name};') + + def attr_put(self, ri, var): + presence = self.presence_type() + ri.cw.block_start(line=f"if ({var}->_{presence}.{self.c_name})") + ri.cw.p(f"i = {var}->_{presence}.{self.c_name} * sizeof(__{self.get('sub-type')});") + ri.cw.p(f"ynl_attr_put(nlh, {self.enum_name}, " + + f"{var}->{self.c_name}, i);") + ri.cw.block_end() + + def _attr_get(self, ri, var): + len_mem = var + '->_count.' + self.c_name + return [f"len = (len / sizeof(__{self.get('sub-type')})) * sizeof(__{self.get('sub-type')});", + f"{var}->{self.c_name} = malloc(len);", + f"if (!{var}->{self.c_name})", + "return YNL_PARSE_CB_ERROR;", + f"{len_mem} = len / sizeof(__{self.get('sub-type')});", + f"memcpy({var}->{self.c_name}, ynl_attr_data(attr), len);"], \ + ['len = ynl_attr_data_len(attr);'], \ + ['unsigned int len;'] + + def _setter_lines(self, ri, member, presence): + return [f"{presence} = count;", + f"count *= sizeof(__{self.get('sub-type')});", + f"{member} = malloc(count);", + f'memcpy({member}, {self.c_name}, count);'] + + +class TypeBitfield32(Type): + def _complex_member_type(self, _ri): + return "struct nla_bitfield32" + + def _attr_typol(self): + return '.type = YNL_PT_BITFIELD32, ' + + def _attr_policy(self, policy): + if 'enum' not in self.attr: + raise Exception('Enum required for bitfield32 attr') + enum = self.family.consts[self.attr['enum']] + mask = enum.get_mask(as_flags=True) + return f"NLA_POLICY_BITFIELD32({mask})" + + def attr_put(self, ri, var): + line = f"ynl_attr_put(nlh, {self.enum_name}, &{var}->{self.c_name}, sizeof(struct nla_bitfield32))" + self._attr_put_line(ri, var, line) + + def _attr_get(self, ri, var): + return f"memcpy(&{var}->{self.c_name}, ynl_attr_data(attr), sizeof(struct nla_bitfield32));", None, None + + def _setter_lines(self, ri, member, presence): + return [f"memcpy(&{member}, {self.c_name}, sizeof(struct nla_bitfield32));"] + + +class TypeNest(Type): + def is_recursive(self): + return self.family.pure_nested_structs[self.nested_attrs].recursive + + def _complex_member_type(self, _ri): + return self.nested_struct_type + + def _free_lines(self, ri, var, ref): + lines = [] + at = '&' + if self.is_recursive_for_op(ri): + at = '' + lines += [f'if ({var}->{ref}{self.c_name})'] + lines += [f'{self.nested_render_name}_free({at}{var}->{ref}{self.c_name});'] + return lines + + def _attr_typol(self): + return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, ' + + def _attr_policy(self, policy): + return 'NLA_POLICY_NESTED(' + self.nested_render_name + '_nl_policy)' + + def attr_put(self, ri, var): + at = '' if self.is_recursive_for_op(ri) else '&' + self._attr_put_line(ri, var, f"{self.nested_render_name}_put(nlh, " + + f"{self.enum_name}, {at}{var}->{self.c_name})") + + def _attr_get(self, ri, var): + pns = self.family.pure_nested_structs[self.nested_attrs] + args = ["&parg", "attr"] + for sel in pns.external_selectors(): + args.append(f'{var}->{sel.name}') + get_lines = [f"if ({self.nested_render_name}_parse({', '.join(args)}))", + "return YNL_PARSE_CB_ERROR;"] + init_lines = [f"parg.rsp_policy = &{self.nested_render_name}_nest;", + f"parg.data = &{var}->{self.c_name};"] + return get_lines, init_lines, None + + def setter(self, ri, _space, direction, deref=False, ref=None, var="req"): + ref = (ref if ref else []) + [self.c_name] + + for _, attr in ri.family.pure_nested_structs[self.nested_attrs].member_list(): + if attr.is_recursive(): + continue + attr.setter(ri, self.nested_attrs, direction, deref=deref, ref=ref, + var=var) + + +class TypeMultiAttr(Type): + def __init__(self, family, attr_set, attr, value, base_type): + super().__init__(family, attr_set, attr, value) + + self.base_type = base_type + + def is_multi_val(self): + return True + + def presence_type(self): + return 'count' + + def _complex_member_type(self, ri): + if 'type' not in self.attr or self.attr['type'] == 'nest': + return self.nested_struct_type + if self.attr['type'] == 'binary' and 'struct' in self.attr: + return None # use arg_member() + if self.attr['type'] == 'string': + return 'struct ynl_string *' + if self.attr['type'] in scalars: + scalar_pfx = '__' if ri.ku_space == 'user' else '' + if self.is_auto_scalar: + name = self.type[0] + '64' + else: + name = self.attr['type'] + return scalar_pfx + name + raise Exception(f"Sub-type {self.attr['type']} not supported yet") + + def arg_member(self, ri): + if self.type == 'binary' and 'struct' in self.attr: + return [f'struct {c_lower(self.attr["struct"])} *{self.c_name}', + f'unsigned int n_{self.c_name}'] + return super().arg_member(ri) + + def free_needs_iter(self): + return self.attr['type'] in {'nest', 'string'} + + def _free_lines(self, _ri, var, ref): + lines = [] + if self.attr['type'] in scalars: + lines += [f"free({var}->{ref}{self.c_name});"] + elif self.attr['type'] == 'binary': + lines += [f"free({var}->{ref}{self.c_name});"] + elif self.attr['type'] == 'string': + lines += [ + f"for (i = 0; i < {var}->{ref}_count.{self.c_name}; i++)", + f"free({var}->{ref}{self.c_name}[i]);", + f"free({var}->{ref}{self.c_name});", + ] + elif 'type' not in self.attr or self.attr['type'] == 'nest': + lines += [ + f"for (i = 0; i < {var}->{ref}_count.{self.c_name}; i++)", + f'{self.nested_render_name}_free(&{var}->{ref}{self.c_name}[i]);', + f"free({var}->{ref}{self.c_name});", + ] + else: + raise Exception(f"Free of MultiAttr sub-type {self.attr['type']} not supported yet") + return lines + + def _attr_policy(self, policy): + return self.base_type._attr_policy(policy) + + def _attr_typol(self): + return self.base_type._attr_typol() + + def _attr_get(self, ri, var): + return f'n_{self.c_name}++;', None, None + + def attr_put(self, ri, var): + if self.attr['type'] in scalars: + put_type = self.type + ri.cw.p(f"for (i = 0; i < {var}->_count.{self.c_name}; i++)") + ri.cw.p(f"ynl_attr_put_{put_type}(nlh, {self.enum_name}, {var}->{self.c_name}[i]);") + elif self.attr['type'] == 'binary' and 'struct' in self.attr: + ri.cw.p(f"for (i = 0; i < {var}->_count.{self.c_name}; i++)") + ri.cw.p(f"ynl_attr_put(nlh, {self.enum_name}, &{var}->{self.c_name}[i], sizeof(struct {c_lower(self.attr['struct'])}));") + elif self.attr['type'] == 'string': + ri.cw.p(f"for (i = 0; i < {var}->_count.{self.c_name}; i++)") + ri.cw.p(f"ynl_attr_put_str(nlh, {self.enum_name}, {var}->{self.c_name}[i]->str);") + elif 'type' not in self.attr or self.attr['type'] == 'nest': + ri.cw.p(f"for (i = 0; i < {var}->_count.{self.c_name}; i++)") + self._attr_put_line(ri, var, f"{self.nested_render_name}_put(nlh, " + + f"{self.enum_name}, &{var}->{self.c_name}[i])") + else: + raise Exception(f"Put of MultiAttr sub-type {self.attr['type']} not supported yet") + + def _setter_lines(self, ri, member, presence): + return [f"{member} = {self.c_name};", + f"{presence} = n_{self.c_name};"] + + +class TypeIndexedArray(Type): + def is_multi_val(self): + return True + + def presence_type(self): + return 'count' + + def _complex_member_type(self, ri): + if 'sub-type' not in self.attr or self.attr['sub-type'] == 'nest': + return self.nested_struct_type + if self.attr['sub-type'] in scalars: + scalar_pfx = '__' if ri.ku_space == 'user' else '' + return scalar_pfx + self.attr['sub-type'] + if self.attr['sub-type'] == 'binary' and 'exact-len' in self.checks: + return None # use arg_member() + raise Exception(f"Sub-type {self.attr['sub-type']} not supported yet") + + def arg_member(self, ri): + if self.sub_type == 'binary' and 'exact-len' in self.checks: + return [f'unsigned char (*{self.c_name})[{self.checks["exact-len"]}]', + f'unsigned int n_{self.c_name}'] + return super().arg_member(ri) + + def _attr_policy(self, policy): + if self.attr['sub-type'] == 'nest': + return f'NLA_POLICY_NESTED_ARRAY({self.nested_render_name}_nl_policy)' + return super()._attr_policy(policy) + + def _attr_typol(self): + if self.attr['sub-type'] in scalars: + return f'.type = YNL_PT_U{c_upper(self.sub_type[1:])}, ' + if self.attr['sub-type'] == 'binary' and 'exact-len' in self.checks: + return f'.type = YNL_PT_BINARY, .len = {self.checks["exact-len"]}, ' + if self.attr['sub-type'] == 'nest': + return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, ' + raise Exception(f"Typol for IndexedArray sub-type {self.attr['sub-type']} not supported, yet") + + def _attr_get(self, ri, var): + local_vars = ['const struct nlattr *attr2;'] + get_lines = [f'attr_{self.c_name} = attr;', + 'ynl_attr_for_each_nested(attr2, attr) {', + '\tif (__ynl_attr_validate(yarg, attr2, type))', + '\t\treturn YNL_PARSE_CB_ERROR;', + f'\tn_{self.c_name}++;', + '}'] + return get_lines, None, local_vars + + def attr_put(self, ri, var): + ri.cw.p(f'array = ynl_attr_nest_start(nlh, {self.enum_name});') + if self.sub_type in scalars: + put_type = self.sub_type + ri.cw.block_start(line=f'for (i = 0; i < {var}->_count.{self.c_name}; i++)') + ri.cw.p(f"ynl_attr_put_{put_type}(nlh, i, {var}->{self.c_name}[i]);") + ri.cw.block_end() + elif self.sub_type == 'binary' and 'exact-len' in self.checks: + ri.cw.p(f'for (i = 0; i < {var}->_count.{self.c_name}; i++)') + ri.cw.p(f"ynl_attr_put(nlh, i, {var}->{self.c_name}[i], {self.checks['exact-len']});") + elif self.sub_type == 'nest': + ri.cw.p(f'for (i = 0; i < {var}->_count.{self.c_name}; i++)') + ri.cw.p(f"{self.nested_render_name}_put(nlh, i, &{var}->{self.c_name}[i]);") + else: + raise Exception(f"Put for IndexedArray sub-type {self.attr['sub-type']} not supported, yet") + ri.cw.p('ynl_attr_nest_end(nlh, array);') + + def _setter_lines(self, ri, member, presence): + return [f"{member} = {self.c_name};", + f"{presence} = n_{self.c_name};"] + + def free_needs_iter(self): + return self.sub_type == 'nest' + + def _free_lines(self, _ri, var, ref): + lines = [] + if self.sub_type == 'nest': + lines += [ + f"for (i = 0; i < {var}->{ref}_count.{self.c_name}; i++)", + f'{self.nested_render_name}_free(&{var}->{ref}{self.c_name}[i]);', + ] + lines += (f"free({var}->{ref}{self.c_name});",) + return lines + +class TypeNestTypeValue(Type): + def _complex_member_type(self, _ri): + return self.nested_struct_type + + def _attr_typol(self): + return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, ' + + def _attr_get(self, ri, var): + prev = 'attr' + tv_args = '' + get_lines = [] + local_vars = [] + init_lines = [f"parg.rsp_policy = &{self.nested_render_name}_nest;", + f"parg.data = &{var}->{self.c_name};"] + if 'type-value' in self.attr: + tv_names = [c_lower(x) for x in self.attr["type-value"]] + local_vars += [f'const struct nlattr *attr_{", *attr_".join(tv_names)};'] + local_vars += [f'__u32 {", ".join(tv_names)};'] + for level in self.attr["type-value"]: + level = c_lower(level) + get_lines += [f'attr_{level} = ynl_attr_data({prev});'] + get_lines += [f'{level} = ynl_attr_type(attr_{level});'] + prev = 'attr_' + level + + tv_args = f", {', '.join(tv_names)}" + + get_lines += [f"{self.nested_render_name}_parse(&parg, {prev}{tv_args});"] + return get_lines, init_lines, local_vars + + +class TypeSubMessage(TypeNest): + def __init__(self, family, attr_set, attr, value): + super().__init__(family, attr_set, attr, value) + + self.selector = Selector(attr, attr_set) + + def _attr_typol(self): + typol = f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, ' + typol += '.is_submsg = 1, ' + # Reverse-parsing of the policy (ynl_err_walk() in ynl.c) does not + # support external selectors. No family uses sub-messages with external + # selector for requests so this is fine for now. + if not self.selector.is_external(): + typol += f'.selector_type = {self.attr_set[self["selector"]].value} ' + return typol + + def _attr_get(self, ri, var): + selector = self['selector'] + sel = c_lower(selector) + if self.selector.is_external(): + sel_var = f"_sel_{sel}" + else: + sel_var = f"{var}->{sel}" + get_lines = [f'if (!{sel_var})', + f'return ynl_submsg_failed(yarg, "{self.name}", "{selector}");', + f"if ({self.nested_render_name}_parse(&parg, {sel_var}, attr))", + "return YNL_PARSE_CB_ERROR;"] + init_lines = [f"parg.rsp_policy = &{self.nested_render_name}_nest;", + f"parg.data = &{var}->{self.c_name};"] + return get_lines, init_lines, None + + +class Selector: + def __init__(self, msg_attr, attr_set): + self.name = msg_attr["selector"] + + if self.name in attr_set: + self.attr = attr_set[self.name] + self.attr.is_selector = True + self._external = False + else: + # The selector will need to get passed down thru the structs + self.attr = None + self._external = True + + def set_attr(self, attr): + self.attr = attr + + def is_external(self): + return self._external + + +class Struct: + def __init__(self, family, space_name, type_list=None, fixed_header=None, + inherited=None, submsg=None): + self.family = family + self.space_name = space_name + self.attr_set = family.attr_sets[space_name] + # Use list to catch comparisons with empty sets + self._inherited = inherited if inherited is not None else [] + self.inherited = [] + self.fixed_header = None + if fixed_header: + self.fixed_header = 'struct ' + c_lower(fixed_header) + self.submsg = submsg + + self.nested = type_list is None + if family.name == c_lower(space_name): + self.render_name = c_lower(family.ident_name) + else: + self.render_name = c_lower(family.ident_name + '-' + space_name) + self.struct_name = 'struct ' + self.render_name + if self.nested and space_name in family.consts: + self.struct_name += '_' + self.ptr_name = self.struct_name + ' *' + # All attr sets this one contains, directly or multiple levels down + self.child_nests = set() + + self.request = False + self.reply = False + self.recursive = False + self.in_multi_val = False # used by a MultiAttr or and legacy arrays + + self.attr_list = [] + self.attrs = {} + if type_list is not None: + for t in type_list: + self.attr_list.append((t, self.attr_set[t]),) + else: + for t in self.attr_set: + self.attr_list.append((t, self.attr_set[t]),) + + max_val = 0 + self.attr_max_val = None + for name, attr in self.attr_list: + if attr.value >= max_val: + max_val = attr.value + self.attr_max_val = attr + self.attrs[name] = attr + + def __iter__(self): + yield from self.attrs + + def __getitem__(self, key): + return self.attrs[key] + + def member_list(self): + return self.attr_list + + def set_inherited(self, new_inherited): + if self._inherited != new_inherited: + raise Exception("Inheriting different members not supported") + self.inherited = [c_lower(x) for x in sorted(self._inherited)] + + def external_selectors(self): + sels = [] + for _name, attr in self.attr_list: + if isinstance(attr, TypeSubMessage) and attr.selector.is_external(): + sels.append(attr.selector) + return sels + + def free_needs_iter(self): + for _, attr in self.attr_list: + if attr.free_needs_iter(): + return True + return False + + +class EnumEntry(SpecEnumEntry): + def __init__(self, enum_set, yaml, prev, value_start): + super().__init__(enum_set, yaml, prev, value_start) + + if prev: + self.value_change = self.value != prev.value + 1 + else: + self.value_change = self.value != 0 + self.value_change = self.value_change or self.enum_set['type'] == 'flags' + + # Added by resolve: + self.c_name = None + delattr(self, "c_name") + + def resolve(self): + self.resolve_up(super()) + + self.c_name = c_upper(self.enum_set.value_pfx + self.name) + + +class EnumSet(SpecEnumSet): + def __init__(self, family, yaml): + self.render_name = c_lower(family.ident_name + '-' + yaml['name']) + + if 'enum-name' in yaml: + if yaml['enum-name']: + self.enum_name = 'enum ' + c_lower(yaml['enum-name']) + self.user_type = self.enum_name + else: + self.enum_name = None + else: + self.enum_name = 'enum ' + self.render_name + + if self.enum_name: + self.user_type = self.enum_name + else: + self.user_type = 'int' + + self.value_pfx = yaml.get('name-prefix', f"{family.ident_name}-{yaml['name']}-") + self.header = yaml.get('header', None) + self.enum_cnt_name = yaml.get('enum-cnt-name', None) + + super().__init__(family, yaml) + + def new_entry(self, entry, prev_entry, value_start): + return EnumEntry(self, entry, prev_entry, value_start) + + def value_range(self): + low = min(x.value for x in self.entries.values()) + high = max(x.value for x in self.entries.values()) + + if high - low + 1 != len(self.entries): + return None, None + + return low, high + + +class AttrSet(SpecAttrSet): + def __init__(self, family, yaml): + super().__init__(family, yaml) + + if self.subset_of is None: + if 'name-prefix' in yaml: + pfx = yaml['name-prefix'] + elif self.name == family.name: + pfx = family.ident_name + '-a-' + else: + pfx = f"{family.ident_name}-a-{self.name}-" + self.name_prefix = c_upper(pfx) + self.max_name = c_upper(self.yaml.get('attr-max-name', f"{self.name_prefix}max")) + self.cnt_name = c_upper(self.yaml.get('attr-cnt-name', f"__{self.name_prefix}max")) + else: + self.name_prefix = family.attr_sets[self.subset_of].name_prefix + self.max_name = family.attr_sets[self.subset_of].max_name + self.cnt_name = family.attr_sets[self.subset_of].cnt_name + + # Added by resolve: + self.c_name = None + delattr(self, "c_name") + + def resolve(self): + self.c_name = c_lower(self.name) + if self.c_name in _C_KW: + self.c_name += '_' + if self.c_name == self.family.c_name: + self.c_name = '' + + def new_attr(self, elem, value): + if elem['type'] in scalars: + t = TypeScalar(self.family, self, elem, value) + elif elem['type'] == 'unused': + t = TypeUnused(self.family, self, elem, value) + elif elem['type'] == 'pad': + t = TypePad(self.family, self, elem, value) + elif elem['type'] == 'flag': + t = TypeFlag(self.family, self, elem, value) + elif elem['type'] == 'string': + t = TypeString(self.family, self, elem, value) + elif elem['type'] == 'binary': + if 'struct' in elem: + t = TypeBinaryStruct(self.family, self, elem, value) + elif elem.get('sub-type') in scalars: + t = TypeBinaryScalarArray(self.family, self, elem, value) + else: + t = TypeBinary(self.family, self, elem, value) + elif elem['type'] == 'bitfield32': + t = TypeBitfield32(self.family, self, elem, value) + elif elem['type'] == 'nest': + t = TypeNest(self.family, self, elem, value) + elif elem['type'] == 'indexed-array' and 'sub-type' in elem: + if elem["sub-type"] in ['binary', 'nest', 'u32']: + t = TypeIndexedArray(self.family, self, elem, value) + else: + raise Exception(f'new_attr: unsupported sub-type {elem["sub-type"]}') + elif elem['type'] == 'nest-type-value': + t = TypeNestTypeValue(self.family, self, elem, value) + elif elem['type'] == 'sub-message': + t = TypeSubMessage(self.family, self, elem, value) + else: + raise Exception(f"No typed class for type {elem['type']}") + + if 'multi-attr' in elem and elem['multi-attr']: + t = TypeMultiAttr(self.family, self, elem, value, t) + + return t + + +class Operation(SpecOperation): + def __init__(self, family, yaml, req_value, rsp_value): + # Fill in missing operation properties (for fixed hdr-only msgs) + for mode in ['do', 'dump', 'event']: + for direction in ['request', 'reply']: + try: + yaml[mode][direction].setdefault('attributes', []) + except KeyError: + pass + + super().__init__(family, yaml, req_value, rsp_value) + + self.render_name = c_lower(family.ident_name + '_' + self.name) + + self.dual_policy = ('do' in yaml and 'request' in yaml['do']) and \ + ('dump' in yaml and 'request' in yaml['dump']) + + self.has_ntf = False + + # Added by resolve: + self.enum_name = None + delattr(self, "enum_name") + + def resolve(self): + self.resolve_up(super()) + + if not self.is_async: + self.enum_name = self.family.op_prefix + c_upper(self.name) + else: + self.enum_name = self.family.async_op_prefix + c_upper(self.name) + + def mark_has_ntf(self): + self.has_ntf = True + + +class SubMessage(SpecSubMessage): + def __init__(self, family, yaml): + super().__init__(family, yaml) + + self.render_name = c_lower(family.ident_name + '-' + yaml['name']) + + def resolve(self): + self.resolve_up(super()) + + +class Family(SpecFamily): + def __init__(self, file_name, exclude_ops, fn_prefix): + # Added by resolve: + self.c_name = None + delattr(self, "c_name") + self.op_prefix = None + delattr(self, "op_prefix") + self.async_op_prefix = None + delattr(self, "async_op_prefix") + self.mcgrps = None + delattr(self, "mcgrps") + self.consts = None + delattr(self, "consts") + self.hooks = None + delattr(self, "hooks") + + self.root_sets = {} + self.pure_nested_structs = {} + self.kernel_policy = None + self.global_policy = None + self.global_policy_set = None + + super().__init__(file_name, exclude_ops=exclude_ops) + + self.fam_key = c_upper(self.yaml.get('c-family-name', self.yaml["name"] + '_FAMILY_NAME')) + self.ver_key = c_upper(self.yaml.get('c-version-name', self.yaml["name"] + '_FAMILY_VERSION')) + + if 'definitions' not in self.yaml: + self.yaml['definitions'] = [] + + if 'uapi-header' in self.yaml: + self.uapi_header = self.yaml['uapi-header'] + else: + self.uapi_header = f"linux/{self.ident_name}.h" + if self.uapi_header.startswith("linux/") and self.uapi_header.endswith('.h'): + self.uapi_header_name = self.uapi_header[6:-2] + else: + self.uapi_header_name = self.ident_name + + self.fn_prefix = fn_prefix if fn_prefix else f'{self.ident_name}-nl' + + def resolve(self): + self.resolve_up(super()) + + self.c_name = c_lower(self.ident_name) + if 'name-prefix' in self.yaml['operations']: + self.op_prefix = c_upper(self.yaml['operations']['name-prefix']) + else: + self.op_prefix = c_upper(self.yaml['name'] + '-cmd-') + if 'async-prefix' in self.yaml['operations']: + self.async_op_prefix = c_upper(self.yaml['operations']['async-prefix']) + else: + self.async_op_prefix = self.op_prefix + + self.mcgrps = self.yaml.get('mcast-groups', {'list': []}) + + self.hooks = {} + for when in ['pre', 'post']: + self.hooks[when] = {} + for op_mode in ['do', 'dump']: + self.hooks[when][op_mode] = {} + self.hooks[when][op_mode]['set'] = set() + self.hooks[when][op_mode]['list'] = [] + + # dict space-name -> 'request': set(attrs), 'reply': set(attrs) + self.root_sets = {} + # dict space-name -> Struct + self.pure_nested_structs = {} + + self._mark_notify() + self._mock_up_events() + + self._load_root_sets() + self._load_nested_sets() + self._load_attr_use() + self._load_selector_passing() + self._load_hooks() + + self.kernel_policy = self.yaml.get('kernel-policy', 'split') + if self.kernel_policy == 'global': + self._load_global_policy() + + def new_enum(self, elem): + return EnumSet(self, elem) + + def new_attr_set(self, elem): + return AttrSet(self, elem) + + def new_operation(self, elem, req_value, rsp_value): + return Operation(self, elem, req_value, rsp_value) + + def new_sub_message(self, elem): + return SubMessage(self, elem) + + def is_classic(self): + return self.proto == 'netlink-raw' + + def _mark_notify(self): + for op in self.msgs.values(): + if 'notify' in op: + self.ops[op['notify']].mark_has_ntf() + + # Fake a 'do' equivalent of all events, so that we can render their response parsing + def _mock_up_events(self): + for op in self.yaml['operations']['list']: + if 'event' in op: + op['do'] = { + 'reply': { + 'attributes': op['event']['attributes'] + } + } + + def _load_root_sets(self): + for _op_name, op in self.msgs.items(): + if 'attribute-set' not in op: + continue + + req_attrs = set() + rsp_attrs = set() + for op_mode in ['do', 'dump']: + if op_mode in op and 'request' in op[op_mode]: + req_attrs.update(set(op[op_mode]['request']['attributes'])) + if op_mode in op and 'reply' in op[op_mode]: + rsp_attrs.update(set(op[op_mode]['reply']['attributes'])) + if 'event' in op: + rsp_attrs.update(set(op['event']['attributes'])) + + if op['attribute-set'] not in self.root_sets: + self.root_sets[op['attribute-set']] = {'request': req_attrs, 'reply': rsp_attrs} + else: + self.root_sets[op['attribute-set']]['request'].update(req_attrs) + self.root_sets[op['attribute-set']]['reply'].update(rsp_attrs) + + def _sort_pure_types(self): + # Try to reorder according to dependencies + pns_key_list = list(self.pure_nested_structs.keys()) + pns_key_seen = set() + rounds = len(pns_key_list) ** 2 # it's basically bubble sort + for _ in range(rounds): + if len(pns_key_list) == 0: + break + name = pns_key_list.pop(0) + finished = True + for _, spec in self.attr_sets[name].items(): + if 'nested-attributes' in spec: + nested = spec['nested-attributes'] + elif 'sub-message' in spec: + nested = spec.sub_message + else: + continue + + # If the unknown nest we hit is recursive it's fine, it'll be a pointer + if self.pure_nested_structs[nested].recursive: + continue + if nested not in pns_key_seen: + # Dicts are sorted, this will make struct last + struct = self.pure_nested_structs.pop(name) + self.pure_nested_structs[name] = struct + finished = False + break + if finished: + pns_key_seen.add(name) + else: + pns_key_list.append(name) + + def _load_nested_set_nest(self, spec): + inherit = set() + nested = spec['nested-attributes'] + if nested not in self.root_sets: + if nested not in self.pure_nested_structs: + self.pure_nested_structs[nested] = \ + Struct(self, nested, inherited=inherit, + fixed_header=spec.get('fixed-header')) + else: + raise Exception(f'Using attr set as root and nested not supported - {nested}') + + if 'type-value' in spec: + if nested in self.root_sets: + raise Exception("Inheriting members to a space used as root not supported") + inherit.update(set(spec['type-value'])) + elif spec['type'] == 'indexed-array': + inherit.add('idx') + self.pure_nested_structs[nested].set_inherited(inherit) + + return nested + + def _load_nested_set_submsg(self, spec): + # Fake the struct type for the sub-message itself + # its not a attr_set but codegen wants attr_sets. + submsg = self.sub_msgs[spec["sub-message"]] + nested = submsg.name + + attrs = [] + for name, fmt in submsg.formats.items(): + attr = { + "name": name, + "parent-sub-message": spec, + } + if 'attribute-set' in fmt: + attr |= { + "type": "nest", + "nested-attributes": fmt['attribute-set'], + } + if 'fixed-header' in fmt: + attr |= { "fixed-header": fmt["fixed-header"] } + elif 'fixed-header' in fmt: + attr |= { + "type": "binary", + "struct": fmt["fixed-header"], + } + else: + attr["type"] = "flag" + attrs.append(attr) + + self.attr_sets[nested] = AttrSet(self, { + "name": nested, + "name-pfx": self.name + '-' + spec.name + '-', + "attributes": attrs + }) + + if nested not in self.pure_nested_structs: + self.pure_nested_structs[nested] = Struct(self, nested, submsg=submsg) + + return nested + + def _load_nested_sets(self): + attr_set_queue = list(self.root_sets.keys()) + attr_set_seen = set(self.root_sets.keys()) + + while attr_set_queue: + a_set = attr_set_queue.pop(0) + for attr, spec in self.attr_sets[a_set].items(): + if 'nested-attributes' in spec: + nested = self._load_nested_set_nest(spec) + elif 'sub-message' in spec: + nested = self._load_nested_set_submsg(spec) + else: + continue + + if nested not in attr_set_seen: + attr_set_queue.append(nested) + attr_set_seen.add(nested) + + for root_set, rs_members in self.root_sets.items(): + for attr, spec in self.attr_sets[root_set].items(): + if 'nested-attributes' in spec: + nested = spec['nested-attributes'] + elif 'sub-message' in spec: + nested = spec.sub_message + else: + nested = None + + if nested: + if attr in rs_members['request']: + self.pure_nested_structs[nested].request = True + if attr in rs_members['reply']: + self.pure_nested_structs[nested].reply = True + + if spec.is_multi_val(): + child = self.pure_nested_structs.get(nested) + child.in_multi_val = True + + self._sort_pure_types() + + # Propagate the request / reply / recursive + for attr_set, struct in reversed(self.pure_nested_structs.items()): + for _, spec in self.attr_sets[attr_set].items(): + if attr_set in struct.child_nests: + struct.recursive = True + + if 'nested-attributes' in spec: + child_name = spec['nested-attributes'] + elif 'sub-message' in spec: + child_name = spec.sub_message + else: + continue + + struct.child_nests.add(child_name) + child = self.pure_nested_structs.get(child_name) + if child: + if not child.recursive: + struct.child_nests.update(child.child_nests) + child.request |= struct.request + child.reply |= struct.reply + if spec.is_multi_val(): + child.in_multi_val = True + + self._sort_pure_types() + + def _load_attr_use(self): + for _, struct in self.pure_nested_structs.items(): + if struct.request: + for _, arg in struct.member_list(): + arg.set_request() + if struct.reply: + for _, arg in struct.member_list(): + arg.set_reply() + + for root_set, rs_members in self.root_sets.items(): + for attr, spec in self.attr_sets[root_set].items(): + if attr in rs_members['request']: + spec.set_request() + if attr in rs_members['reply']: + spec.set_reply() + + def _load_selector_passing(self): + def all_structs(): + for k, v in reversed(self.pure_nested_structs.items()): + yield k, v + for k, _ in self.root_sets.items(): + yield k, None # we don't have a struct, but it must be terminal + + for attr_set, _struct in all_structs(): + for _, spec in self.attr_sets[attr_set].items(): + if 'nested-attributes' in spec: + child_name = spec['nested-attributes'] + elif 'sub-message' in spec: + child_name = spec.sub_message + else: + continue + + child = self.pure_nested_structs.get(child_name) + for selector in child.external_selectors(): + if selector.name in self.attr_sets[attr_set]: + sel_attr = self.attr_sets[attr_set][selector.name] + selector.set_attr(sel_attr) + else: + raise Exception("Passing selector thru more than one layer not supported") + + def _load_global_policy(self): + global_set = set() + attr_set_name = None + for _op_name, op in self.ops.items(): + if not op: + continue + if 'attribute-set' not in op: + continue + + if attr_set_name is None: + attr_set_name = op['attribute-set'] + if attr_set_name != op['attribute-set']: + raise Exception('For a global policy all ops must use the same set') + + for op_mode in ['do', 'dump']: + if op_mode in op: + req = op[op_mode].get('request') + if req: + global_set.update(req.get('attributes', [])) + + self.global_policy = [] + self.global_policy_set = attr_set_name + for attr in self.attr_sets[attr_set_name]: + if attr in global_set: + self.global_policy.append(attr) + + def _load_hooks(self): + for op in self.ops.values(): + for op_mode in ['do', 'dump']: + if op_mode not in op: + continue + for when in ['pre', 'post']: + if when not in op[op_mode]: + continue + name = op[op_mode][when] + if name in self.hooks[when][op_mode]['set']: + continue + self.hooks[when][op_mode]['set'].add(name) + self.hooks[when][op_mode]['list'].append(name) + + +class RenderInfo: + def __init__(self, cw, family, ku_space, op, op_mode, attr_set=None): + self.family = family + self.nl = cw.nlib + self.ku_space = ku_space + self.op_mode = op_mode + self.op = op + + fixed_hdr = op.fixed_header if op else None + self.fixed_hdr_len = 'ys->family->hdr_len' + if op and op.fixed_header: + if op.fixed_header != family.fixed_header: + if family.is_classic(): + self.fixed_hdr_len = f"sizeof(struct {c_lower(fixed_hdr)})" + else: + raise Exception("Per-op fixed header not supported, yet") + + + # 'do' and 'dump' response parsing is identical + self.type_consistent = True + self.type_oneside = False + if op_mode != 'do' and 'dump' in op: + if 'do' in op: + if ('reply' in op['do']) != ('reply' in op["dump"]): + self.type_consistent = False + elif 'reply' in op['do'] and op["do"]["reply"] != op["dump"]["reply"]: + self.type_consistent = False + else: + self.type_consistent = True + self.type_oneside = True + + self.attr_set = attr_set + if not self.attr_set: + self.attr_set = op['attribute-set'] + + self.type_name_conflict = False + if op: + self.type_name = c_lower(op.name) + else: + self.type_name = c_lower(attr_set) + if attr_set in family.consts: + self.type_name_conflict = True + + self.cw = cw + + self.struct = {} + if op_mode == 'notify': + op_mode = 'do' if 'do' in op else 'dump' + for op_dir in ['request', 'reply']: + if op: + type_list = [] + if op_dir in op[op_mode]: + type_list = op[op_mode][op_dir]['attributes'] + self.struct[op_dir] = Struct(family, self.attr_set, + fixed_header=fixed_hdr, + type_list=type_list) + if op_mode == 'event': + self.struct['reply'] = Struct(family, self.attr_set, + fixed_header=fixed_hdr, + type_list=op['event']['attributes']) + + def type_empty(self, key): + return len(self.struct[key].attr_list) == 0 and \ + self.struct['request'].fixed_header is None + + def needs_nlflags(self, direction): + return self.op_mode == 'do' and direction == 'request' and self.family.is_classic() + + +class CodeWriter: + def __init__(self, nlib, out_file=None, overwrite=True): + self.nlib = nlib + self._overwrite = overwrite + + self._nl = False + self._block_end = False + self._silent_block = False + self._ind = 0 + self._ifdef_block = None + if out_file is None: + self._out = os.sys.stdout + else: + # pylint: disable=consider-using-with + self._out = tempfile.NamedTemporaryFile('w+') + self._out_file = out_file + + def __del__(self): + self.close_out_file() + + def close_out_file(self): + if self._out == os.sys.stdout: + return + # Avoid modifying the file if contents didn't change + self._out.flush() + if not self._overwrite and os.path.isfile(self._out_file): + if filecmp.cmp(self._out.name, self._out_file, shallow=False): + return + with open(self._out_file, 'w+', encoding='utf-8') as out_file: + self._out.seek(0) + shutil.copyfileobj(self._out, out_file) + self._out.close() + self._out = os.sys.stdout + + @classmethod + def _is_cond(cls, line): + return line.startswith('if') or line.startswith('while') or line.startswith('for') + + def p(self, line, add_ind=0): + if self._block_end: + self._block_end = False + if line.startswith('else'): + line = '} ' + line + else: + self._out.write('\t' * self._ind + '}\n') + + if self._nl: + self._out.write('\n') + self._nl = False + + ind = self._ind + if line[-1] == ':': + ind -= 1 + if self._silent_block: + ind += 1 + self._silent_block = line.endswith(')') and CodeWriter._is_cond(line) + self._silent_block |= line.strip() == 'else' + if line[0] == '#': + ind = 0 + if add_ind: + ind += add_ind + self._out.write('\t' * ind + line + '\n') + + def nl(self): + self._nl = True + + def block_start(self, line=''): + if line: + line = line + ' ' + self.p(line + '{') + self._ind += 1 + + def block_end(self, line=''): + if line and line[0] not in {';', ','}: + line = ' ' + line + self._ind -= 1 + self._nl = False + if not line: + # Delay printing closing bracket in case "else" comes next + if self._block_end: + self._out.write('\t' * (self._ind + 1) + '}\n') + self._block_end = True + else: + self.p('}' + line) + + def write_doc_line(self, doc, indent=True): + words = doc.split() + line = ' *' + for word in words: + if len(line) + len(word) >= 79: + self.p(line) + line = ' *' + if indent: + line += ' ' + line += ' ' + word + self.p(line) + + def write_func_prot(self, qual_ret, name, args=None, doc=None, suffix=''): + if not args: + args = ['void'] + + if doc: + self.p('/*') + self.p(' * ' + doc) + self.p(' */') + + oneline = qual_ret + if qual_ret[-1] != '*': + oneline += ' ' + oneline += f"{name}({', '.join(args)}){suffix}" + + if len(oneline) < 80: + self.p(oneline) + return + + v = qual_ret + if len(v) > 3: + self.p(v) + v = '' + elif qual_ret[-1] != '*': + v += ' ' + v += name + '(' + ind = '\t' * (len(v) // 8) + ' ' * (len(v) % 8) + delta_ind = len(v) - len(ind) + v += args[0] + i = 1 + while i < len(args): + next_len = len(v) + len(args[i]) + if v[0] == '\t': + next_len += delta_ind + if next_len > 76: + self.p(v + ',') + v = ind + else: + v += ', ' + v += args[i] + i += 1 + self.p(v + ')' + suffix) + + def write_func_lvar(self, local_vars): + if not local_vars: + return + + if isinstance(local_vars, str): + local_vars = [local_vars] + + local_vars.sort(key=len, reverse=True) + for var in local_vars: + self.p(var) + self.nl() + + def write_func(self, qual_ret, name, body, args=None, local_vars=None): + self.write_func_prot(qual_ret=qual_ret, name=name, args=args) + self.block_start() + self.write_func_lvar(local_vars=local_vars) + + for line in body: + self.p(line) + self.block_end() + + def writes_defines(self, defines): + longest = 0 + for define in defines: + longest = max(len(define[0]), longest) + longest = ((longest + 8) // 8) * 8 + for define in defines: + line = '#define ' + define[0] + line += '\t' * ((longest - len(define[0]) + 7) // 8) + if isinstance(define[1], int): + line += str(define[1]) + elif isinstance(define[1], str): + line += '"' + define[1] + '"' + self.p(line) + + def write_struct_init(self, members): + longest = max(len(x[0]) for x in members) + longest += 1 # because we prepend a . + longest = ((longest + 8) // 8) * 8 + for one in members: + line = '.' + one[0] + line += '\t' * ((longest - len(one[0]) - 1 + 7) // 8) + line += '= ' + str(one[1]) + ',' + self.p(line) + + def ifdef_block(self, config): + config_option = None + if config: + config_option = 'CONFIG_' + c_upper(config) + if self._ifdef_block == config_option: + return + + if self._ifdef_block: + self.p('#endif /* ' + self._ifdef_block + ' */') + if config_option: + self.p('#ifdef ' + config_option) + self._ifdef_block = config_option + + +scalars = {'u8', 'u16', 'u32', 'u64', 's8', 's16', 's32', 's64', 'uint', 'sint'} + +direction_to_suffix = { + 'reply': '_rsp', + 'request': '_req', + '': '' +} + +op_mode_to_wrapper = { + 'do': '', + 'dump': '_list', + 'notify': '_ntf', + 'event': '', +} + +_C_KW = { + 'auto', + 'bool', + 'break', + 'case', + 'char', + 'const', + 'continue', + 'default', + 'do', + 'double', + 'else', + 'enum', + 'extern', + 'float', + 'for', + 'goto', + 'if', + 'inline', + 'int', + 'long', + 'register', + 'return', + 'short', + 'signed', + 'sizeof', + 'static', + 'struct', + 'switch', + 'typedef', + 'union', + 'unsigned', + 'void', + 'volatile', + 'while' +} + + +def rdir(direction): + if direction == 'reply': + return 'request' + if direction == 'request': + return 'reply' + return direction + + +def op_prefix(ri, direction, deref=False): + suffix = f"_{ri.type_name}" + + if not ri.op_mode: + pass + elif ri.op_mode == 'do': + suffix += f"{direction_to_suffix[direction]}" + else: + if direction == 'request': + suffix += '_req' + if not ri.type_oneside: + suffix += '_dump' + else: + if ri.type_consistent: + if deref: + suffix += f"{direction_to_suffix[direction]}" + else: + suffix += op_mode_to_wrapper[ri.op_mode] + else: + suffix += '_rsp' + suffix += '_dump' if deref else '_list' + + return f"{ri.family.c_name}{suffix}" + + +def type_name(ri, direction, deref=False): + return f"struct {op_prefix(ri, direction, deref=deref)}" + + +def print_prototype(ri, direction, terminate=True, doc=None): + suffix = ';' if terminate else '' + + fname = ri.op.render_name + if ri.op_mode == 'dump': + fname += '_dump' + + args = ['struct ynl_sock *ys'] + if 'request' in ri.op[ri.op_mode]: + args.append(f"{type_name(ri, direction)} *" + f"{direction_to_suffix[direction][1:]}") + + ret = 'int' + if 'reply' in ri.op[ri.op_mode]: + ret = f"{type_name(ri, rdir(direction))} *" + + ri.cw.write_func_prot(ret, fname, args, doc=doc, suffix=suffix) + + +def print_req_prototype(ri): + print_prototype(ri, "request", doc=ri.op['doc']) + + +def print_dump_prototype(ri): + print_prototype(ri, "request") + + +def put_typol_submsg(cw, struct): + cw.block_start(line=f'const struct ynl_policy_attr {struct.render_name}_policy[] =') + + i = 0 + for name, arg in struct.member_list(): + nest = "" + if arg.type == 'nest': + nest = f" .nest = &{arg.nested_render_name}_nest," + cw.p('[%d] = { .type = YNL_PT_SUBMSG, .name = "%s",%s },' % + (i, name, nest)) + i += 1 + + cw.block_end(line=';') + cw.nl() + + cw.block_start(line=f'const struct ynl_policy_nest {struct.render_name}_nest =') + cw.p(f'.max_attr = {i - 1},') + cw.p(f'.table = {struct.render_name}_policy,') + cw.block_end(line=';') + cw.nl() + + +def put_typol_fwd(cw, struct): + cw.p(f'extern const struct ynl_policy_nest {struct.render_name}_nest;') + + +def put_typol(cw, struct): + if struct.submsg: + put_typol_submsg(cw, struct) + return + + type_max = struct.attr_set.max_name + cw.block_start(line=f'const struct ynl_policy_attr {struct.render_name}_policy[{type_max} + 1] =') + + for _, arg in struct.member_list(): + arg.attr_typol(cw) + + cw.block_end(line=';') + cw.nl() + + cw.block_start(line=f'const struct ynl_policy_nest {struct.render_name}_nest =') + cw.p(f'.max_attr = {type_max},') + cw.p(f'.table = {struct.render_name}_policy,') + cw.block_end(line=';') + cw.nl() + + +def _put_enum_to_str_helper(cw, render_name, map_name, arg_name, enum=None): + args = [f'int {arg_name}'] + if enum: + args = [enum.user_type + ' ' + arg_name] + cw.write_func_prot('const char *', f'{render_name}_str', args) + cw.block_start() + if enum and enum.type == 'flags': + cw.p(f'{arg_name} = ffs({arg_name}) - 1;') + cw.p(f'if ({arg_name} < 0 || {arg_name} >= (int)YNL_ARRAY_SIZE({map_name}))') + cw.p('return NULL;') + cw.p(f'return {map_name}[{arg_name}];') + cw.block_end() + cw.nl() + + +def put_op_name_fwd(family, cw): + cw.write_func_prot('const char *', f'{family.c_name}_op_str', ['int op'], suffix=';') + + +def put_op_name(family, cw): + map_name = f'{family.c_name}_op_strmap' + cw.block_start(line=f"static const char * const {map_name}[] =") + for op_name, op in family.msgs.items(): + if op.rsp_value: + # Make sure we don't add duplicated entries, if multiple commands + # produce the same response in legacy families. + if family.rsp_by_value[op.rsp_value] != op: + cw.p(f'// skip "{op_name}", duplicate reply value') + continue + + if op.req_value == op.rsp_value: + cw.p(f'[{op.enum_name}] = "{op_name}",') + else: + cw.p(f'[{op.rsp_value}] = "{op_name}",') + cw.block_end(line=';') + cw.nl() + + _put_enum_to_str_helper(cw, family.c_name + '_op', map_name, 'op') + + +def put_enum_to_str_fwd(_family, cw, enum): + args = [enum.user_type + ' value'] + cw.write_func_prot('const char *', f'{enum.render_name}_str', args, suffix=';') + + +def put_enum_to_str(_family, cw, enum): + map_name = f'{enum.render_name}_strmap' + cw.block_start(line=f"static const char * const {map_name}[] =") + for entry in enum.entries.values(): + cw.p(f'[{entry.value}] = "{entry.name}",') + cw.block_end(line=';') + cw.nl() + + _put_enum_to_str_helper(cw, enum.render_name, map_name, 'value', enum=enum) + + +def put_local_vars(struct): + local_vars = [] + has_array = False + has_count = False + for _, arg in struct.member_list(): + has_array |= arg.type == 'indexed-array' + has_count |= arg.presence_type() == 'count' + if has_array: + local_vars.append('struct nlattr *array;') + if has_count: + local_vars.append('unsigned int i;') + return local_vars + + +def put_req_nested_prototype(ri, struct, suffix=';'): + func_args = ['struct nlmsghdr *nlh', + 'unsigned int attr_type', + f'{struct.ptr_name}obj'] + + ri.cw.write_func_prot('int', f'{struct.render_name}_put', func_args, + suffix=suffix) + + +def put_req_nested(ri, struct): + local_vars = [] + init_lines = [] + + if struct.submsg is None: + local_vars.append('struct nlattr *nest;') + init_lines.append("nest = ynl_attr_nest_start(nlh, attr_type);") + if struct.fixed_header: + local_vars.append('void *hdr;') + struct_sz = f'sizeof({struct.fixed_header})' + init_lines.append(f"hdr = ynl_nlmsg_put_extra_header(nlh, {struct_sz});") + init_lines.append(f"memcpy(hdr, &obj->_hdr, {struct_sz});") + + local_vars += put_local_vars(struct) + + put_req_nested_prototype(ri, struct, suffix='') + ri.cw.block_start() + ri.cw.write_func_lvar(local_vars) + + for line in init_lines: + ri.cw.p(line) + + for _, arg in struct.member_list(): + arg.attr_put(ri, "obj") + + if struct.submsg is None: + ri.cw.p("ynl_attr_nest_end(nlh, nest);") + + ri.cw.nl() + ri.cw.p('return 0;') + ri.cw.block_end() + ri.cw.nl() + + +def _multi_parse(ri, struct, init_lines, local_vars): + if struct.fixed_header: + local_vars += ['void *hdr;'] + if struct.nested: + if struct.fixed_header: + iter_line = f"ynl_attr_for_each_nested_off(attr, nested, sizeof({struct.fixed_header}))" + else: + iter_line = "ynl_attr_for_each_nested(attr, nested)" + else: + iter_line = "ynl_attr_for_each(attr, nlh, yarg->ys->family->hdr_len)" + if ri.op.fixed_header != ri.family.fixed_header: + if ri.family.is_classic(): + iter_line = f"ynl_attr_for_each(attr, nlh, sizeof({struct.fixed_header}))" + else: + raise Exception("Per-op fixed header not supported, yet") + + indexed_arrays = set() + multi_attrs = set() + needs_parg = False + var_set = set() + for arg, aspec in struct.member_list(): + if aspec['type'] == 'indexed-array' and 'sub-type' in aspec: + if aspec["sub-type"] in {'binary', 'nest'}: + local_vars.append(f'const struct nlattr *attr_{aspec.c_name} = NULL;') + indexed_arrays.add(arg) + elif aspec['sub-type'] in scalars: + local_vars.append(f'const struct nlattr *attr_{aspec.c_name} = NULL;') + indexed_arrays.add(arg) + else: + raise Exception(f'Not supported sub-type {aspec["sub-type"]}') + if 'multi-attr' in aspec: + multi_attrs.add(arg) + needs_parg |= 'nested-attributes' in aspec + needs_parg |= 'sub-message' in aspec + + try: + _, _, l_vars = aspec._attr_get(ri, '') + var_set |= set(l_vars) if l_vars else set() + except Exception: + pass # _attr_get() not implemented by simple types, ignore + local_vars += list(var_set) + if indexed_arrays or multi_attrs: + local_vars.append('int i;') + if needs_parg: + local_vars.append('struct ynl_parse_arg parg;') + init_lines.append('parg.ys = yarg->ys;') + + all_multi = indexed_arrays | multi_attrs + + for arg in sorted(all_multi): + local_vars.append(f"unsigned int n_{struct[arg].c_name} = 0;") + + ri.cw.block_start() + ri.cw.write_func_lvar(local_vars) + + for line in init_lines: + ri.cw.p(line) + ri.cw.nl() + + for arg in struct.inherited: + ri.cw.p(f'dst->{arg} = {arg};') + + if struct.fixed_header: + if struct.nested: + ri.cw.p('hdr = ynl_attr_data(nested);') + elif ri.family.is_classic(): + ri.cw.p('hdr = ynl_nlmsg_data(nlh);') + else: + ri.cw.p('hdr = ynl_nlmsg_data_offset(nlh, sizeof(struct genlmsghdr));') + ri.cw.p(f"memcpy(&dst->_hdr, hdr, sizeof({struct.fixed_header}));") + for arg in sorted(all_multi): + aspec = struct[arg] + ri.cw.p(f"if (dst->{aspec.c_name})") + ri.cw.p(f'return ynl_error_parse(yarg, "attribute already present ({struct.attr_set.name}.{aspec.name})");') + + ri.cw.nl() + ri.cw.block_start(line=iter_line) + ri.cw.p('unsigned int type = ynl_attr_type(attr);') + ri.cw.nl() + + first = True + for _, arg in struct.member_list(): + good = arg.attr_get(ri, 'dst', first=first) + # First may be 'unused' or 'pad', ignore those + first &= not good + + ri.cw.block_end() + ri.cw.nl() + + for arg in sorted(indexed_arrays): + aspec = struct[arg] + + ri.cw.block_start(line=f"if (n_{aspec.c_name})") + ri.cw.p(f"dst->{aspec.c_name} = calloc(n_{aspec.c_name}, sizeof(*dst->{aspec.c_name}));") + ri.cw.p(f"if (!dst->{aspec.c_name})") + ri.cw.p("return YNL_PARSE_CB_ERROR;") + ri.cw.p(f"dst->_count.{aspec.c_name} = n_{aspec.c_name};") + ri.cw.p('i = 0;') + if 'nested-attributes' in aspec: + ri.cw.p(f"parg.rsp_policy = &{aspec.nested_render_name}_nest;") + ri.cw.block_start(line=f"ynl_attr_for_each_nested(attr, attr_{aspec.c_name})") + if 'nested-attributes' in aspec: + ri.cw.p(f"parg.data = &dst->{aspec.c_name}[i];") + ri.cw.p(f"if ({aspec.nested_render_name}_parse(&parg, attr, ynl_attr_type(attr)))") + ri.cw.p('return YNL_PARSE_CB_ERROR;') + elif aspec.sub_type in scalars: + ri.cw.p(f"dst->{aspec.c_name}[i] = ynl_attr_get_{aspec.sub_type}(attr);") + elif aspec.sub_type == 'binary' and 'exact-len' in aspec.checks: + # Length is validated by typol + ri.cw.p(f'memcpy(dst->{aspec.c_name}[i], ynl_attr_data(attr), {aspec.checks["exact-len"]});') + else: + raise Exception(f"Nest parsing type not supported in {aspec['name']}") + ri.cw.p('i++;') + ri.cw.block_end() + ri.cw.block_end() + ri.cw.nl() + + for arg in sorted(multi_attrs): + aspec = struct[arg] + ri.cw.block_start(line=f"if (n_{aspec.c_name})") + ri.cw.p(f"dst->{aspec.c_name} = calloc(n_{aspec.c_name}, sizeof(*dst->{aspec.c_name}));") + ri.cw.p(f"if (!dst->{aspec.c_name})") + ri.cw.p("return YNL_PARSE_CB_ERROR;") + ri.cw.p(f"dst->_count.{aspec.c_name} = n_{aspec.c_name};") + ri.cw.p('i = 0;') + if 'nested-attributes' in aspec: + ri.cw.p(f"parg.rsp_policy = &{aspec.nested_render_name}_nest;") + ri.cw.block_start(line=iter_line) + ri.cw.block_start(line=f"if (ynl_attr_type(attr) == {aspec.enum_name})") + if 'nested-attributes' in aspec: + ri.cw.p(f"parg.data = &dst->{aspec.c_name}[i];") + ri.cw.p(f"if ({aspec.nested_render_name}_parse(&parg, attr))") + ri.cw.p('return YNL_PARSE_CB_ERROR;') + elif aspec.type in scalars: + ri.cw.p(f"dst->{aspec.c_name}[i] = ynl_attr_get_{aspec.type}(attr);") + elif aspec.type == 'binary' and 'struct' in aspec: + ri.cw.p('size_t len = ynl_attr_data_len(attr);') + ri.cw.nl() + ri.cw.p(f'if (len > sizeof(dst->{aspec.c_name}[0]))') + ri.cw.p(f'len = sizeof(dst->{aspec.c_name}[0]);') + ri.cw.p(f"memcpy(&dst->{aspec.c_name}[i], ynl_attr_data(attr), len);") + elif aspec.type == 'string': + ri.cw.p('unsigned int len;') + ri.cw.nl() + ri.cw.p('len = strnlen(ynl_attr_get_str(attr), ynl_attr_data_len(attr));') + ri.cw.p(f'dst->{aspec.c_name}[i] = malloc(sizeof(struct ynl_string) + len + 1);') + ri.cw.p(f"if (!dst->{aspec.c_name}[i])") + ri.cw.p("return YNL_PARSE_CB_ERROR;") + ri.cw.p(f"dst->{aspec.c_name}[i]->len = len;") + ri.cw.p(f"memcpy(dst->{aspec.c_name}[i]->str, ynl_attr_get_str(attr), len);") + ri.cw.p(f"dst->{aspec.c_name}[i]->str[len] = 0;") + else: + raise Exception(f'Nest parsing of type {aspec.type} not supported yet') + ri.cw.p('i++;') + ri.cw.block_end() + ri.cw.block_end() + ri.cw.block_end() + ri.cw.nl() + + if struct.nested: + ri.cw.p('return 0;') + else: + ri.cw.p('return YNL_PARSE_CB_OK;') + ri.cw.block_end() + ri.cw.nl() + + +def parse_rsp_submsg(ri, struct): + parse_rsp_nested_prototype(ri, struct, suffix='') + + var = 'dst' + local_vars = {'const struct nlattr *attr = nested;', + f'{struct.ptr_name}{var} = yarg->data;', + 'struct ynl_parse_arg parg;'} + + for _, arg in struct.member_list(): + _, _, l_vars = arg._attr_get(ri, var) + local_vars |= set(l_vars) if l_vars else set() + + ri.cw.block_start() + ri.cw.write_func_lvar(list(local_vars)) + ri.cw.p('parg.ys = yarg->ys;') + ri.cw.nl() + + first = True + for name, arg in struct.member_list(): + kw = 'if' if first else 'else if' + first = False + + ri.cw.block_start(line=f'{kw} (!strcmp(sel, "{name}"))') + get_lines, init_lines, _ = arg._attr_get(ri, var) + for line in init_lines or []: + ri.cw.p(line) + for line in get_lines: + ri.cw.p(line) + if arg.presence_type() == 'present': + ri.cw.p(f"{var}->_present.{arg.c_name} = 1;") + ri.cw.block_end() + ri.cw.p('return 0;') + ri.cw.block_end() + ri.cw.nl() + + +def parse_rsp_nested_prototype(ri, struct, suffix=';'): + func_args = ['struct ynl_parse_arg *yarg', + 'const struct nlattr *nested'] + for sel in struct.external_selectors(): + func_args.append('const char *_sel_' + sel.name) + if struct.submsg: + func_args.insert(1, 'const char *sel') + for arg in struct.inherited: + func_args.append('__u32 ' + arg) + + ri.cw.write_func_prot('int', f'{struct.render_name}_parse', func_args, + suffix=suffix) + + +def parse_rsp_nested(ri, struct): + if struct.submsg: + parse_rsp_submsg(ri, struct) + return + + parse_rsp_nested_prototype(ri, struct, suffix='') + + local_vars = ['const struct nlattr *attr;', + f'{struct.ptr_name}dst = yarg->data;'] + init_lines = [] + + if struct.member_list(): + _multi_parse(ri, struct, init_lines, local_vars) + else: + # Empty nest + ri.cw.block_start() + ri.cw.p('return 0;') + ri.cw.block_end() + ri.cw.nl() + + +def parse_rsp_msg(ri, deref=False): + if 'reply' not in ri.op[ri.op_mode] and ri.op_mode != 'event': + return + + func_args = ['const struct nlmsghdr *nlh', + 'struct ynl_parse_arg *yarg'] + + local_vars = [f'{type_name(ri, "reply", deref=deref)} *dst;', + 'const struct nlattr *attr;'] + init_lines = ['dst = yarg->data;'] + + ri.cw.write_func_prot('int', f'{op_prefix(ri, "reply", deref=deref)}_parse', func_args) + + if ri.struct["reply"].member_list(): + _multi_parse(ri, ri.struct["reply"], init_lines, local_vars) + else: + # Empty reply + ri.cw.block_start() + ri.cw.p('return YNL_PARSE_CB_OK;') + ri.cw.block_end() + ri.cw.nl() + + +def print_req(ri): + ret_ok = '0' + ret_err = '-1' + direction = "request" + local_vars = ['struct ynl_req_state yrs = { .yarg = { .ys = ys, }, };', + 'struct nlmsghdr *nlh;', + 'int err;'] + + if 'reply' in ri.op[ri.op_mode]: + ret_ok = 'rsp' + ret_err = 'NULL' + local_vars += [f'{type_name(ri, rdir(direction))} *rsp;'] + + if ri.struct["request"].fixed_header: + local_vars += ['size_t hdr_len;', + 'void *hdr;'] + + local_vars += put_local_vars(ri.struct['request']) + + print_prototype(ri, direction, terminate=False) + ri.cw.block_start() + ri.cw.write_func_lvar(local_vars) + + if ri.family.is_classic(): + ri.cw.p(f"nlh = ynl_msg_start_req(ys, {ri.op.enum_name}, req->_nlmsg_flags);") + else: + ri.cw.p(f"nlh = ynl_gemsg_start_req(ys, {ri.nl.get_family_id()}, {ri.op.enum_name}, 1);") + + ri.cw.p(f"ys->req_policy = &{ri.struct['request'].render_name}_nest;") + ri.cw.p(f"ys->req_hdr_len = {ri.fixed_hdr_len};") + if 'reply' in ri.op[ri.op_mode]: + ri.cw.p(f"yrs.yarg.rsp_policy = &{ri.struct['reply'].render_name}_nest;") + ri.cw.nl() + + if ri.struct['request'].fixed_header: + ri.cw.p("hdr_len = sizeof(req->_hdr);") + ri.cw.p("hdr = ynl_nlmsg_put_extra_header(nlh, hdr_len);") + ri.cw.p("memcpy(hdr, &req->_hdr, hdr_len);") + ri.cw.nl() + + for _, attr in ri.struct["request"].member_list(): + attr.attr_put(ri, "req") + ri.cw.nl() + + if 'reply' in ri.op[ri.op_mode]: + ri.cw.p('rsp = calloc(1, sizeof(*rsp));') + ri.cw.p('if (!rsp)') + ri.cw.p(f'return {ret_err};') + ri.cw.p('yrs.yarg.data = rsp;') + ri.cw.p(f"yrs.cb = {op_prefix(ri, 'reply')}_parse;") + if ri.op.value is not None: + ri.cw.p(f'yrs.rsp_cmd = {ri.op.enum_name};') + else: + ri.cw.p(f'yrs.rsp_cmd = {ri.op.rsp_value};') + ri.cw.nl() + ri.cw.p("err = ynl_exec(ys, nlh, &yrs);") + ri.cw.p('if (err < 0)') + if 'reply' in ri.op[ri.op_mode]: + ri.cw.p('goto err_free;') + else: + ri.cw.p('return -1;') + ri.cw.nl() + + ri.cw.p(f"return {ret_ok};") + ri.cw.nl() + + if 'reply' in ri.op[ri.op_mode]: + ri.cw.p('err_free:') + ri.cw.p(f"{call_free(ri, rdir(direction), 'rsp')}") + ri.cw.p(f"return {ret_err};") + + ri.cw.block_end() + + +def print_dump(ri): + direction = "request" + print_prototype(ri, direction, terminate=False) + ri.cw.block_start() + local_vars = ['struct ynl_dump_state yds = {};', + 'struct nlmsghdr *nlh;', + 'int err;'] + + if ri.struct['request'].fixed_header: + local_vars += ['size_t hdr_len;', + 'void *hdr;'] + + if 'request' in ri.op[ri.op_mode]: + local_vars += put_local_vars(ri.struct['request']) + + ri.cw.write_func_lvar(local_vars) + + ri.cw.p('yds.yarg.ys = ys;') + ri.cw.p(f"yds.yarg.rsp_policy = &{ri.struct['reply'].render_name}_nest;") + ri.cw.p("yds.yarg.data = NULL;") + ri.cw.p(f"yds.alloc_sz = sizeof({type_name(ri, rdir(direction))});") + ri.cw.p(f"yds.cb = {op_prefix(ri, 'reply', deref=True)}_parse;") + if ri.op.value is not None: + ri.cw.p(f'yds.rsp_cmd = {ri.op.enum_name};') + else: + ri.cw.p(f'yds.rsp_cmd = {ri.op.rsp_value};') + ri.cw.nl() + if ri.family.is_classic(): + ri.cw.p(f"nlh = ynl_msg_start_dump(ys, {ri.op.enum_name});") + else: + ri.cw.p(f"nlh = ynl_gemsg_start_dump(ys, {ri.nl.get_family_id()}, {ri.op.enum_name}, 1);") + + if ri.struct['request'].fixed_header: + ri.cw.p("hdr_len = sizeof(req->_hdr);") + ri.cw.p("hdr = ynl_nlmsg_put_extra_header(nlh, hdr_len);") + ri.cw.p("memcpy(hdr, &req->_hdr, hdr_len);") + ri.cw.nl() + + if "request" in ri.op[ri.op_mode]: + ri.cw.p(f"ys->req_policy = &{ri.struct['request'].render_name}_nest;") + ri.cw.p(f"ys->req_hdr_len = {ri.fixed_hdr_len};") + ri.cw.nl() + for _, attr in ri.struct["request"].member_list(): + attr.attr_put(ri, "req") + ri.cw.nl() + + ri.cw.p('err = ynl_exec_dump(ys, nlh, &yds);') + ri.cw.p('if (err < 0)') + ri.cw.p('goto free_list;') + ri.cw.nl() + + ri.cw.p('return yds.first;') + ri.cw.nl() + ri.cw.p('free_list:') + ri.cw.p(call_free(ri, rdir(direction), 'yds.first')) + ri.cw.p('return NULL;') + ri.cw.block_end() + + +def call_free(ri, direction, var): + return f"{op_prefix(ri, direction)}_free({var});" + + +def free_arg_name(direction): + if direction: + return direction_to_suffix[direction][1:] + return 'obj' + + +def print_alloc_wrapper(ri, direction, struct=None): + name = op_prefix(ri, direction) + struct_name = name + if ri.type_name_conflict: + struct_name += '_' + + args = ["void"] + cnt = "1" + if struct and struct.in_multi_val: + args = ["unsigned int n"] + cnt = "n" + + ri.cw.write_func_prot(f'static inline struct {struct_name} *', + f"{name}_alloc", args) + ri.cw.block_start() + ri.cw.p(f'return calloc({cnt}, sizeof(struct {struct_name}));') + ri.cw.block_end() + + +def print_free_prototype(ri, direction, suffix=';'): + name = op_prefix(ri, direction) + struct_name = name + if ri.type_name_conflict: + struct_name += '_' + arg = free_arg_name(direction) + ri.cw.write_func_prot('void', f"{name}_free", [f"struct {struct_name} *{arg}"], suffix=suffix) + + +def print_nlflags_set(ri, direction): + name = op_prefix(ri, direction) + ri.cw.write_func_prot('static inline void', f"{name}_set_nlflags", + [f"struct {name} *req", "__u16 nl_flags"]) + ri.cw.block_start() + ri.cw.p('req->_nlmsg_flags = nl_flags;') + ri.cw.block_end() + ri.cw.nl() + + +def _print_type(ri, direction, struct): + suffix = f'_{ri.type_name}{direction_to_suffix[direction]}' + if not direction and ri.type_name_conflict: + suffix += '_' + + if ri.op_mode == 'dump' and not ri.type_oneside: + suffix += '_dump' + + ri.cw.block_start(line=f"struct {ri.family.c_name}{suffix}") + + if ri.needs_nlflags(direction): + ri.cw.p('__u16 _nlmsg_flags;') + ri.cw.nl() + if struct.fixed_header: + ri.cw.p(struct.fixed_header + ' _hdr;') + ri.cw.nl() + + for type_filter in ['present', 'len', 'count']: + meta_started = False + for _, attr in struct.member_list(): + line = attr.presence_member(ri.ku_space, type_filter) + if line: + if not meta_started: + ri.cw.block_start(line="struct") + meta_started = True + ri.cw.p(line) + if meta_started: + ri.cw.block_end(line=f'_{type_filter};') + ri.cw.nl() + + for arg in struct.inherited: + ri.cw.p(f"__u32 {arg};") + + for _, attr in struct.member_list(): + attr.struct_member(ri) + + ri.cw.block_end(line=';') + ri.cw.nl() + + +def print_type(ri, direction): + _print_type(ri, direction, ri.struct[direction]) + + +def print_type_full(ri, struct): + _print_type(ri, "", struct) + + if struct.request and struct.in_multi_val: + print_alloc_wrapper(ri, "", struct) + ri.cw.nl() + free_rsp_nested_prototype(ri) + ri.cw.nl() + + # Name conflicts are too hard to deal with with the current code base, + # they are very rare so don't bother printing setters in that case. + if ri.ku_space == 'user' and not ri.type_name_conflict: + for _, attr in struct.member_list(): + attr.setter(ri, ri.attr_set, "", var="obj") + ri.cw.nl() + + +def print_type_helpers(ri, direction, deref=False): + print_free_prototype(ri, direction) + ri.cw.nl() + + if ri.needs_nlflags(direction): + print_nlflags_set(ri, direction) + + if ri.ku_space == 'user' and direction == 'request': + for _, attr in ri.struct[direction].member_list(): + attr.setter(ri, ri.attr_set, direction, deref=deref) + ri.cw.nl() + + +def print_req_type_helpers(ri): + if ri.type_empty("request"): + return + print_alloc_wrapper(ri, "request") + print_type_helpers(ri, "request") + + +def print_rsp_type_helpers(ri): + if 'reply' not in ri.op[ri.op_mode]: + return + print_type_helpers(ri, "reply") + + +def print_parse_prototype(ri, direction, terminate=True): + suffix = "_rsp" if direction == "reply" else "_req" + term = ';' if terminate else '' + + ri.cw.write_func_prot('void', f"{ri.op.render_name}{suffix}_parse", + ['const struct nlattr **tb', + f"struct {ri.op.render_name}{suffix} *req"], + suffix=term) + + +def print_req_type(ri): + if ri.type_empty("request"): + return + print_type(ri, "request") + + +def print_req_free(ri): + if 'request' not in ri.op[ri.op_mode]: + return + _free_type(ri, 'request', ri.struct['request']) + + +def print_rsp_type(ri): + if ri.op_mode in ('do', 'dump') and 'reply' in ri.op[ri.op_mode]: + direction = 'reply' + elif ri.op_mode == 'event': + direction = 'reply' + else: + return + print_type(ri, direction) + + +def print_wrapped_type(ri): + ri.cw.block_start(line=f"{type_name(ri, 'reply')}") + if ri.op_mode == 'dump': + ri.cw.p(f"{type_name(ri, 'reply')} *next;") + elif ri.op_mode in ('notify', 'event'): + ri.cw.p('__u16 family;') + ri.cw.p('__u8 cmd;') + ri.cw.p('struct ynl_ntf_base_type *next;') + ri.cw.p(f"void (*free)({type_name(ri, 'reply')} *ntf);") + ri.cw.p(f"{type_name(ri, 'reply', deref=True)} obj __attribute__((aligned(8)));") + ri.cw.block_end(line=';') + ri.cw.nl() + print_free_prototype(ri, 'reply') + ri.cw.nl() + + +def _free_type_members_iter(ri, struct): + if struct.free_needs_iter(): + ri.cw.p('unsigned int i;') + ri.cw.nl() + + +def _free_type_members(ri, var, struct, ref=''): + for _, attr in struct.member_list(): + attr.free(ri, var, ref) + + +def _free_type(ri, direction, struct): + var = free_arg_name(direction) + + print_free_prototype(ri, direction, suffix='') + ri.cw.block_start() + _free_type_members_iter(ri, struct) + _free_type_members(ri, var, struct) + if direction: + ri.cw.p(f'free({var});') + ri.cw.block_end() + ri.cw.nl() + + +def free_rsp_nested_prototype(ri): + print_free_prototype(ri, "") + + +def free_rsp_nested(ri, struct): + _free_type(ri, "", struct) + + +def print_rsp_free(ri): + if 'reply' not in ri.op[ri.op_mode]: + return + _free_type(ri, 'reply', ri.struct['reply']) + + +def print_dump_type_free(ri): + sub_type = type_name(ri, 'reply') + + print_free_prototype(ri, 'reply', suffix='') + ri.cw.block_start() + ri.cw.p(f"{sub_type} *next = rsp;") + ri.cw.nl() + ri.cw.p('if (!next)') + ri.cw.p('return;') + ri.cw.nl() + ri.cw.block_start(line='while ((void *)next != YNL_LIST_END)') + _free_type_members_iter(ri, ri.struct['reply']) + ri.cw.p('rsp = next;') + ri.cw.p('next = rsp->next;') + ri.cw.nl() + + _free_type_members(ri, 'rsp', ri.struct['reply'], ref='obj.') + ri.cw.p('free(rsp);') + ri.cw.block_end() + ri.cw.block_end() + ri.cw.nl() + + +def print_ntf_type_free(ri): + print_free_prototype(ri, 'reply', suffix='') + ri.cw.block_start() + _free_type_members_iter(ri, ri.struct['reply']) + _free_type_members(ri, 'rsp', ri.struct['reply'], ref='obj.') + ri.cw.p('free(rsp);') + ri.cw.block_end() + ri.cw.nl() + + +def print_req_policy_fwd(cw, struct, ri=None, terminate=True): + if terminate and ri and policy_should_be_static(struct.family): + return + + if terminate: + prefix = 'extern ' + else: + if ri and policy_should_be_static(struct.family): + prefix = 'static ' + else: + prefix = '' + + suffix = ';' if terminate else ' = {' + + max_attr = struct.attr_max_val + if ri: + name = ri.op.render_name + if ri.op.dual_policy: + name += '_' + ri.op_mode + else: + name = struct.render_name + cw.p(f"{prefix}const struct nla_policy {name}_nl_policy[{max_attr.enum_name} + 1]{suffix}") + + +def print_req_policy(cw, struct, ri=None): + if ri and ri.op: + cw.ifdef_block(ri.op.get('config-cond', None)) + print_req_policy_fwd(cw, struct, ri=ri, terminate=False) + for _, arg in struct.member_list(): + arg.attr_policy(cw) + cw.p("};") + cw.ifdef_block(None) + cw.nl() + + +def kernel_can_gen_family_struct(family): + return family.proto == 'genetlink' + + +def policy_should_be_static(family): + return family.kernel_policy == 'split' or kernel_can_gen_family_struct(family) + + +def print_kernel_policy_ranges(family, cw): + first = True + for _, attr_set in family.attr_sets.items(): + if attr_set.subset_of: + continue + + for _, attr in attr_set.items(): + if not attr.request: + continue + if 'full-range' not in attr.checks: + continue + + if first: + cw.p('/* Integer value ranges */') + first = False + + sign = '' if attr.type[0] == 'u' else '_signed' + suffix = 'ULL' if attr.type[0] == 'u' else 'LL' + cw.block_start(line=f'static const struct netlink_range_validation{sign} {c_lower(attr.enum_name)}_range =') + members = [] + if 'min' in attr.checks: + members.append(('min', attr.get_limit_str('min', suffix=suffix))) + if 'max' in attr.checks: + members.append(('max', attr.get_limit_str('max', suffix=suffix))) + cw.write_struct_init(members) + cw.block_end(line=';') + cw.nl() + + +def print_kernel_policy_sparse_enum_validates(family, cw): + first = True + for _, attr_set in family.attr_sets.items(): + if attr_set.subset_of: + continue + + for _, attr in attr_set.items(): + if not attr.request: + continue + if not attr.enum_name: + continue + if 'sparse' not in attr.checks: + continue + + if first: + cw.p('/* Sparse enums validation callbacks */') + first = False + + cw.write_func_prot('static int', f'{c_lower(attr.enum_name)}_validate', + ['const struct nlattr *attr', 'struct netlink_ext_ack *extack']) + cw.block_start() + cw.block_start(line=f'switch (nla_get_{attr["type"]}(attr))') + enum = family.consts[attr['enum']] + first_entry = True + for entry in enum.entries.values(): + if first_entry: + first_entry = False + else: + cw.p('fallthrough;') + cw.p(f'case {entry.c_name}:') + cw.p('return 0;') + cw.block_end() + cw.p('NL_SET_ERR_MSG_ATTR(extack, attr, "invalid enum value");') + cw.p('return -EINVAL;') + cw.block_end() + cw.nl() + + +def print_kernel_op_table_fwd(family, cw, terminate): + exported = not kernel_can_gen_family_struct(family) + + if not terminate or exported: + cw.p(f"/* Ops table for {family.ident_name} */") + + pol_to_struct = {'global': 'genl_small_ops', + 'per-op': 'genl_ops', + 'split': 'genl_split_ops'} + struct_type = pol_to_struct[family.kernel_policy] + + if not exported: + cnt = "" + elif family.kernel_policy == 'split': + cnt = 0 + for op in family.ops.values(): + if 'do' in op: + cnt += 1 + if 'dump' in op: + cnt += 1 + else: + cnt = len(family.ops) + + qual = 'static const' if not exported else 'const' + line = f"{qual} struct {struct_type} {family.c_name}_nl_ops[{cnt}]" + if terminate: + cw.p(f"extern {line};") + else: + cw.block_start(line=line + ' =') + + if not terminate: + return + + cw.nl() + for name in family.hooks['pre']['do']['list']: + cw.write_func_prot('int', c_lower(name), + ['const struct genl_split_ops *ops', + 'struct sk_buff *skb', 'struct genl_info *info'], suffix=';') + for name in family.hooks['post']['do']['list']: + cw.write_func_prot('void', c_lower(name), + ['const struct genl_split_ops *ops', + 'struct sk_buff *skb', 'struct genl_info *info'], suffix=';') + for name in family.hooks['pre']['dump']['list']: + cw.write_func_prot('int', c_lower(name), + ['struct netlink_callback *cb'], suffix=';') + for name in family.hooks['post']['dump']['list']: + cw.write_func_prot('int', c_lower(name), + ['struct netlink_callback *cb'], suffix=';') + + cw.nl() + + for op_name, op in family.ops.items(): + if op.is_async: + continue + + if 'do' in op: + name = c_lower(f"{family.fn_prefix}-{op_name}-doit") + cw.write_func_prot('int', name, + ['struct sk_buff *skb', 'struct genl_info *info'], suffix=';') + + if 'dump' in op: + name = c_lower(f"{family.fn_prefix}-{op_name}-dumpit") + cw.write_func_prot('int', name, + ['struct sk_buff *skb', 'struct netlink_callback *cb'], suffix=';') + cw.nl() + + +def print_kernel_op_table_hdr(family, cw): + print_kernel_op_table_fwd(family, cw, terminate=True) + + +def print_kernel_op_table(family, cw): + print_kernel_op_table_fwd(family, cw, terminate=False) + if family.kernel_policy in ('global', 'per-op'): + for op_name, op in family.ops.items(): + if op.is_async: + continue + + cw.ifdef_block(op.get('config-cond', None)) + cw.block_start() + members = [('cmd', op.enum_name)] + if 'dont-validate' in op: + members.append(('validate', + ' | '.join([c_upper('genl-dont-validate-' + x) + for x in op['dont-validate']])), ) + for op_mode in ['do', 'dump']: + if op_mode in op: + name = c_lower(f"{family.fn_prefix}-{op_name}-{op_mode}it") + members.append((op_mode + 'it', name)) + if family.kernel_policy == 'per-op': + struct = Struct(family, op['attribute-set'], + type_list=op['do']['request']['attributes']) + + name = c_lower(f"{family.ident_name}-{op_name}-nl-policy") + members.append(('policy', name)) + members.append(('maxattr', struct.attr_max_val.enum_name)) + if 'flags' in op: + members.append(('flags', ' | '.join([c_upper('genl-' + x) for x in op['flags']]))) + cw.write_struct_init(members) + cw.block_end(line=',') + elif family.kernel_policy == 'split': + cb_names = {'do': {'pre': 'pre_doit', 'post': 'post_doit'}, + 'dump': {'pre': 'start', 'post': 'done'}} + + for op_name, op in family.ops.items(): + for op_mode in ['do', 'dump']: + if op.is_async or op_mode not in op: + continue + + cw.ifdef_block(op.get('config-cond', None)) + cw.block_start() + members = [('cmd', op.enum_name)] + if 'dont-validate' in op: + dont_validate = [] + for x in op['dont-validate']: + if op_mode == 'do' and x in ['dump', 'dump-strict']: + continue + if op_mode == "dump" and x == 'strict': + continue + dont_validate.append(x) + + if dont_validate: + members.append(('validate', + ' | '.join([c_upper('genl-dont-validate-' + x) + for x in dont_validate])), ) + name = c_lower(f"{family.fn_prefix}-{op_name}-{op_mode}it") + if 'pre' in op[op_mode]: + members.append((cb_names[op_mode]['pre'], c_lower(op[op_mode]['pre']))) + members.append((op_mode + 'it', name)) + if 'post' in op[op_mode]: + members.append((cb_names[op_mode]['post'], c_lower(op[op_mode]['post']))) + if 'request' in op[op_mode]: + struct = Struct(family, op['attribute-set'], + type_list=op[op_mode]['request']['attributes']) + + if op.dual_policy: + name = c_lower(f"{family.ident_name}-{op_name}-{op_mode}-nl-policy") + else: + name = c_lower(f"{family.ident_name}-{op_name}-nl-policy") + members.append(('policy', name)) + members.append(('maxattr', struct.attr_max_val.enum_name)) + flags = (op['flags'] if 'flags' in op else []) + ['cmd-cap-' + op_mode] + members.append(('flags', ' | '.join([c_upper('genl-' + x) for x in flags]))) + cw.write_struct_init(members) + cw.block_end(line=',') + cw.ifdef_block(None) + + cw.block_end(line=';') + cw.nl() + + +def print_kernel_mcgrp_hdr(family, cw): + if not family.mcgrps['list']: + return + + cw.block_start('enum') + for grp in family.mcgrps['list']: + grp_id = c_upper(f"{family.ident_name}-nlgrp-{grp['name']},") + cw.p(grp_id) + cw.block_end(';') + cw.nl() + + +def print_kernel_mcgrp_src(family, cw): + if not family.mcgrps['list']: + return + + cw.block_start('static const struct genl_multicast_group ' + family.c_name + '_nl_mcgrps[] =') + for grp in family.mcgrps['list']: + name = grp['name'] + grp_id = c_upper(f"{family.ident_name}-nlgrp-{name}") + cw.p('[' + grp_id + '] = { "' + name + '", },') + cw.block_end(';') + cw.nl() + + +def print_kernel_family_struct_hdr(family, cw): + if not kernel_can_gen_family_struct(family): + return + + cw.p(f"extern struct genl_family {family.c_name}_nl_family;") + cw.nl() + if 'sock-priv' in family.kernel_family: + cw.p(f'void {family.c_name}_nl_sock_priv_init({family.kernel_family["sock-priv"]} *priv);') + cw.p(f'void {family.c_name}_nl_sock_priv_destroy({family.kernel_family["sock-priv"]} *priv);') + cw.nl() + + +def print_kernel_family_struct_src(family, cw): + if not kernel_can_gen_family_struct(family): + return + + if 'sock-priv' in family.kernel_family: + # Generate "trampolines" to make CFI happy + cw.write_func("static void", f"__{family.c_name}_nl_sock_priv_init", + [f"{family.c_name}_nl_sock_priv_init(priv);"], + ["void *priv"]) + cw.nl() + cw.write_func("static void", f"__{family.c_name}_nl_sock_priv_destroy", + [f"{family.c_name}_nl_sock_priv_destroy(priv);"], + ["void *priv"]) + cw.nl() + + cw.block_start(f"struct genl_family {family.ident_name}_nl_family __ro_after_init =") + cw.p('.name\t\t= ' + family.fam_key + ',') + cw.p('.version\t= ' + family.ver_key + ',') + cw.p('.netnsok\t= true,') + cw.p('.parallel_ops\t= true,') + cw.p('.module\t\t= THIS_MODULE,') + if family.kernel_policy == 'per-op': + cw.p(f'.ops\t\t= {family.c_name}_nl_ops,') + cw.p(f'.n_ops\t\t= ARRAY_SIZE({family.c_name}_nl_ops),') + elif family.kernel_policy == 'split': + cw.p(f'.split_ops\t= {family.c_name}_nl_ops,') + cw.p(f'.n_split_ops\t= ARRAY_SIZE({family.c_name}_nl_ops),') + if family.mcgrps['list']: + cw.p(f'.mcgrps\t\t= {family.c_name}_nl_mcgrps,') + cw.p(f'.n_mcgrps\t= ARRAY_SIZE({family.c_name}_nl_mcgrps),') + if 'sock-priv' in family.kernel_family: + cw.p(f'.sock_priv_size\t= sizeof({family.kernel_family["sock-priv"]}),') + cw.p(f'.sock_priv_init\t= __{family.c_name}_nl_sock_priv_init,') + cw.p(f'.sock_priv_destroy = __{family.c_name}_nl_sock_priv_destroy,') + cw.block_end(';') + + +def uapi_enum_start(family, cw, obj, ckey='', enum_name='enum-name'): + start_line = 'enum' + if enum_name in obj: + if obj[enum_name]: + start_line = 'enum ' + c_lower(obj[enum_name]) + elif ckey and ckey in obj: + start_line = 'enum ' + family.c_name + '_' + c_lower(obj[ckey]) + cw.block_start(line=start_line) + + +def render_uapi_unified(family, cw, max_by_define, separate_ntf): + max_name = c_upper(family.get('cmd-max-name', f"{family.op_prefix}MAX")) + cnt_name = c_upper(family.get('cmd-cnt-name', f"__{family.op_prefix}MAX")) + max_value = f"({cnt_name} - 1)" + + uapi_enum_start(family, cw, family['operations'], 'enum-name') + val = 0 + for op in family.msgs.values(): + if separate_ntf and ('notify' in op or 'event' in op): + continue + + suffix = ',' + if op.value != val: + suffix = f" = {op.value}," + val = op.value + cw.p(op.enum_name + suffix) + val += 1 + cw.nl() + cw.p(cnt_name + ('' if max_by_define else ',')) + if not max_by_define: + cw.p(f"{max_name} = {max_value}") + cw.block_end(line=';') + if max_by_define: + cw.p(f"#define {max_name} {max_value}") + cw.nl() + + +def render_uapi_directional(family, cw, max_by_define): + max_name = f"{family.op_prefix}USER_MAX" + cnt_name = f"__{family.op_prefix}USER_CNT" + max_value = f"({cnt_name} - 1)" + + cw.block_start(line='enum') + cw.p(c_upper(f'{family.name}_MSG_USER_NONE = 0,')) + val = 0 + for op in family.msgs.values(): + if 'do' in op and 'event' not in op: + suffix = ',' + if op.value and op.value != val: + suffix = f" = {op.value}," + val = op.value + cw.p(op.enum_name + suffix) + val += 1 + cw.nl() + cw.p(cnt_name + ('' if max_by_define else ',')) + if not max_by_define: + cw.p(f"{max_name} = {max_value}") + cw.block_end(line=';') + if max_by_define: + cw.p(f"#define {max_name} {max_value}") + cw.nl() + + max_name = f"{family.op_prefix}KERNEL_MAX" + cnt_name = f"__{family.op_prefix}KERNEL_CNT" + max_value = f"({cnt_name} - 1)" + + cw.block_start(line='enum') + cw.p(c_upper(f'{family.name}_MSG_KERNEL_NONE = 0,')) + val = 0 + for op in family.msgs.values(): + if ('do' in op and 'reply' in op['do']) or 'notify' in op or 'event' in op: + enum_name = op.enum_name + if 'event' not in op and 'notify' not in op: + enum_name = f'{enum_name}_REPLY' + + suffix = ',' + if op.value and op.value != val: + suffix = f" = {op.value}," + val = op.value + cw.p(enum_name + suffix) + val += 1 + cw.nl() + cw.p(cnt_name + ('' if max_by_define else ',')) + if not max_by_define: + cw.p(f"{max_name} = {max_value}") + cw.block_end(line=';') + if max_by_define: + cw.p(f"#define {max_name} {max_value}") + cw.nl() + + +def render_uapi(family, cw): + hdr_prot = f"_UAPI_LINUX_{c_upper(family.uapi_header_name)}_H" + hdr_prot = hdr_prot.replace('/', '_') + cw.p('#ifndef ' + hdr_prot) + cw.p('#define ' + hdr_prot) + cw.nl() + + defines = [(family.fam_key, family["name"]), + (family.ver_key, family.get('version', 1))] + cw.writes_defines(defines) + cw.nl() + + defines = [] + for const in family['definitions']: + if const.get('header'): + continue + if const.get('scope', 'uapi') != 'uapi': + continue + + if const['type'] != 'const': + cw.writes_defines(defines) + defines = [] + cw.nl() + + # Write kdoc for enum and flags (one day maybe also structs) + if const['type'] == 'enum' or const['type'] == 'flags': + enum = family.consts[const['name']] + + if enum.header: + continue + + if enum.has_doc(): + if enum.has_entry_doc(): + cw.p('/**') + doc = '' + if 'doc' in enum: + doc = ' - ' + enum['doc'] + cw.write_doc_line(enum.enum_name + doc) + else: + cw.p('/*') + cw.write_doc_line(enum['doc'], indent=False) + for entry in enum.entries.values(): + if entry.has_doc(): + doc = '@' + entry.c_name + ': ' + entry['doc'] + cw.write_doc_line(doc) + cw.p(' */') + + uapi_enum_start(family, cw, const, 'name') + name_pfx = const.get('name-prefix', f"{family.ident_name}-{const['name']}-") + for entry in enum.entries.values(): + suffix = ',' + if entry.value_change: + suffix = f" = {entry.user_value()}" + suffix + cw.p(entry.c_name + suffix) + + if const.get('render-max', False): + cw.nl() + cw.p('/* private: */') + if const['type'] == 'flags': + max_name = c_upper(name_pfx + 'mask') + max_val = f' = {enum.get_mask()},' + cw.p(max_name + max_val) + else: + cnt_name = enum.enum_cnt_name + max_name = c_upper(name_pfx + 'max') + if not cnt_name: + cnt_name = '__' + name_pfx + 'max' + cw.p(c_upper(cnt_name) + ',') + cw.p(max_name + ' = (' + c_upper(cnt_name) + ' - 1)') + cw.block_end(line=';') + cw.nl() + elif const['type'] == 'const': + name_pfx = const.get('name-prefix', f"{family.ident_name}-") + defines.append([c_upper(family.get('c-define-name', + f"{name_pfx}{const['name']}")), + const['value']]) + + if defines: + cw.writes_defines(defines) + cw.nl() + + max_by_define = family.get('max-by-define', False) + + for _, attr_set in family.attr_sets.items(): + if attr_set.subset_of: + continue + + max_value = f"({attr_set.cnt_name} - 1)" + + val = 0 + uapi_enum_start(family, cw, attr_set.yaml, 'enum-name') + for _, attr in attr_set.items(): + suffix = ',' + if attr.value != val: + suffix = f" = {attr.value}," + val = attr.value + val += 1 + cw.p(attr.enum_name + suffix) + if attr_set.items(): + cw.nl() + cw.p(attr_set.cnt_name + ('' if max_by_define else ',')) + if not max_by_define: + cw.p(f"{attr_set.max_name} = {max_value}") + cw.block_end(line=';') + if max_by_define: + cw.p(f"#define {attr_set.max_name} {max_value}") + cw.nl() + + # Commands + separate_ntf = 'async-prefix' in family['operations'] + + if family.msg_id_model == 'unified': + render_uapi_unified(family, cw, max_by_define, separate_ntf) + elif family.msg_id_model == 'directional': + render_uapi_directional(family, cw, max_by_define) + else: + raise Exception(f'Unsupported message enum-model {family.msg_id_model}') + + if separate_ntf: + uapi_enum_start(family, cw, family['operations'], enum_name='async-enum') + for op in family.msgs.values(): + if separate_ntf and not ('notify' in op or 'event' in op): + continue + + suffix = ',' + if 'value' in op: + suffix = f" = {op['value']}," + cw.p(op.enum_name + suffix) + cw.block_end(line=';') + cw.nl() + + # Multicast + defines = [] + for grp in family.mcgrps['list']: + name = grp['name'] + defines.append([c_upper(grp.get('c-define-name', f"{family.ident_name}-mcgrp-{name}")), + f'{name}']) + cw.nl() + if defines: + cw.writes_defines(defines) + cw.nl() + + cw.p(f'#endif /* {hdr_prot} */') + + +def render_scoped_consts(family, cw, scope): + defines = [] + for const in family['definitions']: + if const['type'] != 'const': + continue + if const.get('header'): + continue + if const.get('scope') != scope: + continue + name_pfx = const.get('name-prefix', f"{family.ident_name}-") + defines.append([ + c_upper(family.get('c-define-name', + f"{name_pfx}{const['name']}")), + const['value']]) + if defines: + cw.writes_defines(defines) + cw.nl() + + +def _render_user_ntf_entry(ri, op): + if not ri.family.is_classic(): + ri.cw.block_start(line=f"[{op.enum_name}] = ") + else: + crud_op = ri.family.req_by_value[op.rsp_value] + ri.cw.block_start(line=f"[{crud_op.enum_name}] = ") + ri.cw.p(f".alloc_sz\t= sizeof({type_name(ri, 'event')}),") + ri.cw.p(f".cb\t\t= {op_prefix(ri, 'reply', deref=True)}_parse,") + ri.cw.p(f".policy\t\t= &{ri.struct['reply'].render_name}_nest,") + ri.cw.p(f".free\t\t= (void *){op_prefix(ri, 'notify')}_free,") + ri.cw.block_end(line=',') + + +def render_user_family(family, cw, prototype): + symbol = f'const struct ynl_family ynl_{family.c_name}_family' + if prototype: + cw.p(f'extern {symbol};') + return + + if family.ntfs: + cw.block_start(line=f"static const struct ynl_ntf_info {family.c_name}_ntf_info[] = ") + for ntf_op_name, ntf_op in family.ntfs.items(): + if 'notify' in ntf_op: + op = family.ops[ntf_op['notify']] + ri = RenderInfo(cw, family, "user", op, "notify") + elif 'event' in ntf_op: + ri = RenderInfo(cw, family, "user", ntf_op, "event") + else: + raise Exception('Invalid notification ' + ntf_op_name) + _render_user_ntf_entry(ri, ntf_op) + for _op_name, op in family.ops.items(): + if 'event' not in op: + continue + ri = RenderInfo(cw, family, "user", op, "event") + _render_user_ntf_entry(ri, op) + cw.block_end(line=";") + cw.nl() + + cw.block_start(f'{symbol} = ') + cw.p(f'.name\t\t= "{family.c_name}",') + if family.is_classic(): + cw.p('.is_classic\t= true,') + cw.p(f'.classic_id\t= {family.get("protonum")},') + if family.is_classic(): + if family.fixed_header: + cw.p(f'.hdr_len\t= sizeof(struct {c_lower(family.fixed_header)}),') + elif family.fixed_header: + cw.p(f'.hdr_len\t= sizeof(struct genlmsghdr) + sizeof(struct {c_lower(family.fixed_header)}),') + else: + cw.p('.hdr_len\t= sizeof(struct genlmsghdr),') + if family.ntfs: + cw.p(f".ntf_info\t= {family.c_name}_ntf_info,") + cw.p(f".ntf_info_size\t= YNL_ARRAY_SIZE({family.c_name}_ntf_info),") + cw.block_end(line=';') + + +def family_contains_bitfield32(family): + for _, attr_set in family.attr_sets.items(): + if attr_set.subset_of: + continue + for _, attr in attr_set.items(): + if attr.type == "bitfield32": + return True + return False + + +def find_kernel_root(full_path): + sub_path = '' + while True: + sub_path = os.path.join(os.path.basename(full_path), sub_path) + full_path = os.path.dirname(full_path) + maintainers = os.path.join(full_path, "MAINTAINERS") + if os.path.exists(maintainers): + return full_path, sub_path[:-1] + + +def main(): + parser = argparse.ArgumentParser(description='Netlink simple parsing generator') + parser.add_argument('--mode', dest='mode', type=str, required=True, + choices=('user', 'kernel', 'uapi')) + parser.add_argument('--spec', dest='spec', type=str, required=True) + parser.add_argument('--header', dest='header', action='store_true', default=None) + parser.add_argument('--source', dest='header', action='store_false') + parser.add_argument('--user-header', nargs='+', default=[]) + parser.add_argument('--cmp-out', action='store_true', default=None, + help='Do not overwrite the output file if the new output is identical to the old') + parser.add_argument('--exclude-op', action='append', default=[]) + parser.add_argument('-o', dest='out_file', type=str, default=None) + parser.add_argument('--function-prefix', dest='fn_prefix', type=str) + args = parser.parse_args() + + if args.header is None: + parser.error("--header or --source is required") + + exclude_ops = [re.compile(expr) for expr in args.exclude_op] + + try: + parsed = Family(args.spec, exclude_ops, args.fn_prefix) + if parsed.license != '((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)': + print('Spec license:', parsed.license) + print('License must be: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)') + os.sys.exit(1) + except pyyaml.YAMLError as exc: + print(exc) + os.sys.exit(1) + + cw = CodeWriter(BaseNlLib(), args.out_file, overwrite=not args.cmp_out) + + _, spec_kernel = find_kernel_root(args.spec) + if args.mode == 'uapi' or args.header: + cw.p(f'/* SPDX-License-Identifier: {parsed.license} */') + else: + cw.p(f'// SPDX-License-Identifier: {parsed.license}') + cw.p("/* Do not edit directly, auto-generated from: */") + cw.p(f"/*\t{spec_kernel} */") + cw.p(f"/* YNL-GEN {args.mode} {'header' if args.header else 'source'} */") + if args.exclude_op or args.user_header or args.fn_prefix: + line = '' + if args.user_header: + line += ' --user-header '.join([''] + args.user_header) + if args.exclude_op: + line += ' --exclude-op '.join([''] + args.exclude_op) + if args.fn_prefix: + line += f' --function-prefix {args.fn_prefix}' + cw.p(f'/* YNL-ARG{line} */') + cw.p('/* To regenerate run: tools/net/ynl/ynl-regen.sh */') + cw.nl() + + if args.mode == 'uapi': + render_uapi(parsed, cw) + return + + hdr_prot = f"_LINUX_{parsed.c_name.upper()}_GEN_H" + if args.header: + cw.p('#ifndef ' + hdr_prot) + cw.p('#define ' + hdr_prot) + cw.nl() + + if args.out_file: + hdr_file = os.path.basename(args.out_file[:-2]) + ".h" + else: + hdr_file = "generated_header_file.h" + + if args.mode == 'kernel': + cw.p('#include <net/netlink.h>') + cw.p('#include <net/genetlink.h>') + cw.nl() + if not args.header: + if args.out_file: + cw.p(f'#include "{hdr_file}"') + cw.nl() + headers = ['uapi/' + parsed.uapi_header] + headers += parsed.kernel_family.get('headers', []) + else: + cw.p('#include <stdlib.h>') + cw.p('#include <string.h>') + if args.header: + cw.p('#include <linux/types.h>') + if family_contains_bitfield32(parsed): + cw.p('#include <linux/netlink.h>') + else: + cw.p(f'#include "{hdr_file}"') + cw.p('#include "ynl.h"') + headers = [] + for definition in parsed['definitions'] + parsed['attribute-sets']: + if 'header' not in definition: + continue + scope = definition.get('scope', 'uapi') + if scope != 'uapi' and scope != args.mode: + continue + headers.append(definition['header']) + if args.mode == 'user': + headers.append(parsed.uapi_header) + seen_header = [] + for one in headers: + if one not in seen_header: + cw.p(f"#include <{one}>") + seen_header.append(one) + cw.nl() + + if args.mode == "user": + if not args.header: + cw.p("#include <linux/genetlink.h>") + cw.nl() + for one in args.user_header: + cw.p(f'#include "{one}"') + else: + render_scoped_consts(parsed, cw, 'user') + cw.p('struct ynl_sock;') + cw.nl() + render_user_family(parsed, cw, True) + cw.nl() + + if args.mode == "kernel": + if args.header: + render_scoped_consts(parsed, cw, 'kernel') + for _, struct in sorted(parsed.pure_nested_structs.items()): + if struct.request: + cw.p('/* Common nested types */') + break + for attr_set, struct in sorted(parsed.pure_nested_structs.items()): + if struct.request: + print_req_policy_fwd(cw, struct) + cw.nl() + + if parsed.kernel_policy == 'global': + cw.p(f"/* Global operation policy for {parsed.name} */") + + struct = Struct(parsed, parsed.global_policy_set, type_list=parsed.global_policy) + print_req_policy_fwd(cw, struct) + cw.nl() + + if parsed.kernel_policy in {'per-op', 'split'}: + for _op_name, op in parsed.ops.items(): + if 'do' in op and 'event' not in op: + ri = RenderInfo(cw, parsed, args.mode, op, "do") + print_req_policy_fwd(cw, ri.struct['request'], ri=ri) + cw.nl() + + print_kernel_op_table_hdr(parsed, cw) + print_kernel_mcgrp_hdr(parsed, cw) + print_kernel_family_struct_hdr(parsed, cw) + else: + print_kernel_policy_ranges(parsed, cw) + print_kernel_policy_sparse_enum_validates(parsed, cw) + + for _, struct in sorted(parsed.pure_nested_structs.items()): + if struct.request: + cw.p('/* Common nested types */') + break + for attr_set, struct in sorted(parsed.pure_nested_structs.items()): + if struct.request: + print_req_policy(cw, struct) + cw.nl() + + if parsed.kernel_policy == 'global': + cw.p(f"/* Global operation policy for {parsed.name} */") + + struct = Struct(parsed, parsed.global_policy_set, type_list=parsed.global_policy) + print_req_policy(cw, struct) + cw.nl() + + for _op_name, op in parsed.ops.items(): + if parsed.kernel_policy in {'per-op', 'split'}: + for op_mode in ['do', 'dump']: + if op_mode in op and 'request' in op[op_mode]: + cw.p(f"/* {op.enum_name} - {op_mode} */") + ri = RenderInfo(cw, parsed, args.mode, op, op_mode) + print_req_policy(cw, ri.struct['request'], ri=ri) + cw.nl() + + print_kernel_op_table(parsed, cw) + print_kernel_mcgrp_src(parsed, cw) + print_kernel_family_struct_src(parsed, cw) + + if args.mode == "user": + if args.header: + cw.p('/* Enums */') + put_op_name_fwd(parsed, cw) + + for name, const in parsed.consts.items(): + if isinstance(const, EnumSet): + put_enum_to_str_fwd(parsed, cw, const) + cw.nl() + + cw.p('/* Common nested types */') + for attr_set, struct in parsed.pure_nested_structs.items(): + ri = RenderInfo(cw, parsed, args.mode, "", "", attr_set) + print_type_full(ri, struct) + + for _op_name, op in parsed.ops.items(): + cw.p(f"/* ============== {op.enum_name} ============== */") + + if 'do' in op and 'event' not in op: + cw.p(f"/* {op.enum_name} - do */") + ri = RenderInfo(cw, parsed, args.mode, op, "do") + print_req_type(ri) + print_req_type_helpers(ri) + cw.nl() + print_rsp_type(ri) + print_rsp_type_helpers(ri) + cw.nl() + print_req_prototype(ri) + cw.nl() + + if 'dump' in op: + cw.p(f"/* {op.enum_name} - dump */") + ri = RenderInfo(cw, parsed, args.mode, op, 'dump') + print_req_type(ri) + print_req_type_helpers(ri) + if not ri.type_consistent or ri.type_oneside: + print_rsp_type(ri) + print_wrapped_type(ri) + print_dump_prototype(ri) + cw.nl() + + if op.has_ntf: + cw.p(f"/* {op.enum_name} - notify */") + ri = RenderInfo(cw, parsed, args.mode, op, 'notify') + if not ri.type_consistent: + raise Exception(f'Only notifications with consistent types supported ({op.name})') + print_wrapped_type(ri) + + for _op_name, op in parsed.ntfs.items(): + if 'event' in op: + ri = RenderInfo(cw, parsed, args.mode, op, 'event') + cw.p(f"/* {op.enum_name} - event */") + print_rsp_type(ri) + cw.nl() + print_wrapped_type(ri) + cw.nl() + else: + cw.p('/* Enums */') + put_op_name(parsed, cw) + + for name, const in parsed.consts.items(): + if isinstance(const, EnumSet): + put_enum_to_str(parsed, cw, const) + cw.nl() + + has_recursive_nests = False + cw.p('/* Policies */') + for struct in parsed.pure_nested_structs.values(): + if struct.recursive: + put_typol_fwd(cw, struct) + has_recursive_nests = True + if has_recursive_nests: + cw.nl() + for struct in parsed.pure_nested_structs.values(): + put_typol(cw, struct) + for name in parsed.root_sets: + struct = Struct(parsed, name) + put_typol(cw, struct) + + cw.p('/* Common nested types */') + if has_recursive_nests: + for attr_set, struct in parsed.pure_nested_structs.items(): + ri = RenderInfo(cw, parsed, args.mode, "", "", attr_set) + free_rsp_nested_prototype(ri) + if struct.request: + put_req_nested_prototype(ri, struct) + if struct.reply: + parse_rsp_nested_prototype(ri, struct) + cw.nl() + for attr_set, struct in parsed.pure_nested_structs.items(): + ri = RenderInfo(cw, parsed, args.mode, "", "", attr_set) + + free_rsp_nested(ri, struct) + if struct.request: + put_req_nested(ri, struct) + if struct.reply: + parse_rsp_nested(ri, struct) + + for _op_name, op in parsed.ops.items(): + cw.p(f"/* ============== {op.enum_name} ============== */") + if 'do' in op and 'event' not in op: + cw.p(f"/* {op.enum_name} - do */") + ri = RenderInfo(cw, parsed, args.mode, op, "do") + print_req_free(ri) + print_rsp_free(ri) + parse_rsp_msg(ri) + print_req(ri) + cw.nl() + + if 'dump' in op: + cw.p(f"/* {op.enum_name} - dump */") + ri = RenderInfo(cw, parsed, args.mode, op, "dump") + if not ri.type_consistent or ri.type_oneside: + parse_rsp_msg(ri, deref=True) + print_req_free(ri) + print_dump_type_free(ri) + print_dump(ri) + cw.nl() + + if op.has_ntf: + cw.p(f"/* {op.enum_name} - notify */") + ri = RenderInfo(cw, parsed, args.mode, op, 'notify') + if not ri.type_consistent: + raise Exception(f'Only notifications with consistent types supported ({op.name})') + print_ntf_type_free(ri) + + for _op_name, op in parsed.ntfs.items(): + if 'event' in op: + cw.p(f"/* {op.enum_name} - event */") + + ri = RenderInfo(cw, parsed, args.mode, op, "do") + parse_rsp_msg(ri) + + ri = RenderInfo(cw, parsed, args.mode, op, "event") + print_ntf_type_free(ri) + cw.nl() + render_user_family(parsed, cw, False) + + if args.header: + cw.p(f'#endif /* {hdr_prot} */') + + +if __name__ == "__main__": + main() diff --git a/tools/net/ynl/pyynl/ynl_gen_rst.py b/tools/net/ynl/pyynl/ynl_gen_rst.py new file mode 100755 index 000000000..30324e2fd --- /dev/null +++ b/tools/net/ynl/pyynl/ynl_gen_rst.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# -*- coding: utf-8; mode: python -*- + +""" + Script to auto generate the documentation for Netlink specifications. + + :copyright: Copyright (C) 2023 Breno Leitao <leitao@debian.org> + :license: GPL Version 2, June 1991 see linux/COPYING for details. + + This script performs extensive parsing to the Linux kernel's netlink YAML + spec files, in an effort to avoid needing to heavily mark up the original + YAML file. It uses the library code from scripts/lib. +""" + +import os.path +import pathlib +import sys +import argparse +import logging + +# pylint: disable=no-name-in-module,wrong-import-position +sys.path.append(pathlib.Path(__file__).resolve().parent.as_posix()) +from lib import YnlDocGenerator # pylint: disable=C0413 + +def parse_arguments() -> argparse.Namespace: + """Parse arguments from user""" + parser = argparse.ArgumentParser(description="Netlink RST generator") + + parser.add_argument("-v", "--verbose", action="store_true") + parser.add_argument("-o", "--output", help="Output file name") + + # Index and input are mutually exclusive + group = parser.add_mutually_exclusive_group() + group.add_argument("-i", "--input", help="YAML file name") + + args = parser.parse_args() + + if args.verbose: + logging.basicConfig(level=logging.DEBUG) + + if args.input and not os.path.isfile(args.input): + logging.warning("%s is not a valid file.", args.input) + sys.exit(-1) + + if not args.output: + logging.error("No output file specified.") + sys.exit(-1) + + if os.path.isfile(args.output): + logging.debug("%s already exists. Overwriting it.", args.output) + + return args + + +def write_to_rstfile(content: str, filename: str) -> None: + """Write the generated content into an RST file""" + logging.debug("Saving RST file to %s", filename) + + with open(filename, "w", encoding="utf-8") as rst_file: + rst_file.write(content) + + +# pylint: disable=broad-exception-caught +def main() -> None: + """Main function that reads the YAML files and generates the RST files""" + + args = parse_arguments() + + parser = YnlDocGenerator() + + if args.input: + logging.debug("Parsing %s", args.input) + try: + content = parser.parse_yaml_file(os.path.join(args.input)) + except Exception as exception: + logging.warning("Failed to parse %s.", args.input) + logging.warning(exception) + sys.exit(-1) + + write_to_rstfile(content, args.output) + + +if __name__ == "__main__": + main() diff --git a/tools/net/ynl/requirements.txt b/tools/net/ynl/requirements.txt new file mode 100644 index 000000000..0db6ad0c1 --- /dev/null +++ b/tools/net/ynl/requirements.txt @@ -0,0 +1,2 @@ +jsonschema==4.* +PyYAML==6.* diff --git a/tools/net/ynl/tests/.gitignore b/tools/net/ynl/tests/.gitignore new file mode 100644 index 000000000..a7832ebfd --- /dev/null +++ b/tools/net/ynl/tests/.gitignore @@ -0,0 +1,10 @@ +devlink +ethtool +netdev +ovs +rt-addr +rt-link +rt-route +tc +tc-filter-add +wireguard diff --git a/tools/net/ynl/tests/Makefile b/tools/net/ynl/tests/Makefile new file mode 100644 index 000000000..40827ca8e --- /dev/null +++ b/tools/net/ynl/tests/Makefile @@ -0,0 +1,97 @@ +# SPDX-License-Identifier: GPL-2.0 +# Makefile for YNL tests + +include ../Makefile.deps + +CC=gcc +CFLAGS += -std=gnu11 -O2 -W -Wall -Wextra -Wno-unused-parameter -Wshadow \ + -I../lib/ -I../generated/ -I../../../testing/selftests/ \ + -idirafter $(UAPI_PATH) +ifneq ("$(NDEBUG)","1") + CFLAGS += -g -fsanitize=address -fsanitize=leak -static-libasan +endif + +LDLIBS=../lib/ynl.a ../generated/protos.a + +TEST_PROGS := \ + devlink.sh \ + ethtool.sh \ + rt-addr.sh \ + rt-route.sh \ + test_ynl_cli.sh \ + test_ynl_ethtool.sh \ +# end of TEST_PROGS + +TEST_GEN_PROGS := \ + netdev \ + ovs \ + rt-link \ + tc \ +# end of TEST_GEN_PROGS + +TEST_GEN_FILES := \ + devlink \ + ethtool \ + rt-addr \ + rt-route \ +# end of TEST_GEN_FILES + +TEST_FILES := \ + ethtool.py \ + ynl_nsim_lib.sh \ +# end of TEST_FILES + +CFLAGS_netdev:=$(CFLAGS_netdev) $(CFLAGS_rt-link) +CFLAGS_ovs:=$(CFLAGS_ovs_datapath) + +include $(wildcard *.d) + +INSTALL_PATH ?= $(DESTDIR)/usr/share/kselftest + +all: $(TEST_GEN_PROGS) $(TEST_GEN_FILES) + +../lib/ynl.a: + @$(MAKE) -C ../lib + +../generated/protos.a: + @$(MAKE) -C ../generated + +$(TEST_GEN_PROGS) $(TEST_GEN_FILES): %: %.c ../lib/ynl.a ../generated/protos.a + @echo -e '\tCC test $@' + @$(COMPILE.c) $(CFLAGS_$@) $@.c -o $@.o + @$(LINK.c) $@.o -o $@ $(LDLIBS) + +run_tests: + @for test in $(TEST_PROGS); do \ + ./$$test; \ + done + +install: $(TEST_GEN_PROGS) $(TEST_GEN_FILES) + @mkdir -p $(INSTALL_PATH)/ynl + @cp ../../../testing/selftests/kselftest/ktap_helpers.sh $(INSTALL_PATH)/ + @for test in $(TEST_PROGS); do \ + name=$$(basename $$test); \ + sed -e 's|^ynl=.*|ynl="ynl"|' \ + -e 's|^ynl_ethtool=.*|ynl_ethtool="ynl-ethtool"|' \ + -e 's|KSELFTEST_KTAP_HELPERS=.*|KSELFTEST_KTAP_HELPERS="$(INSTALL_PATH)/ktap_helpers.sh"|' \ + $$test > $(INSTALL_PATH)/ynl/$$name; \ + chmod +x $(INSTALL_PATH)/ynl/$$name; \ + done + @for file in $(TEST_FILES); do \ + cp $$file $(INSTALL_PATH)/ynl/$$file; \ + done + @for bin in $(TEST_GEN_PROGS) $(TEST_GEN_FILES); do \ + cp $$bin $(INSTALL_PATH)/ynl/$$bin; \ + done + @for test in $(TEST_PROGS) $(TEST_GEN_PROGS); do \ + echo "ynl:$$test"; \ + done > $(INSTALL_PATH)/kselftest-list.txt + +clean: + rm -f *.o *.d *~ + +distclean: clean + rm -f $(TEST_GEN_PROGS) $(TEST_GEN_FILES) + +.PHONY: all install clean distclean run_tests +.DEFAULT_GOAL=all diff --git a/tools/net/ynl/tests/config b/tools/net/ynl/tests/config new file mode 100644 index 000000000..75c0fe723 --- /dev/null +++ b/tools/net/ynl/tests/config @@ -0,0 +1,14 @@ +CONFIG_DUMMY=m +CONFIG_INET_DIAG=y +CONFIG_IPV6=y +CONFIG_NET_ACT_VLAN=m +CONFIG_NET_CLS_ACT=y +CONFIG_NET_CLS_FLOWER=m +CONFIG_NET_SCH_FQ_CODEL=m +CONFIG_NET_SCH_INGRESS=m +CONFIG_NET_NS=y +CONFIG_NET_SCHED=y +CONFIG_NETDEVSIM=m +CONFIG_NETKIT=y +CONFIG_OPENVSWITCH=m +CONFIG_VETH=m diff --git a/tools/net/ynl/tests/devlink.c b/tools/net/ynl/tests/devlink.c new file mode 100644 index 000000000..2e668bb15 --- /dev/null +++ b/tools/net/ynl/tests/devlink.c @@ -0,0 +1,101 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <stdio.h> +#include <string.h> + +#include <ynl.h> + +#include <kselftest_harness.h> + +#include "devlink-user.h" + +FIXTURE(devlink) +{ + struct ynl_sock *ys; +}; + +FIXTURE_SETUP(devlink) +{ + self->ys = ynl_sock_create(&ynl_devlink_family, NULL); + ASSERT_NE(NULL, self->ys) + TH_LOG("failed to create devlink socket"); +} + +FIXTURE_TEARDOWN(devlink) +{ + ynl_sock_destroy(self->ys); +} + +TEST_F(devlink, dump) +{ + struct devlink_get_list *devs; + + devs = devlink_get_dump(self->ys); + ASSERT_NE(NULL, devs) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + if (ynl_dump_empty(devs)) { + devlink_get_list_free(devs); + SKIP(return, "no entries in dump"); + } + + ynl_dump_foreach(devs, d) { + EXPECT_TRUE((bool)d->_len.bus_name); + EXPECT_TRUE((bool)d->_len.dev_name); + ksft_print_msg("%s/%s\n", d->bus_name, d->dev_name); + } + + devlink_get_list_free(devs); +} + +TEST_F(devlink, info) +{ + struct devlink_get_list *devs; + + devs = devlink_get_dump(self->ys); + ASSERT_NE(NULL, devs) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + if (ynl_dump_empty(devs)) { + devlink_get_list_free(devs); + SKIP(return, "no devices to query"); + } + + ynl_dump_foreach(devs, d) { + struct devlink_info_get_req *info_req; + struct devlink_info_get_rsp *info_rsp; + unsigned int i; + + EXPECT_TRUE((bool)d->_len.bus_name); + EXPECT_TRUE((bool)d->_len.dev_name); + ksft_print_msg("%s/%s:\n", d->bus_name, d->dev_name); + + info_req = devlink_info_get_req_alloc(); + ASSERT_NE(NULL, info_req); + devlink_info_get_req_set_bus_name(info_req, d->bus_name); + devlink_info_get_req_set_dev_name(info_req, d->dev_name); + + info_rsp = devlink_info_get(self->ys, info_req); + devlink_info_get_req_free(info_req); + ASSERT_NE(NULL, info_rsp) { + devlink_get_list_free(devs); + TH_LOG("info_get failed: %s", self->ys->err.msg); + } + + EXPECT_TRUE((bool)info_rsp->_len.info_driver_name); + if (info_rsp->_len.info_driver_name) + ksft_print_msg(" driver: %s\n", + info_rsp->info_driver_name); + if (info_rsp->_count.info_version_running) + ksft_print_msg(" running fw:\n"); + for (i = 0; i < info_rsp->_count.info_version_running; i++) + ksft_print_msg(" %s: %s\n", + info_rsp->info_version_running[i].info_version_name, + info_rsp->info_version_running[i].info_version_value); + devlink_info_get_rsp_free(info_rsp); + } + devlink_get_list_free(devs); +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/devlink.sh b/tools/net/ynl/tests/devlink.sh new file mode 100755 index 000000000..a684c749a --- /dev/null +++ b/tools/net/ynl/tests/devlink.sh @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +source "$(dirname "$(realpath "$0")")/ynl_nsim_lib.sh" +nsim_setup +"$(dirname "$(realpath "$0")")/devlink" diff --git a/tools/net/ynl/tests/ethtool.c b/tools/net/ynl/tests/ethtool.c new file mode 100644 index 000000000..926a75d23 --- /dev/null +++ b/tools/net/ynl/tests/ethtool.c @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <stdio.h> +#include <string.h> + +#include <ynl.h> + +#include <net/if.h> + +#include <kselftest_harness.h> + +#include "ethtool-user.h" + +FIXTURE(ethtool) +{ + struct ynl_sock *ys; +}; + +FIXTURE_SETUP(ethtool) +{ + self->ys = ynl_sock_create(&ynl_ethtool_family, NULL); + ASSERT_NE(NULL, self->ys) + TH_LOG("failed to create ethtool socket"); +} + +FIXTURE_TEARDOWN(ethtool) +{ + ynl_sock_destroy(self->ys); +} + +TEST_F(ethtool, channels) +{ + struct ethtool_channels_get_req_dump creq = {}; + struct ethtool_channels_get_list *channels; + + creq._present.header = 1; /* ethtool needs an empty nest */ + channels = ethtool_channels_get_dump(self->ys, &creq); + ASSERT_NE(NULL, channels) { + TH_LOG("channels dump failed: %s", self->ys->err.msg); + } + + if (ynl_dump_empty(channels)) { + ethtool_channels_get_list_free(channels); + SKIP(return, "no entries in channels dump"); + } + + ynl_dump_foreach(channels, dev) { + EXPECT_TRUE((bool)dev->header._len.dev_name); + ksft_print_msg("%8s: ", dev->header.dev_name); + EXPECT_TRUE(dev->_present.rx_count || + dev->_present.tx_count || + dev->_present.combined_count); + if (dev->_present.rx_count) + printf("rx %d ", dev->rx_count); + if (dev->_present.tx_count) + printf("tx %d ", dev->tx_count); + if (dev->_present.combined_count) + printf("combined %d ", dev->combined_count); + printf("\n"); + } + ethtool_channels_get_list_free(channels); +} + +TEST_F(ethtool, rings) +{ + struct ethtool_rings_get_req_dump rreq = {}; + struct ethtool_rings_get_list *rings; + + rreq._present.header = 1; /* ethtool needs an empty nest */ + rings = ethtool_rings_get_dump(self->ys, &rreq); + ASSERT_NE(NULL, rings) { + TH_LOG("rings dump failed: %s", self->ys->err.msg); + } + + if (ynl_dump_empty(rings)) { + ethtool_rings_get_list_free(rings); + SKIP(return, "no entries in rings dump"); + } + + ynl_dump_foreach(rings, dev) { + EXPECT_TRUE((bool)dev->header._len.dev_name); + ksft_print_msg("%8s: ", dev->header.dev_name); + EXPECT_TRUE(dev->_present.rx || dev->_present.tx); + if (dev->_present.rx) + printf("rx %d ", dev->rx); + if (dev->_present.tx) + printf("tx %d ", dev->tx); + printf("\n"); + } + ethtool_rings_get_list_free(rings); +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/ethtool.py b/tools/net/ynl/tests/ethtool.py new file mode 100755 index 000000000..0ee0c8e87 --- /dev/null +++ b/tools/net/ynl/tests/ethtool.py @@ -0,0 +1,464 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +# +# pylint: disable=too-many-locals, too-many-branches, too-many-statements +# pylint: disable=too-many-return-statements + +""" YNL ethtool utility """ + +import argparse +import pathlib +import pprint +import sys +import re + +# pylint: disable=no-name-in-module,wrong-import-position +sys.path.append(pathlib.Path(__file__).resolve().parent.parent.joinpath('pyynl').as_posix()) +# pylint: disable=import-error +from lib import YnlFamily + + +def args_to_req(ynl, op_name, args, req): + """ + Verify and convert command-line arguments to the ynl-compatible request. + """ + valid_attrs = ynl.operation_do_attributes(op_name) + valid_attrs.remove('header') # not user-provided + + if len(args) == 0: + print(f'no attributes, expected: {valid_attrs}') + sys.exit(1) + + i = 0 + while i < len(args): + attr = args[i] + if i + 1 >= len(args): + print(f'expected value for \'{attr}\'') + sys.exit(1) + + if attr not in valid_attrs: + print(f'invalid attribute \'{attr}\', expected: {valid_attrs}') + sys.exit(1) + + val = args[i+1] + i += 2 + + req[attr] = val + +def print_field(reply, *desc): + """ + Pretty-print a set of fields from the reply. desc specifies the + fields and the optional type (bool/yn). + """ + if not reply: + return + + if len(desc) == 0: + print_field(reply, *zip(reply.keys(), reply.keys())) + return + + for spec in desc: + try: + field, name, tp = spec + except ValueError: + field, name = spec + tp = 'int' + + value = reply.get(field, None) + if tp == 'yn': + value = 'yes' if value else 'no' + elif tp == 'bool' or isinstance(value, bool): + value = 'on' if value else 'off' + else: + value = 'n/a' if value is None else value + + print(f'{name}: {value}') + +def print_speed(name, value): + """ + Print out the speed-like strings from the value dict. + """ + speed_re = re.compile(r'[0-9]+base[^/]+/.+') + speed = [ k for k, v in value.items() if v and speed_re.match(k) ] + print(f'{name}: {" ".join(speed)}') + +def do_set(ynl, args, op_name): + """ + Prepare request header, parse arguments and do a set operation. + """ + req = { + 'header': { + 'dev-name': args.device, + }, + } + + args_to_req(ynl, op_name, args.args, req) + ynl.do(op_name, req) + +def do_get(ynl, args, op_name, extra=None): + """ + Prepare request header and get info for a specific device using doit. + """ + extra = extra or {} + req = {'header': {'dev-name': args.device}} + req['header'].update(extra.pop('header', {})) + req.update(extra) + + reply = ynl.do(op_name, req) + if not reply: + return {} + + if args.json: + pprint.PrettyPrinter().pprint(reply) + sys.exit(0) + reply.pop('header', None) + return reply + +def bits_to_dict(attr): + """ + Convert ynl-formatted bitmask to a dict of bit=value. + """ + ret = {} + if 'bits' not in attr: + return {} + if 'bit' not in attr['bits']: + return {} + for bit in attr['bits']['bit']: + if bit['name'] == '': + continue + name = bit['name'] + value = bit.get('value', False) + ret[name] = value + return ret + +def main(): + """ YNL ethtool utility """ + + parser = argparse.ArgumentParser(description='ethtool wannabe') + parser.add_argument('--json', action=argparse.BooleanOptionalAction) + parser.add_argument('--show-priv-flags', action=argparse.BooleanOptionalAction) + parser.add_argument('--set-priv-flags', action=argparse.BooleanOptionalAction) + parser.add_argument('--show-eee', action=argparse.BooleanOptionalAction) + parser.add_argument('--set-eee', action=argparse.BooleanOptionalAction) + parser.add_argument('-a', '--show-pause', action=argparse.BooleanOptionalAction) + parser.add_argument('-A', '--set-pause', action=argparse.BooleanOptionalAction) + parser.add_argument('-c', '--show-coalesce', action=argparse.BooleanOptionalAction) + parser.add_argument('-C', '--set-coalesce', action=argparse.BooleanOptionalAction) + parser.add_argument('-g', '--show-ring', action=argparse.BooleanOptionalAction) + parser.add_argument('-G', '--set-ring', action=argparse.BooleanOptionalAction) + parser.add_argument('-k', '--show-features', action=argparse.BooleanOptionalAction) + parser.add_argument('-K', '--set-features', action=argparse.BooleanOptionalAction) + parser.add_argument('-l', '--show-channels', action=argparse.BooleanOptionalAction) + parser.add_argument('-L', '--set-channels', action=argparse.BooleanOptionalAction) + parser.add_argument('-T', '--show-time-stamping', action=argparse.BooleanOptionalAction) + parser.add_argument('-S', '--statistics', action=argparse.BooleanOptionalAction) + # TODO: --show-tunnels tunnel-info-get + # TODO: --show-module module-get + # TODO: --get-plca-cfg plca-get + # TODO: --get-plca-status plca-get-status + # TODO: --show-mm mm-get + # TODO: --show-fec fec-get + # TODO: --dump-module-eerpom module-eeprom-get + # TODO: pse-get + # TODO: rss-get + parser.add_argument('device', metavar='device', type=str) + parser.add_argument('args', metavar='args', type=str, nargs='*') + + dbg_group = parser.add_argument_group('Debug options') + dbg_group.add_argument('--dbg-small-recv', default=0, const=4000, + action='store', nargs='?', type=int, metavar='INT', + help="Length of buffers used for recv()") + + args = parser.parse_args() + + ynl = YnlFamily(family='ethtool', recv_size=args.dbg_small_recv) + if args.dbg_small_recv: + ynl.set_recv_dbg(True) + + if args.set_priv_flags: + # TODO: parse the bitmask + print("not implemented") + return + + if args.set_eee: + do_set(ynl, args, 'eee-set') + return + + if args.set_pause: + do_set(ynl, args, 'pause-set') + return + + if args.set_coalesce: + do_set(ynl, args, 'coalesce-set') + return + + if args.set_features: + # TODO: parse the bitmask + print("not implemented") + return + + if args.set_channels: + do_set(ynl, args, 'channels-set') + return + + if args.set_ring: + do_set(ynl, args, 'rings-set') + return + + if args.show_priv_flags: + flags = bits_to_dict(do_get(ynl, args, 'privflags-get')['flags']) + print_field(flags) + return + + if args.show_eee: + eee = do_get(ynl, args, 'eee-get') + ours = bits_to_dict(eee['modes-ours']) + peer = bits_to_dict(eee['modes-peer']) + + if 'enabled' in eee: + status = 'enabled' if eee['enabled'] else 'disabled' + if 'active' in eee and eee['active']: + status = status + ' - active' + else: + status = status + ' - inactive' + else: + status = 'not supported' + + print(f'EEE status: {status}') + print_field(eee, ('tx-lpi-timer', 'Tx LPI')) + print_speed('Advertised EEE link modes', ours) + print_speed('Link partner advertised EEE link modes', peer) + + return + + if args.show_pause: + print_field(do_get(ynl, args, 'pause-get'), + ('autoneg', 'Autonegotiate', 'bool'), + ('rx', 'RX', 'bool'), + ('tx', 'TX', 'bool')) + return + + if args.show_coalesce: + print_field(do_get(ynl, args, 'coalesce-get')) + return + + if args.show_features: + reply = do_get(ynl, args, 'features-get') + available = bits_to_dict(reply['hw']) + requested = bits_to_dict(reply['wanted']).keys() + active = bits_to_dict(reply['active']).keys() + never_changed = bits_to_dict(reply['nochange']).keys() + + for f in sorted(available): + value = "off" + if f in active: + value = "on" + + fixed = "" + if f not in available or f in never_changed: + fixed = " [fixed]" + + req = "" + if f in requested: + if f in active: + req = " [requested on]" + else: + req = " [requested off]" + + print(f'{f}: {value}{fixed}{req}') + + return + + if args.show_channels: + reply = do_get(ynl, args, 'channels-get') + print(f'Channel parameters for {args.device}:') + + print('Pre-set maximums:') + print_field(reply, + ('rx-max', 'RX'), + ('tx-max', 'TX'), + ('other-max', 'Other'), + ('combined-max', 'Combined')) + + print('Current hardware settings:') + print_field(reply, + ('rx-count', 'RX'), + ('tx-count', 'TX'), + ('other-count', 'Other'), + ('combined-count', 'Combined')) + + return + + if args.show_ring: + reply = do_get(ynl, args, 'channels-get') + + print(f'Ring parameters for {args.device}:') + + print('Pre-set maximums:') + print_field(reply, + ('rx-max', 'RX'), + ('rx-mini-max', 'RX Mini'), + ('rx-jumbo-max', 'RX Jumbo'), + ('tx-max', 'TX')) + + print('Current hardware settings:') + print_field(reply, + ('rx', 'RX'), + ('rx-mini', 'RX Mini'), + ('rx-jumbo', 'RX Jumbo'), + ('tx', 'TX')) + + print_field(reply, + ('rx-buf-len', 'RX Buf Len'), + ('cqe-size', 'CQE Size'), + ('tx-push', 'TX Push', 'bool')) + + return + + if args.statistics: + print('NIC statistics:') + + # TODO: pass id? + strset = do_get(ynl, args, 'strset-get') + pprint.PrettyPrinter().pprint(strset) + + req = { + 'groups': { + 'size': 1, + 'bits': { + 'bit': + # TODO: support passing the bitmask + #[ + #{ 'name': 'eth-phy', 'value': True }, + { 'name': 'eth-mac', 'value': True }, + #{ 'name': 'eth-ctrl', 'value': True }, + #{ 'name': 'rmon', 'value': True }, + #], + }, + }, + } + + rsp = do_get(ynl, args, 'stats-get', req) + pprint.PrettyPrinter().pprint(rsp) + return + + if args.show_time_stamping: + req = { + 'header': { + 'flags': 'stats', + }, + } + + tsinfo = do_get(ynl, args, 'tsinfo-get', req) + + print(f'Time stamping parameters for {args.device}:') + + print('Capabilities:') + _ = [print(f'\t{v}') for v in bits_to_dict(tsinfo['timestamping'])] + + print(f'PTP Hardware Clock: {tsinfo.get("phc-index", "none")}') + + if 'tx-types' in tsinfo: + print('Hardware Transmit Timestamp Modes:') + _ = [print(f'\t{v}') for v in bits_to_dict(tsinfo['tx-types'])] + else: + print('Hardware Transmit Timestamp Modes: none') + + if 'rx-filters' in tsinfo: + print('Hardware Receive Filter Modes:') + _ = [print(f'\t{v}') for v in bits_to_dict(tsinfo['rx-filters'])] + else: + print('Hardware Receive Filter Modes: none') + + if 'stats' in tsinfo and tsinfo['stats']: + print('Statistics:') + _ = [print(f'\t{k}: {v}') for k, v in tsinfo['stats'].items()] + + return + + print(f'Settings for {args.device}:') + linkmodes = do_get(ynl, args, 'linkmodes-get') + ours = bits_to_dict(linkmodes['ours']) + + supported_ports = ('TP', 'AUI', 'BNC', 'MII', 'FIBRE', 'Backplane') + ports = [ p for p in supported_ports if ours.get(p, False)] + print(f'Supported ports: [ {" ".join(ports)} ]') + + print_speed('Supported link modes', ours) + + print_field(ours, ('Pause', 'Supported pause frame use', 'yn')) + print_field(ours, ('Autoneg', 'Supports auto-negotiation', 'yn')) + + supported_fec = ('None', 'PS', 'BASER', 'LLRS') + fec = [ p for p in supported_fec if ours.get(p, False)] + fec_str = " ".join(fec) + if len(fec) == 0: + fec_str = "Not reported" + + print(f'Supported FEC modes: {fec_str}') + + speed = 'Unknown!' + if linkmodes['speed'] > 0 and linkmodes['speed'] < 0xffffffff: + speed = f'{linkmodes["speed"]}Mb/s' + print(f'Speed: {speed}') + + duplex_modes = { + 0: 'Half', + 1: 'Full', + } + duplex = duplex_modes.get(linkmodes["duplex"], None) + if not duplex: + duplex = f'Unknown! ({linkmodes["duplex"]})' + print(f'Duplex: {duplex}') + + autoneg = "off" + if linkmodes.get("autoneg", 0) != 0: + autoneg = "on" + print(f'Auto-negotiation: {autoneg}') + + ports = { + 0: 'Twisted Pair', + 1: 'AUI', + 2: 'MII', + 3: 'FIBRE', + 4: 'BNC', + 5: 'Directly Attached Copper', + 0xef: 'None', + } + linkinfo = do_get(ynl, args, 'linkinfo-get') + print(f'Port: {ports.get(linkinfo["port"], "Other")}') + + print_field(linkinfo, ('phyaddr', 'PHYAD')) + + transceiver = { + 0: 'Internal', + 1: 'External', + } + print(f'Transceiver: {transceiver.get(linkinfo["transceiver"], "Unknown")}') + + mdix_ctrl = { + 1: 'off', + 2: 'on', + } + mdix = mdix_ctrl.get(linkinfo['tp-mdix-ctrl'], None) + if mdix: + mdix = mdix + ' (forced)' + else: + mdix = mdix_ctrl.get(linkinfo['tp-mdix'], 'Unknown (auto)') + print(f'MDI-X: {mdix}') + + debug = do_get(ynl, args, 'debug-get') + msgmask = bits_to_dict(debug.get("msgmask", [])).keys() + print(f'Current message level: {" ".join(msgmask)}') + + linkstate = do_get(ynl, args, 'linkstate-get') + detected_states = { + 0: 'no', + 1: 'yes', + } + # TODO: wol-get + detected = detected_states.get(linkstate['link'], 'unknown') + print(f'Link detected: {detected}') + +if __name__ == '__main__': + main() diff --git a/tools/net/ynl/tests/ethtool.sh b/tools/net/ynl/tests/ethtool.sh new file mode 100755 index 000000000..0859ddd69 --- /dev/null +++ b/tools/net/ynl/tests/ethtool.sh @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +source "$(dirname "$(realpath "$0")")/ynl_nsim_lib.sh" +nsim_setup +"$(dirname "$(realpath "$0")")/ethtool" diff --git a/tools/net/ynl/tests/netdev.c b/tools/net/ynl/tests/netdev.c new file mode 100644 index 000000000..f849e3d7f --- /dev/null +++ b/tools/net/ynl/tests/netdev.c @@ -0,0 +1,231 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <stdio.h> +#include <string.h> + +#include <ynl.h> + +#include <net/if.h> + +#include <kselftest_harness.h> + +#include "netdev-user.h" +#include "rt-link-user.h" + +static void netdev_print_device(struct __test_metadata *_metadata, + struct netdev_dev_get_rsp *d, unsigned int op) +{ + char ifname[IF_NAMESIZE]; + const char *name; + + EXPECT_TRUE((bool)d->_present.ifindex); + if (!d->_present.ifindex) + return; + + name = if_indextoname(d->ifindex, ifname); + EXPECT_TRUE((bool)name); + if (name) + ksft_print_msg("%8s[%d]\t", name, d->ifindex); + else + ksft_print_msg("[%d]\t", d->ifindex); + + EXPECT_TRUE((bool)d->_present.xdp_features); + if (!d->_present.xdp_features) + return; + + printf("xdp-features (%llx):", d->xdp_features); + for (int i = 0; d->xdp_features >= 1U << i; i++) { + if (d->xdp_features & (1U << i)) + printf(" %s", netdev_xdp_act_str(1 << i)); + } + + printf(" xdp-rx-metadata-features (%llx):", + d->xdp_rx_metadata_features); + for (int i = 0; d->xdp_rx_metadata_features >= 1U << i; i++) { + if (d->xdp_rx_metadata_features & (1U << i)) + printf(" %s", + netdev_xdp_rx_metadata_str(1 << i)); + } + + printf(" xsk-features (%llx):", d->xsk_features); + for (int i = 0; d->xsk_features >= 1U << i; i++) { + if (d->xsk_features & (1U << i)) + printf(" %s", netdev_xsk_flags_str(1 << i)); + } + + printf(" xdp-zc-max-segs=%u", d->xdp_zc_max_segs); + + name = netdev_op_str(op); + if (name) + printf(" (ntf: %s)", name); + printf("\n"); +} + +static int veth_create(struct ynl_sock *ys_link) +{ + struct rt_link_getlink_ntf *ntf_gl; + struct rt_link_newlink_req *req; + struct ynl_ntf_base_type *ntf; + int ret; + + req = rt_link_newlink_req_alloc(); + if (!req) + return -1; + + rt_link_newlink_req_set_nlflags(req, NLM_F_CREATE | NLM_F_ECHO); + rt_link_newlink_req_set_linkinfo_kind(req, "veth"); + + ret = rt_link_newlink(ys_link, req); + rt_link_newlink_req_free(req); + if (ret) + return -1; + + if (!ynl_has_ntf(ys_link)) + return 0; + + ntf = ynl_ntf_dequeue(ys_link); + if (!ntf || ntf->cmd != RTM_NEWLINK) { + ynl_ntf_free(ntf); + return 0; + } + ntf_gl = (void *)ntf; + ret = ntf_gl->obj._hdr.ifi_index; + ynl_ntf_free(ntf); + + return ret; +} + +static void veth_delete(struct __test_metadata *_metadata, + struct ynl_sock *ys_link, int ifindex) +{ + struct rt_link_dellink_req *req; + + req = rt_link_dellink_req_alloc(); + ASSERT_NE(NULL, req); + + req->_hdr.ifi_index = ifindex; + EXPECT_EQ(0, rt_link_dellink(ys_link, req)); + rt_link_dellink_req_free(req); +} + +FIXTURE(netdev) +{ + struct ynl_sock *ys; + struct ynl_sock *ys_link; +}; + +FIXTURE_SETUP(netdev) +{ + struct ynl_error yerr; + + self->ys = ynl_sock_create(&ynl_netdev_family, &yerr); + ASSERT_NE(NULL, self->ys) { + TH_LOG("Failed to create YNL netdev socket: %s", yerr.msg); + } +} + +FIXTURE_TEARDOWN(netdev) +{ + if (self->ys_link) + ynl_sock_destroy(self->ys_link); + ynl_sock_destroy(self->ys); +} + +TEST_F(netdev, dump) +{ + struct netdev_dev_get_list *devs; + + devs = netdev_dev_get_dump(self->ys); + ASSERT_NE(NULL, devs) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + if (ynl_dump_empty(devs)) { + netdev_dev_get_list_free(devs); + SKIP(return, "no entries in dump"); + } + + ynl_dump_foreach(devs, d) + netdev_print_device(_metadata, d, 0); + + netdev_dev_get_list_free(devs); +} + +TEST_F(netdev, get) +{ + struct netdev_dev_get_list *devs; + struct netdev_dev_get_req *req; + struct netdev_dev_get_rsp *dev; + int ifindex = 0; + + devs = netdev_dev_get_dump(self->ys); + ASSERT_NE(NULL, devs) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + ynl_dump_foreach(devs, d) { + if (d->_present.ifindex) { + ifindex = d->ifindex; + break; + } + } + netdev_dev_get_list_free(devs); + + if (!ifindex) + SKIP(return, "no device to query"); + + req = netdev_dev_get_req_alloc(); + ASSERT_NE(NULL, req); + netdev_dev_get_req_set_ifindex(req, ifindex); + + dev = netdev_dev_get(self->ys, req); + netdev_dev_get_req_free(req); + ASSERT_NE(NULL, dev) { + TH_LOG("dev_get failed: %s", self->ys->err.msg); + } + + netdev_print_device(_metadata, dev, 0); + netdev_dev_get_rsp_free(dev); +} + +TEST_F(netdev, ntf_check) +{ + struct ynl_ntf_base_type *ntf; + int veth_ifindex; + bool received; + int ret; + + ret = ynl_subscribe(self->ys, "mgmt"); + ASSERT_EQ(0, ret) { + TH_LOG("subscribe failed: %s", self->ys->err.msg); + } + + self->ys_link = ynl_sock_create(&ynl_rt_link_family, NULL); + ASSERT_NE(NULL, self->ys_link) + TH_LOG("failed to create rt-link socket"); + + veth_ifindex = veth_create(self->ys_link); + ASSERT_GT(veth_ifindex, 0) + TH_LOG("failed to create veth"); + + ynl_ntf_check(self->ys); + + ntf = ynl_ntf_dequeue(self->ys); + received = ntf; + if (ntf) { + netdev_print_device(_metadata, + (struct netdev_dev_get_rsp *)&ntf->data, + ntf->cmd); + ynl_ntf_free(ntf); + } + + /* Drain any remaining notifications */ + while ((ntf = ynl_ntf_dequeue(self->ys))) + ynl_ntf_free(ntf); + + veth_delete(_metadata, self->ys_link, veth_ifindex); + + ASSERT_TRUE(received) + TH_LOG("no notification received"); +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/ovs.c b/tools/net/ynl/tests/ovs.c new file mode 100644 index 000000000..d49f5a8e6 --- /dev/null +++ b/tools/net/ynl/tests/ovs.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <stdio.h> +#include <string.h> + +#include <ynl.h> + +#include <kselftest_harness.h> + +#include "ovs_datapath-user.h" + +static void ovs_print_datapath(struct __test_metadata *_metadata, + struct ovs_datapath_get_rsp *dp) +{ + EXPECT_TRUE((bool)dp->_len.name); + if (!dp->_len.name) + return; + + EXPECT_TRUE((bool)dp->_hdr.dp_ifindex); + ksft_print_msg("%s(%d): pid:%u cache:%u\n", + dp->name, dp->_hdr.dp_ifindex, + dp->upcall_pid, dp->masks_cache_size); +} + +FIXTURE(ovs) +{ + struct ynl_sock *ys; + char *dp_name; +}; + +FIXTURE_SETUP(ovs) +{ + self->ys = ynl_sock_create(&ynl_ovs_datapath_family, NULL); + ASSERT_NE(NULL, self->ys) + TH_LOG("failed to create OVS datapath socket"); +} + +FIXTURE_TEARDOWN(ovs) +{ + if (self->dp_name) { + struct ovs_datapath_del_req *req; + + req = ovs_datapath_del_req_alloc(); + if (req) { + ovs_datapath_del_req_set_name(req, self->dp_name); + ovs_datapath_del(self->ys, req); + ovs_datapath_del_req_free(req); + } + } + ynl_sock_destroy(self->ys); +} + +TEST_F(ovs, crud) +{ + struct ovs_datapath_get_req_dump *dreq; + struct ovs_datapath_new_req *new_req; + struct ovs_datapath_get_list *dps; + struct ovs_datapath_get_rsp *dp; + struct ovs_datapath_get_req *req; + bool found = false; + int err; + + new_req = ovs_datapath_new_req_alloc(); + ASSERT_NE(NULL, new_req); + ovs_datapath_new_req_set_upcall_pid(new_req, 1); + ovs_datapath_new_req_set_name(new_req, "ynl-test"); + + err = ovs_datapath_new(self->ys, new_req); + ovs_datapath_new_req_free(new_req); + ASSERT_EQ(0, err) { + TH_LOG("new failed: %s", self->ys->err.msg); + } + self->dp_name = "ynl-test"; + + ksft_print_msg("get:\n"); + req = ovs_datapath_get_req_alloc(); + ASSERT_NE(NULL, req); + ovs_datapath_get_req_set_name(req, "ynl-test"); + + dp = ovs_datapath_get(self->ys, req); + ovs_datapath_get_req_free(req); + ASSERT_NE(NULL, dp) { + TH_LOG("get failed: %s", self->ys->err.msg); + } + + ovs_print_datapath(_metadata, dp); + EXPECT_STREQ("ynl-test", dp->name); + ovs_datapath_get_rsp_free(dp); + + ksft_print_msg("dump:\n"); + dreq = ovs_datapath_get_req_dump_alloc(); + ASSERT_NE(NULL, dreq); + + dps = ovs_datapath_get_dump(self->ys, dreq); + ovs_datapath_get_req_dump_free(dreq); + ASSERT_NE(NULL, dps) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + ynl_dump_foreach(dps, d) { + ovs_print_datapath(_metadata, d); + if (d->name && !strcmp(d->name, "ynl-test")) + found = true; + } + ovs_datapath_get_list_free(dps); + EXPECT_TRUE(found); +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/rt-addr.c b/tools/net/ynl/tests/rt-addr.c new file mode 100644 index 000000000..f6c3715b2 --- /dev/null +++ b/tools/net/ynl/tests/rt-addr.c @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <stdio.h> +#include <string.h> + +#include <ynl.h> + +#include <arpa/inet.h> +#include <net/if.h> + +#include <kselftest_harness.h> + +#include "rt-addr-user.h" + +static void rt_addr_print(struct __test_metadata *_metadata, + struct rt_addr_getaddr_rsp *a) +{ + char ifname[IF_NAMESIZE]; + char addr_str[64]; + const char *addr; + const char *name; + + name = if_indextoname(a->_hdr.ifa_index, ifname); + EXPECT_NE(NULL, name); + if (name) + ksft_print_msg("%16s: ", name); + + EXPECT_TRUE(a->_len.address == 4 || a->_len.address == 16); + switch (a->_len.address) { + case 4: + addr = inet_ntop(AF_INET, a->address, + addr_str, sizeof(addr_str)); + break; + case 16: + addr = inet_ntop(AF_INET6, a->address, + addr_str, sizeof(addr_str)); + break; + default: + addr = NULL; + break; + } + if (addr) + printf("%s", addr); + else + printf("[%d]", a->_len.address); + + printf("\n"); +} + +FIXTURE(rt_addr) +{ + struct ynl_sock *ys; +}; + +FIXTURE_SETUP(rt_addr) +{ + struct ynl_error yerr; + + self->ys = ynl_sock_create(&ynl_rt_addr_family, &yerr); + ASSERT_NE(NULL, self->ys) + TH_LOG("failed to create rt-addr socket: %s", yerr.msg); +} + +FIXTURE_TEARDOWN(rt_addr) +{ + ynl_sock_destroy(self->ys); +} + +TEST_F(rt_addr, dump) +{ + struct rt_addr_getaddr_list *rsp; + struct rt_addr_getaddr_req *req; + struct in6_addr v6_expected; + struct in_addr v4_expected; + bool found_v4 = false; + bool found_v6 = false; + + /* The bash wrapper for this test adds these addresses on nsim0, + * make sure we can find them in the dump. + */ + inet_pton(AF_INET, "192.168.1.1", &v4_expected); + inet_pton(AF_INET6, "2001:db8::1", &v6_expected); + + req = rt_addr_getaddr_req_alloc(); + ASSERT_NE(NULL, req); + + rsp = rt_addr_getaddr_dump(self->ys, req); + rt_addr_getaddr_req_free(req); + ASSERT_NE(NULL, rsp) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + ASSERT_FALSE(ynl_dump_empty(rsp)) { + rt_addr_getaddr_list_free(rsp); + TH_LOG("no addresses reported"); + } + + ynl_dump_foreach(rsp, addr) { + rt_addr_print(_metadata, addr); + + found_v4 |= addr->_len.address == 4 && + !memcmp(addr->address, &v4_expected, 4); + found_v6 |= addr->_len.address == 16 && + !memcmp(addr->address, &v6_expected, 16); + } + rt_addr_getaddr_list_free(rsp); + + EXPECT_TRUE(found_v4); + EXPECT_TRUE(found_v6); +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/rt-addr.sh b/tools/net/ynl/tests/rt-addr.sh new file mode 100755 index 000000000..87661236d --- /dev/null +++ b/tools/net/ynl/tests/rt-addr.sh @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +source "$(dirname "$(realpath "$0")")/ynl_nsim_lib.sh" +nsim_setup +"$(dirname "$(realpath "$0")")/rt-addr" diff --git a/tools/net/ynl/tests/rt-link.c b/tools/net/ynl/tests/rt-link.c new file mode 100644 index 000000000..ef619ce61 --- /dev/null +++ b/tools/net/ynl/tests/rt-link.c @@ -0,0 +1,206 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <stdio.h> +#include <string.h> + +#include <ynl.h> + +#include <arpa/inet.h> +#include <net/if.h> + +#include <kselftest_harness.h> + +#include "rt-link-user.h" + +static void rt_link_print(struct __test_metadata *_metadata, + struct rt_link_getlink_rsp *r) +{ + unsigned int i; + + EXPECT_TRUE((bool)r->_hdr.ifi_index); + ksft_print_msg("%3d: ", r->_hdr.ifi_index); + + EXPECT_TRUE((bool)r->_len.ifname); + if (r->_len.ifname) + printf("%6s: ", r->ifname); + + if (r->_present.mtu) + printf("mtu %5d ", r->mtu); + + if (r->linkinfo._len.kind) + printf("kind %-8s ", r->linkinfo.kind); + else + printf(" %8s ", ""); + + if (r->prop_list._count.alt_ifname) { + printf("altname "); + for (i = 0; i < r->prop_list._count.alt_ifname; i++) + printf("%s ", r->prop_list.alt_ifname[i]->str); + printf(" "); + } + + if (r->linkinfo._present.data && r->linkinfo.data._present.netkit) { + struct rt_link_linkinfo_netkit_attrs *netkit; + const char *name; + + netkit = &r->linkinfo.data.netkit; + printf("primary %d ", netkit->primary); + + name = NULL; + if (netkit->_present.policy) + name = rt_link_netkit_policy_str(netkit->policy); + if (name) + printf("policy %s ", name); + } + + printf("\n"); +} + +static int netkit_create(struct ynl_sock *ys) +{ + struct rt_link_getlink_ntf *ntf_gl; + struct rt_link_newlink_req *req; + struct ynl_ntf_base_type *ntf; + int ret; + + req = rt_link_newlink_req_alloc(); + if (!req) + return -1; + + rt_link_newlink_req_set_nlflags(req, NLM_F_CREATE | NLM_F_ECHO); + rt_link_newlink_req_set_linkinfo_kind(req, "netkit"); + rt_link_newlink_req_set_linkinfo_data_netkit_policy(req, NETKIT_DROP); + + ret = rt_link_newlink(ys, req); + rt_link_newlink_req_free(req); + if (ret) + return -1; + + if (!ynl_has_ntf(ys)) + return 0; + + ntf = ynl_ntf_dequeue(ys); + if (!ntf || ntf->cmd != RTM_NEWLINK) { + ynl_ntf_free(ntf); + return 0; + } + ntf_gl = (void *)ntf; + ret = ntf_gl->obj._hdr.ifi_index; + ynl_ntf_free(ntf); + + return ret; +} + +static void netkit_delete(struct __test_metadata *_metadata, + struct ynl_sock *ys, int ifindex) +{ + struct rt_link_dellink_req *req; + + req = rt_link_dellink_req_alloc(); + ASSERT_NE(NULL, req); + + req->_hdr.ifi_index = ifindex; + EXPECT_EQ(0, rt_link_dellink(ys, req)); + rt_link_dellink_req_free(req); +} + +FIXTURE(rt_link) +{ + struct ynl_sock *ys; +}; + +FIXTURE_SETUP(rt_link) +{ + struct ynl_error yerr; + + self->ys = ynl_sock_create(&ynl_rt_link_family, &yerr); + ASSERT_NE(NULL, self->ys) { + TH_LOG("failed to create rt-link socket: %s", yerr.msg); + } +} + +FIXTURE_TEARDOWN(rt_link) +{ + ynl_sock_destroy(self->ys); +} + +TEST_F(rt_link, dump) +{ + struct rt_link_getlink_req_dump *req; + struct rt_link_getlink_list *rsp; + + req = rt_link_getlink_req_dump_alloc(); + ASSERT_NE(NULL, req); + rsp = rt_link_getlink_dump(self->ys, req); + rt_link_getlink_req_dump_free(req); + ASSERT_NE(NULL, rsp) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + ASSERT_FALSE(ynl_dump_empty(rsp)); + + ynl_dump_foreach(rsp, link) + rt_link_print(_metadata, link); + + rt_link_getlink_list_free(rsp); +} + +TEST_F(rt_link, netkit) +{ + struct rt_link_getlink_req_dump *dreq; + struct rt_link_getlink_list *rsp; + bool found = false; + int netkit_ifindex; + + /* Create netkit with valid policy */ + netkit_ifindex = netkit_create(self->ys); + ASSERT_GT(netkit_ifindex, 0) + TH_LOG("failed to create netkit: %s", self->ys->err.msg); + + /* Verify it appears in a dump */ + dreq = rt_link_getlink_req_dump_alloc(); + ASSERT_NE(NULL, dreq); + rsp = rt_link_getlink_dump(self->ys, dreq); + rt_link_getlink_req_dump_free(dreq); + ASSERT_NE(NULL, rsp) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + ynl_dump_foreach(rsp, link) { + if (link->_hdr.ifi_index == netkit_ifindex) { + rt_link_print(_metadata, link); + found = true; + } + } + rt_link_getlink_list_free(rsp); + EXPECT_TRUE(found); + + netkit_delete(_metadata, self->ys, netkit_ifindex); +} + +TEST_F(rt_link, netkit_err_msg) +{ + struct rt_link_newlink_req *req; + int ret; + + /* Test creating netkit with bad policy - should fail */ + req = rt_link_newlink_req_alloc(); + ASSERT_NE(NULL, req); + rt_link_newlink_req_set_nlflags(req, NLM_F_CREATE); + rt_link_newlink_req_set_linkinfo_kind(req, "netkit"); + rt_link_newlink_req_set_linkinfo_data_netkit_policy(req, 10); + + ret = rt_link_newlink(self->ys, req); + rt_link_newlink_req_free(req); + EXPECT_NE(0, ret) { + TH_LOG("creating netkit with bad policy should fail"); + } + + /* Expect: + * Kernel error: 'Provided default xmit policy not supported' (bad attribute: .linkinfo.data(netkit).policy) + */ + EXPECT_NE(NULL, strstr(self->ys->err.msg, "bad attribute: .linkinfo.data(netkit).policy")) { + TH_LOG("expected extack msg not found: %s", + self->ys->err.msg); + } +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/rt-route.c b/tools/net/ynl/tests/rt-route.c new file mode 100644 index 000000000..c9fd2bc48 --- /dev/null +++ b/tools/net/ynl/tests/rt-route.c @@ -0,0 +1,113 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <stdio.h> +#include <string.h> + +#include <ynl.h> + +#include <arpa/inet.h> +#include <net/if.h> + +#include <kselftest_harness.h> + +#include "rt-route-user.h" + +static void rt_route_print(struct __test_metadata *_metadata, + struct rt_route_getroute_rsp *r) +{ + char ifname[IF_NAMESIZE]; + char route_str[64]; + const char *route; + const char *name; + + /* Ignore local */ + if (r->_hdr.rtm_table == RT_TABLE_LOCAL) + return; + + if (r->_present.oif) { + name = if_indextoname(r->oif, ifname); + EXPECT_NE(NULL, name); + if (name) + ksft_print_msg("oif: %-16s ", name); + } + + if (r->_len.dst) { + route = inet_ntop(r->_hdr.rtm_family, r->dst, + route_str, sizeof(route_str)); + printf("dst: %s/%d", route, r->_hdr.rtm_dst_len); + } + + if (r->_len.gateway) { + route = inet_ntop(r->_hdr.rtm_family, r->gateway, + route_str, sizeof(route_str)); + printf("gateway: %s ", route); + } + + printf("\n"); +} + +FIXTURE(rt_route) +{ + struct ynl_sock *ys; +}; + +FIXTURE_SETUP(rt_route) +{ + struct ynl_error yerr; + + self->ys = ynl_sock_create(&ynl_rt_route_family, &yerr); + ASSERT_NE(NULL, self->ys) + TH_LOG("failed to create rt-route socket: %s", yerr.msg); +} + +FIXTURE_TEARDOWN(rt_route) +{ + ynl_sock_destroy(self->ys); +} + +TEST_F(rt_route, dump) +{ + struct rt_route_getroute_req_dump *req; + struct rt_route_getroute_list *rsp; + struct in6_addr v6_expected; + struct in_addr v4_expected; + bool found_v4 = false; + bool found_v6 = false; + + /* The bash wrapper configures 192.168.1.1/24 and 2001:db8::1/64, + * make sure we can find the connected routes in the dump. + */ + inet_pton(AF_INET, "192.168.1.0", &v4_expected); + inet_pton(AF_INET6, "2001:db8::", &v6_expected); + + req = rt_route_getroute_req_dump_alloc(); + ASSERT_NE(NULL, req); + + rsp = rt_route_getroute_dump(self->ys, req); + rt_route_getroute_req_dump_free(req); + ASSERT_NE(NULL, rsp) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + + ASSERT_FALSE(ynl_dump_empty(rsp)) { + rt_route_getroute_list_free(rsp); + TH_LOG("no routes reported"); + } + + ynl_dump_foreach(rsp, route) { + rt_route_print(_metadata, route); + + if (route->_hdr.rtm_table == RT_TABLE_LOCAL) + continue; + + if (route->_len.dst == 4 && route->_hdr.rtm_dst_len == 24) + found_v4 |= !memcmp(route->dst, &v4_expected, 4); + if (route->_len.dst == 16 && route->_hdr.rtm_dst_len == 64) + found_v6 |= !memcmp(route->dst, &v6_expected, 16); + } + rt_route_getroute_list_free(rsp); + + EXPECT_TRUE(found_v4); + EXPECT_TRUE(found_v6); +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/rt-route.sh b/tools/net/ynl/tests/rt-route.sh new file mode 100755 index 000000000..020338f0a --- /dev/null +++ b/tools/net/ynl/tests/rt-route.sh @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +source "$(dirname "$(realpath "$0")")/ynl_nsim_lib.sh" +nsim_setup +"$(dirname "$(realpath "$0")")/rt-route" diff --git a/tools/net/ynl/tests/tc.c b/tools/net/ynl/tests/tc.c new file mode 100644 index 000000000..6ff138765 --- /dev/null +++ b/tools/net/ynl/tests/tc.c @@ -0,0 +1,409 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE +#include <sched.h> +#include <stdio.h> +#include <string.h> +#include <stdlib.h> +#include <arpa/inet.h> +#include <linux/pkt_sched.h> +#include <linux/tc_act/tc_vlan.h> +#include <linux/tc_act/tc_gact.h> +#include <linux/if_ether.h> +#include <net/if.h> + +#include <ynl.h> + +#include <kselftest_harness.h> + +#include "tc-user.h" + +#define TC_HANDLE (0xFFFF << 16) + +static bool tc_qdisc_print(struct __test_metadata *_metadata, + struct tc_getqdisc_rsp *q) +{ + bool was_fq_codel = false; + char ifname[IF_NAMESIZE]; + const char *name; + + name = if_indextoname(q->_hdr.tcm_ifindex, ifname); + EXPECT_TRUE((bool)name); + ksft_print_msg("%16s: ", name ?: "no-name"); + + if (q->_len.kind) { + printf("%s ", q->kind); + + if (q->options._present.fq_codel) { + struct tc_fq_codel_attrs *fq_codel; + struct tc_fq_codel_xstats *stats; + + fq_codel = &q->options.fq_codel; + stats = q->stats2.app.fq_codel; + + EXPECT_EQ(true, + fq_codel->_present.limit && + fq_codel->_present.target && + q->stats2.app._len.fq_codel); + + if (fq_codel->_present.limit) + printf("limit: %dp ", fq_codel->limit); + if (fq_codel->_present.target) + printf("target: %dms ", + (fq_codel->target + 500) / 1000); + if (q->stats2.app._len.fq_codel) + printf("new_flow_cnt: %d ", + stats->qdisc_stats.new_flow_count); + was_fq_codel = true; + } + } + printf("\n"); + + return was_fq_codel; +} + +static const char *vlan_act_name(struct tc_vlan *p) +{ + switch (p->v_action) { + case TCA_VLAN_ACT_POP: + return "pop"; + case TCA_VLAN_ACT_PUSH: + return "push"; + case TCA_VLAN_ACT_MODIFY: + return "modify"; + default: + break; + } + + return "not supported"; +} + +static const char *gact_act_name(struct tc_gact *p) +{ + switch (p->action) { + case TC_ACT_SHOT: + return "drop"; + case TC_ACT_OK: + return "ok"; + case TC_ACT_PIPE: + return "pipe"; + default: + break; + } + + return "not supported"; +} + +static void print_vlan(struct tc_act_vlan_attrs *vlan) +{ + printf("%s ", vlan_act_name(vlan->parms)); + if (vlan->_present.push_vlan_id) + printf("id %u ", vlan->push_vlan_id); + if (vlan->_present.push_vlan_protocol) + printf("protocol %#x ", ntohs(vlan->push_vlan_protocol)); + if (vlan->_present.push_vlan_priority) + printf("priority %u ", vlan->push_vlan_priority); +} + +static void print_gact(struct tc_act_gact_attrs *gact) +{ + struct tc_gact *p = gact->parms; + + printf("%s ", gact_act_name(p)); +} + +static void flower_print(struct tc_flower_attrs *flower, const char *kind) +{ + struct tc_act_attrs *a; + unsigned int i; + + ksft_print_msg("%s:\n", kind); + + if (flower->_present.key_vlan_id) + ksft_print_msg(" vlan_id: %u\n", flower->key_vlan_id); + if (flower->_present.key_vlan_prio) + ksft_print_msg(" vlan_prio: %u\n", flower->key_vlan_prio); + if (flower->_present.key_num_of_vlans) + ksft_print_msg(" num_of_vlans: %u\n", + flower->key_num_of_vlans); + + for (i = 0; i < flower->_count.act; i++) { + a = &flower->act[i]; + ksft_print_msg("action order: %i %s ", i + 1, a->kind); + if (a->options._present.vlan) + print_vlan(&a->options.vlan); + else if (a->options._present.gact) + print_gact(&a->options.gact); + printf("\n"); + } +} + +static void tc_filter_print(struct __test_metadata *_metadata, + struct tc_gettfilter_rsp *f) +{ + struct tc_options_msg *opt = &f->options; + + if (opt->_present.flower) { + EXPECT_TRUE((bool)f->_len.kind); + flower_print(&opt->flower, f->kind); + } else if (f->_len.kind) { + ksft_print_msg("%s pref %u proto: %#x\n", f->kind, + (f->_hdr.tcm_info >> 16), + ntohs(TC_H_MIN(f->_hdr.tcm_info))); + } +} + +static int tc_clsact_add(struct ynl_sock *ys, int ifi) +{ + struct tc_newqdisc_req *req; + int ret; + + req = tc_newqdisc_req_alloc(); + if (!req) + return -1; + memset(req, 0, sizeof(*req)); + + req->_hdr.tcm_ifindex = ifi; + req->_hdr.tcm_parent = TC_H_CLSACT; + req->_hdr.tcm_handle = TC_HANDLE; + tc_newqdisc_req_set_nlflags(req, + NLM_F_REQUEST | NLM_F_EXCL | NLM_F_CREATE); + tc_newqdisc_req_set_kind(req, "clsact"); + + ret = tc_newqdisc(ys, req); + tc_newqdisc_req_free(req); + + return ret; +} + +static int tc_clsact_del(struct ynl_sock *ys, int ifi) +{ + struct tc_delqdisc_req *req; + int ret; + + req = tc_delqdisc_req_alloc(); + if (!req) + return -1; + memset(req, 0, sizeof(*req)); + + req->_hdr.tcm_ifindex = ifi; + req->_hdr.tcm_parent = TC_H_CLSACT; + req->_hdr.tcm_handle = TC_HANDLE; + tc_delqdisc_req_set_nlflags(req, NLM_F_REQUEST); + + ret = tc_delqdisc(ys, req); + tc_delqdisc_req_free(req); + + return ret; +} + +static int tc_filter_add(struct ynl_sock *ys, int ifi) +{ + struct tc_newtfilter_req *req; + struct tc_act_attrs *acts; + struct tc_vlan p = { + .action = TC_ACT_PIPE, + .v_action = TCA_VLAN_ACT_PUSH + }; + int ret; + + req = tc_newtfilter_req_alloc(); + if (!req) + return -1; + memset(req, 0, sizeof(*req)); + + acts = tc_act_attrs_alloc(3); + if (!acts) { + tc_newtfilter_req_free(req); + return -1; + } + memset(acts, 0, sizeof(*acts) * 3); + + req->_hdr.tcm_ifindex = ifi; + req->_hdr.tcm_parent = TC_H_MAKE(TC_H_CLSACT, TC_H_MIN_INGRESS); + req->_hdr.tcm_info = TC_H_MAKE(1 << 16, htons(ETH_P_8021Q)); + req->chain = 0; + + tc_newtfilter_req_set_nlflags(req, NLM_F_REQUEST | NLM_F_EXCL | NLM_F_CREATE); + tc_newtfilter_req_set_kind(req, "flower"); + tc_newtfilter_req_set_options_flower_key_vlan_id(req, 100); + tc_newtfilter_req_set_options_flower_key_vlan_prio(req, 5); + tc_newtfilter_req_set_options_flower_key_num_of_vlans(req, 3); + + __tc_newtfilter_req_set_options_flower_act(req, acts, 3); + + /* Skip action at index 0 because in TC, the action array + * index starts at 1, with each index defining the action's + * order. In contrast, in YNL indexed arrays start at index 0. + */ + tc_act_attrs_set_kind(&acts[1], "vlan"); + tc_act_attrs_set_options_vlan_parms(&acts[1], &p, sizeof(p)); + tc_act_attrs_set_options_vlan_push_vlan_id(&acts[1], 200); + tc_act_attrs_set_kind(&acts[2], "vlan"); + tc_act_attrs_set_options_vlan_parms(&acts[2], &p, sizeof(p)); + tc_act_attrs_set_options_vlan_push_vlan_id(&acts[2], 300); + + tc_newtfilter_req_set_options_flower_flags(req, 0); + tc_newtfilter_req_set_options_flower_key_eth_type(req, htons(0x8100)); + + ret = tc_newtfilter(ys, req); + tc_newtfilter_req_free(req); + + return ret; +} + +static int tc_filter_del(struct ynl_sock *ys, int ifi) +{ + struct tc_deltfilter_req *req; + int ret; + + req = tc_deltfilter_req_alloc(); + if (!req) + return -1; + memset(req, 0, sizeof(*req)); + + req->_hdr.tcm_ifindex = ifi; + req->_hdr.tcm_parent = TC_H_MAKE(TC_H_CLSACT, TC_H_MIN_INGRESS); + req->_hdr.tcm_info = TC_H_MAKE(1 << 16, htons(ETH_P_8021Q)); + tc_deltfilter_req_set_nlflags(req, NLM_F_REQUEST); + + ret = tc_deltfilter(ys, req); + tc_deltfilter_req_free(req); + + return ret; +} + +FIXTURE(tc) +{ + struct ynl_sock *ys; + int ifindex; +}; + +FIXTURE_SETUP(tc) +{ + struct ynl_error yerr; + int ret; + + ret = unshare(CLONE_NEWNET); + ASSERT_EQ(0, ret); + + self->ifindex = 1; /* loopback */ + + self->ys = ynl_sock_create(&ynl_tc_family, &yerr); + ASSERT_NE(NULL, self->ys) { + TH_LOG("failed to create tc socket: %s", yerr.msg); + } +} + +FIXTURE_TEARDOWN(tc) +{ + ynl_sock_destroy(self->ys); +} + +TEST_F(tc, qdisc) +{ + struct tc_getqdisc_req_dump *dreq; + struct tc_newqdisc_req *add_req; + struct tc_delqdisc_req *del_req; + struct tc_getqdisc_list *rsp; + bool found = false; + int ret; + + add_req = tc_newqdisc_req_alloc(); + ASSERT_NE(NULL, add_req); + memset(add_req, 0, sizeof(*add_req)); + + add_req->_hdr.tcm_ifindex = self->ifindex; + add_req->_hdr.tcm_parent = TC_H_ROOT; + tc_newqdisc_req_set_nlflags(add_req, + NLM_F_REQUEST | NLM_F_CREATE); + tc_newqdisc_req_set_kind(add_req, "fq_codel"); + + ret = tc_newqdisc(self->ys, add_req); + tc_newqdisc_req_free(add_req); + ASSERT_EQ(0, ret) { + TH_LOG("qdisc add failed: %s", self->ys->err.msg); + } + + dreq = tc_getqdisc_req_dump_alloc(); + ASSERT_NE(NULL, dreq); + rsp = tc_getqdisc_dump(self->ys, dreq); + tc_getqdisc_req_dump_free(dreq); + ASSERT_NE(NULL, rsp) { + TH_LOG("dump failed: %s", self->ys->err.msg); + } + ASSERT_FALSE(ynl_dump_empty(rsp)); + + ynl_dump_foreach(rsp, qdisc) { + found |= tc_qdisc_print(_metadata, qdisc); + } + tc_getqdisc_list_free(rsp); + EXPECT_TRUE(found); + + del_req = tc_delqdisc_req_alloc(); + ASSERT_NE(NULL, del_req); + memset(del_req, 0, sizeof(*del_req)); + + del_req->_hdr.tcm_ifindex = self->ifindex; + del_req->_hdr.tcm_parent = TC_H_ROOT; + tc_delqdisc_req_set_nlflags(del_req, NLM_F_REQUEST); + + ret = tc_delqdisc(self->ys, del_req); + tc_delqdisc_req_free(del_req); + EXPECT_EQ(0, ret) { + TH_LOG("qdisc del failed: %s", self->ys->err.msg); + } +} + +TEST_F(tc, flower) +{ + struct tc_gettfilter_req_dump *dreq; + struct tc_gettfilter_list *rsp; + bool found = false; + int ret; + + ret = tc_clsact_add(self->ys, self->ifindex); + if (ret) + SKIP(return, "clsact not supported: %s", self->ys->err.msg); + + ret = tc_filter_add(self->ys, self->ifindex); + ASSERT_EQ(0, ret) { + TH_LOG("filter add failed: %s", self->ys->err.msg); + } + + dreq = tc_gettfilter_req_dump_alloc(); + ASSERT_NE(NULL, dreq); + memset(dreq, 0, sizeof(*dreq)); + dreq->_hdr.tcm_ifindex = self->ifindex; + dreq->_hdr.tcm_parent = TC_H_MAKE(TC_H_CLSACT, TC_H_MIN_INGRESS); + dreq->_present.chain = 1; + dreq->chain = 0; + + rsp = tc_gettfilter_dump(self->ys, dreq); + tc_gettfilter_req_dump_free(dreq); + ASSERT_NE(NULL, rsp) { + TH_LOG("filter dump failed: %s", self->ys->err.msg); + } + + ynl_dump_foreach(rsp, flt) { + tc_filter_print(_metadata, flt); + if (flt->options._present.flower) { + EXPECT_EQ(100, flt->options.flower.key_vlan_id); + EXPECT_EQ(5, flt->options.flower.key_vlan_prio); + found = true; + } + } + tc_gettfilter_list_free(rsp); + EXPECT_TRUE(found); + + ret = tc_filter_del(self->ys, self->ifindex); + EXPECT_EQ(0, ret) { + TH_LOG("filter del failed: %s", self->ys->err.msg); + } + + ret = tc_clsact_del(self->ys, self->ifindex); + EXPECT_EQ(0, ret) { + TH_LOG("clsact del failed: %s", self->ys->err.msg); + } +} + +TEST_HARNESS_MAIN diff --git a/tools/net/ynl/tests/test_ynl_cli.sh b/tools/net/ynl/tests/test_ynl_cli.sh new file mode 100755 index 000000000..7c0722a08 --- /dev/null +++ b/tools/net/ynl/tests/test_ynl_cli.sh @@ -0,0 +1,327 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Test YNL CLI functionality + +# Load KTAP test helpers +KSELFTEST_KTAP_HELPERS="$(dirname "$(realpath "$0")")/../../../testing/selftests/kselftest/ktap_helpers.sh" +# shellcheck source=../../../testing/selftests/kselftest/ktap_helpers.sh +source "$KSELFTEST_KTAP_HELPERS" + +# Default ynl path for direct execution, can be overridden by make install +ynl="../pyynl/cli.py" + +readonly NSIM_ID="1338" +readonly NSIM_DEV_NAME="nsim${NSIM_ID}" +readonly VETH_A="veth_a" +readonly VETH_B="veth_b" + +testns="ynl-$(mktemp -u XXXXXX)" +TESTS_NO=0 + +# Test listing available families +cli_list_families() +{ + if $ynl --list-families &>/dev/null; then + ktap_test_pass "YNL CLI list families" + else + ktap_test_fail "YNL CLI list families" + fi +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test netdev family operations (dev-get, queue-get) +cli_netdev_ops() +{ + local dev_output + local ifindex + + ifindex=$(ip netns exec "$testns" cat /sys/class/net/"$NSIM_DEV_NAME"/ifindex 2>/dev/null) + + dev_output=$(ip netns exec "$testns" $ynl --family netdev \ + --do dev-get --json "{\"ifindex\": $ifindex}" 2>/dev/null) + + if ! echo "$dev_output" | grep -q "ifindex"; then + ktap_test_fail "YNL CLI netdev operations (netdev dev-get output missing ifindex)" + return + fi + + if ! ip netns exec "$testns" $ynl --family netdev \ + --dump queue-get --json "{\"ifindex\": $ifindex}" &>/dev/null; then + ktap_test_fail "YNL CLI netdev operations (failed to get netdev queue info)" + return + fi + + ktap_test_pass "YNL CLI netdev operations" +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test ethtool family operations (rings-get, linkinfo-get) +cli_ethtool_ops() +{ + local rings_output + local linkinfo_output + + rings_output=$(ip netns exec "$testns" $ynl --family ethtool \ + --do rings-get --json "{\"header\": {\"dev-name\": \"$NSIM_DEV_NAME\"}}" 2>/dev/null) + + if ! echo "$rings_output" | grep -q "header"; then + ktap_test_fail "YNL CLI ethtool operations (ethtool rings-get output missing header)" + return + fi + + linkinfo_output=$(ip netns exec "$testns" $ynl --family ethtool \ + --do linkinfo-get --json "{\"header\": {\"dev-name\": \"$VETH_A\"}}" 2>/dev/null) + + if ! echo "$linkinfo_output" | grep -q "header"; then + ktap_test_fail "YNL CLI ethtool operations (ethtool linkinfo-get output missing header)" + return + fi + + ktap_test_pass "YNL CLI ethtool operations" +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test rt-route family operations +cli_rt_route_ops() +{ + local ifindex + + if ! $ynl --list-families 2>/dev/null | grep -q "rt-route"; then + ktap_test_skip "YNL CLI rt-route operations (rt-route family not available)" + return + fi + + ifindex=$(ip netns exec "$testns" cat /sys/class/net/"$NSIM_DEV_NAME"/ifindex 2>/dev/null) + + # Add route: 192.0.2.0/24 dev $dev scope link + if ! ip netns exec "$testns" $ynl --family rt-route --do newroute --create \ + --json "{\"dst\": \"192.0.2.0\", \"oif\": $ifindex, \"rtm-dst-len\": 24, \"rtm-family\": 2, \"rtm-scope\": 253, \"rtm-type\": 1, \"rtm-protocol\": 3, \"rtm-table\": 254}" &>/dev/null; then + ktap_test_fail "YNL CLI rt-route operations (failed to add route)" + return + fi + + local route_output + route_output=$(ip netns exec "$testns" $ynl --family rt-route --dump getroute 2>/dev/null) + if echo "$route_output" | grep -q "192.0.2.0"; then + ktap_test_pass "YNL CLI rt-route operations" + else + ktap_test_fail "YNL CLI rt-route operations (failed to verify route)" + fi + + ip netns exec "$testns" $ynl --family rt-route --do delroute \ + --json "{\"dst\": \"192.0.2.0\", \"oif\": $ifindex, \"rtm-dst-len\": 24, \"rtm-family\": 2, \"rtm-scope\": 253, \"rtm-type\": 1, \"rtm-protocol\": 3, \"rtm-table\": 254}" &>/dev/null +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test rt-addr family operations +cli_rt_addr_ops() +{ + local ifindex + + if ! $ynl --list-families 2>/dev/null | grep -q "rt-addr"; then + ktap_test_skip "YNL CLI rt-addr operations (rt-addr family not available)" + return + fi + + ifindex=$(ip netns exec "$testns" cat /sys/class/net/"$NSIM_DEV_NAME"/ifindex 2>/dev/null) + + if ! ip netns exec "$testns" $ynl --family rt-addr --do newaddr \ + --json "{\"ifa-index\": $ifindex, \"local\": \"192.0.2.100\", \"ifa-prefixlen\": 24, \"ifa-family\": 2}" &>/dev/null; then + ktap_test_fail "YNL CLI rt-addr operations (failed to add address)" + return + fi + + local addr_output + addr_output=$(ip netns exec "$testns" $ynl --family rt-addr --dump getaddr 2>/dev/null) + if echo "$addr_output" | grep -q "192.0.2.100"; then + ktap_test_pass "YNL CLI rt-addr operations" + else + ktap_test_fail "YNL CLI rt-addr operations (failed to verify address)" + fi + + ip netns exec "$testns" $ynl --family rt-addr --do deladdr \ + --json "{\"ifa-index\": $ifindex, \"local\": \"192.0.2.100\", \"ifa-prefixlen\": 24, \"ifa-family\": 2}" &>/dev/null +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test rt-link family operations +cli_rt_link_ops() +{ + if ! $ynl --list-families 2>/dev/null | grep -q "rt-link"; then + ktap_test_skip "YNL CLI rt-link operations (rt-link family not available)" + return + fi + + if ! ip netns exec "$testns" $ynl --family rt-link --do newlink --create \ + --json "{\"ifname\": \"dummy0\", \"linkinfo\": {\"kind\": \"dummy\"}}" &>/dev/null; then + ktap_test_fail "YNL CLI rt-link operations (failed to add link)" + return + fi + + local link_output + link_output=$(ip netns exec "$testns" $ynl --family rt-link --dump getlink 2>/dev/null) + if echo "$link_output" | grep -q "$NSIM_DEV_NAME" && echo "$link_output" | grep -q "dummy0"; then + ktap_test_pass "YNL CLI rt-link operations" + else + ktap_test_fail "YNL CLI rt-link operations (failed to verify link)" + fi + + ip netns exec "$testns" $ynl --family rt-link --do dellink \ + --json "{\"ifname\": \"dummy0\"}" &>/dev/null +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test rt-neigh family operations +cli_rt_neigh_ops() +{ + local ifindex + + if ! $ynl --list-families 2>/dev/null | grep -q "rt-neigh"; then + ktap_test_skip "YNL CLI rt-neigh operations (rt-neigh family not available)" + return + fi + + ifindex=$(ip netns exec "$testns" cat /sys/class/net/"$NSIM_DEV_NAME"/ifindex 2>/dev/null) + + # Add neighbor: 192.0.2.1 dev nsim1338 lladdr 11:22:33:44:55:66 PERMANENT + if ! ip netns exec "$testns" $ynl --family rt-neigh --do newneigh --create \ + --json "{\"ndm-ifindex\": $ifindex, \"dst\": \"192.0.2.1\", \"lladdr\": \"11:22:33:44:55:66\", \"ndm-family\": 2, \"ndm-state\": 128}" &>/dev/null; then + ktap_test_fail "YNL CLI rt-neigh operations (failed to add neighbor)" + fi + + local neigh_output + neigh_output=$(ip netns exec "$testns" $ynl --family rt-neigh --dump getneigh 2>/dev/null) + if echo "$neigh_output" | grep -q "192.0.2.1"; then + ktap_test_pass "YNL CLI rt-neigh operations" + else + ktap_test_fail "YNL CLI rt-neigh operations (failed to verify neighbor)" + fi + + ip netns exec "$testns" $ynl --family rt-neigh --do delneigh \ + --json "{\"ndm-ifindex\": $ifindex, \"dst\": \"192.0.2.1\", \"lladdr\": \"11:22:33:44:55:66\", \"ndm-family\": 2}" &>/dev/null +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test rt-rule family operations +cli_rt_rule_ops() +{ + if ! $ynl --list-families 2>/dev/null | grep -q "rt-rule"; then + ktap_test_skip "YNL CLI rt-rule operations (rt-rule family not available)" + return + fi + + # Add rule: from 192.0.2.0/24 lookup 100 none + if ! ip netns exec "$testns" $ynl --family rt-rule --do newrule \ + --json "{\"family\": 2, \"src-len\": 24, \"src\": \"192.0.2.0\", \"table\": 100}" &>/dev/null; then + ktap_test_fail "YNL CLI rt-rule operations (failed to add rule)" + return + fi + + local rule_output + rule_output=$(ip netns exec "$testns" $ynl --family rt-rule --dump getrule 2>/dev/null) + if echo "$rule_output" | grep -q "192.0.2.0"; then + ktap_test_pass "YNL CLI rt-rule operations" + else + ktap_test_fail "YNL CLI rt-rule operations (failed to verify rule)" + fi + + ip netns exec "$testns" $ynl --family rt-rule --do delrule \ + --json "{\"family\": 2, \"src-len\": 24, \"src\": \"192.0.2.0\", \"table\": 100}" &>/dev/null +} +TESTS_NO=$((TESTS_NO + 1)) + +# Test nlctrl family operations +cli_nlctrl_ops() +{ + local family_output + + if ! family_output=$($ynl --family nlctrl \ + --do getfamily --json "{\"family-name\": \"netdev\"}" 2>/dev/null); then + ktap_test_fail "YNL CLI nlctrl getfamily (failed to get nlctrl family info)" + return + fi + + if ! echo "$family_output" | grep -q "family-name"; then + ktap_test_fail "YNL CLI nlctrl getfamily (nlctrl getfamily output missing family-name)" + return + fi + + if ! echo "$family_output" | grep -q "family-id"; then + ktap_test_fail "YNL CLI nlctrl getfamily (nlctrl getfamily output missing family-id)" + return + fi + + ktap_test_pass "YNL CLI nlctrl getfamily" +} +TESTS_NO=$((TESTS_NO + 1)) + +setup() +{ + modprobe netdevsim &> /dev/null + if ! [ -f /sys/bus/netdevsim/new_device ]; then + ktap_skip_all "netdevsim module not available" + exit "$KSFT_SKIP" + fi + + if ! ip netns add "$testns" 2>/dev/null; then + ktap_skip_all "failed to create test namespace" + exit "$KSFT_SKIP" + fi + + echo "$NSIM_ID 1" | ip netns exec "$testns" tee /sys/bus/netdevsim/new_device >/dev/null 2>&1 || { + ktap_skip_all "failed to create netdevsim device" + exit "$KSFT_SKIP" + } + + local dev + dev=$(ip netns exec "$testns" ls /sys/bus/netdevsim/devices/netdevsim$NSIM_ID/net 2>/dev/null | head -1) + if [[ -z "$dev" ]]; then + ktap_skip_all "failed to find netdevsim device" + exit "$KSFT_SKIP" + fi + + ip -netns "$testns" link set dev "$dev" name "$NSIM_DEV_NAME" 2>/dev/null || { + ktap_skip_all "failed to rename netdevsim device" + exit "$KSFT_SKIP" + } + + ip -netns "$testns" link set dev "$NSIM_DEV_NAME" up 2>/dev/null + + if ! ip -n "$testns" link add "$VETH_A" type veth peer name "$VETH_B" 2>/dev/null; then + ktap_skip_all "failed to create veth pair" + exit "$KSFT_SKIP" + fi + + ip -n "$testns" link set "$VETH_A" up 2>/dev/null + ip -n "$testns" link set "$VETH_B" up 2>/dev/null +} + +cleanup() +{ + ip netns exec "$testns" bash -c "echo $NSIM_ID > /sys/bus/netdevsim/del_device" 2>/dev/null || true + ip netns del "$testns" 2>/dev/null || true +} + +# Check if ynl command is available +if ! command -v $ynl &>/dev/null && [[ ! -x $ynl ]]; then + ktap_skip_all "ynl command not found: $ynl" + exit "$KSFT_SKIP" +fi + +trap cleanup EXIT + +ktap_print_header +setup +ktap_set_plan "${TESTS_NO}" + +cli_list_families +cli_netdev_ops +cli_ethtool_ops +cli_rt_route_ops +cli_rt_addr_ops +cli_rt_link_ops +cli_rt_neigh_ops +cli_rt_rule_ops +cli_nlctrl_ops + +ktap_finished diff --git a/tools/net/ynl/tests/test_ynl_ethtool.sh b/tools/net/ynl/tests/test_ynl_ethtool.sh new file mode 100755 index 000000000..b4480e9be --- /dev/null +++ b/tools/net/ynl/tests/test_ynl_ethtool.sh @@ -0,0 +1,222 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Test YNL ethtool functionality + +# Load KTAP test helpers +KSELFTEST_KTAP_HELPERS="$(dirname "$(realpath "$0")")/../../../testing/selftests/kselftest/ktap_helpers.sh" +# shellcheck source=../../../testing/selftests/kselftest/ktap_helpers.sh +source "$KSELFTEST_KTAP_HELPERS" + +# Default ynl-ethtool path for direct execution, can be overridden by make install +ynl_ethtool="./ethtool.py" + +readonly NSIM_ID="1337" +readonly NSIM_DEV_NAME="nsim${NSIM_ID}" +readonly VETH_A="veth_a" +readonly VETH_B="veth_b" + +testns="ynl-ethtool-$(mktemp -u XXXXXX)" +TESTS_NO=0 + +# Uses veth device as netdevsim doesn't support basic ethtool device info +ethtool_device_info() +{ + local info_output + + info_output=$(ip netns exec "$testns" $ynl_ethtool "$VETH_A" 2>/dev/null) + + if ! echo "$info_output" | grep -q "Settings for"; then + ktap_test_fail "YNL ethtool device info (device info output missing expected content)" + return + fi + + ktap_test_pass "YNL ethtool device info" +} +TESTS_NO=$((TESTS_NO + 1)) + +ethtool_statistics() +{ + local stats_output + + stats_output=$(ip netns exec "$testns" $ynl_ethtool --statistics "$NSIM_DEV_NAME" 2>/dev/null) + + if ! echo "$stats_output" | grep -q -E "(NIC statistics|packets|bytes)"; then + ktap_test_fail "YNL ethtool statistics (statistics output missing expected content)" + return + fi + + ktap_test_pass "YNL ethtool statistics" +} +TESTS_NO=$((TESTS_NO + 1)) + +ethtool_ring_params() +{ + local ring_output + + ring_output=$(ip netns exec "$testns" $ynl_ethtool --show-ring "$NSIM_DEV_NAME" 2>/dev/null) + + if ! echo "$ring_output" | grep -q -E "(Ring parameters|RX|TX)"; then + ktap_test_fail "YNL ethtool ring parameters (ring parameters output missing expected content)" + return + fi + + if ! ip netns exec "$testns" $ynl_ethtool --set-ring "$NSIM_DEV_NAME" rx 64 2>/dev/null; then + ktap_test_fail "YNL ethtool ring parameters (set-ring command failed unexpectedly)" + return + fi + + ktap_test_pass "YNL ethtool ring parameters (show/set)" +} +TESTS_NO=$((TESTS_NO + 1)) + +ethtool_coalesce_params() +{ + if ! ip netns exec "$testns" $ynl_ethtool --show-coalesce "$NSIM_DEV_NAME" &>/dev/null; then + ktap_test_fail "YNL ethtool coalesce parameters (failed to get coalesce parameters)" + return + fi + + if ! ip netns exec "$testns" $ynl_ethtool --set-coalesce "$NSIM_DEV_NAME" rx-usecs 50 2>/dev/null; then + ktap_test_fail "YNL ethtool coalesce parameters (set-coalesce command failed unexpectedly)" + return + fi + + ktap_test_pass "YNL ethtool coalesce parameters (show/set)" +} +TESTS_NO=$((TESTS_NO + 1)) + +ethtool_pause_params() +{ + if ! ip netns exec "$testns" $ynl_ethtool --show-pause "$NSIM_DEV_NAME" &>/dev/null; then + ktap_test_fail "YNL ethtool pause parameters (failed to get pause parameters)" + return + fi + + if ! ip netns exec "$testns" $ynl_ethtool --set-pause "$NSIM_DEV_NAME" tx 1 rx 1 2>/dev/null; then + ktap_test_fail "YNL ethtool pause parameters (set-pause command failed unexpectedly)" + return + fi + + ktap_test_pass "YNL ethtool pause parameters (show/set)" +} +TESTS_NO=$((TESTS_NO + 1)) + +ethtool_features_info() +{ + local features_output + + features_output=$(ip netns exec "$testns" $ynl_ethtool --show-features "$NSIM_DEV_NAME" 2>/dev/null) + + if ! echo "$features_output" | grep -q -E "(Features|offload)"; then + ktap_test_fail "YNL ethtool features info (features output missing expected content)" + return + fi + + ktap_test_pass "YNL ethtool features info (show/set)" +} +TESTS_NO=$((TESTS_NO + 1)) + +ethtool_channels_info() +{ + local channels_output + + channels_output=$(ip netns exec "$testns" $ynl_ethtool --show-channels "$NSIM_DEV_NAME" 2>/dev/null) + + if ! echo "$channels_output" | grep -q -E "(Channel|Combined|RX|TX)"; then + ktap_test_fail "YNL ethtool channels info (channels output missing expected content)" + return + fi + + if ! ip netns exec "$testns" $ynl_ethtool --set-channels "$NSIM_DEV_NAME" combined-count 1 2>/dev/null; then + ktap_test_fail "YNL ethtool channels info (set-channels command failed unexpectedly)" + return + fi + + ktap_test_pass "YNL ethtool channels info (show/set)" +} +TESTS_NO=$((TESTS_NO + 1)) + +ethtool_time_stamping() +{ + local ts_output + + ts_output=$(ip netns exec "$testns" $ynl_ethtool --show-time-stamping "$NSIM_DEV_NAME" 2>/dev/null) + + if ! echo "$ts_output" | grep -q -E "(Time stamping|timestamping|SOF_TIMESTAMPING)"; then + ktap_test_fail "YNL ethtool time stamping (time stamping output missing expected content)" + return + fi + + ktap_test_pass "YNL ethtool time stamping" +} +TESTS_NO=$((TESTS_NO + 1)) + +setup() +{ + modprobe netdevsim &> /dev/null + if ! [ -f /sys/bus/netdevsim/new_device ]; then + ktap_skip_all "netdevsim module not available" + exit "$KSFT_SKIP" + fi + + if ! ip netns add "$testns" 2>/dev/null; then + ktap_skip_all "failed to create test namespace" + exit "$KSFT_SKIP" + fi + + echo "$NSIM_ID 1" | ip netns exec "$testns" tee /sys/bus/netdevsim/new_device >/dev/null 2>&1 || { + ktap_skip_all "failed to create netdevsim device" + exit "$KSFT_SKIP" + } + + local dev + dev=$(ip netns exec "$testns" ls /sys/bus/netdevsim/devices/netdevsim$NSIM_ID/net 2>/dev/null | head -1) + if [[ -z "$dev" ]]; then + ktap_skip_all "failed to find netdevsim device" + exit "$KSFT_SKIP" + fi + + ip -netns "$testns" link set dev "$dev" name "$NSIM_DEV_NAME" 2>/dev/null || { + ktap_skip_all "failed to rename netdevsim device" + exit "$KSFT_SKIP" + } + + ip -netns "$testns" link set dev "$NSIM_DEV_NAME" up 2>/dev/null + + if ! ip -n "$testns" link add "$VETH_A" type veth peer name "$VETH_B" 2>/dev/null; then + ktap_skip_all "failed to create veth pair" + exit "$KSFT_SKIP" + fi + + ip -n "$testns" link set "$VETH_A" up 2>/dev/null + ip -n "$testns" link set "$VETH_B" up 2>/dev/null +} + +cleanup() +{ + ip netns exec "$testns" bash -c "echo $NSIM_ID > /sys/bus/netdevsim/del_device" 2>/dev/null || true + ip netns del "$testns" 2>/dev/null || true +} + +# Check if ynl-ethtool command is available +if ! command -v $ynl_ethtool &>/dev/null && [[ ! -x $ynl_ethtool ]]; then + ktap_skip_all "ynl-ethtool command not found: $ynl_ethtool" + exit "$KSFT_SKIP" +fi + +trap cleanup EXIT + +ktap_print_header +setup +ktap_set_plan "${TESTS_NO}" + +ethtool_device_info +ethtool_statistics +ethtool_ring_params +ethtool_coalesce_params +ethtool_pause_params +ethtool_features_info +ethtool_channels_info +ethtool_time_stamping + +ktap_finished diff --git a/tools/net/ynl/tests/wireguard.c b/tools/net/ynl/tests/wireguard.c new file mode 100644 index 000000000..df601e742 --- /dev/null +++ b/tools/net/ynl/tests/wireguard.c @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <arpa/inet.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> +#include <ynl.h> + +#include "wireguard-user.h" + +static void print_allowed_ip(const struct wireguard_wgallowedip *aip) +{ + char addr_out[INET6_ADDRSTRLEN]; + + if (!inet_ntop(aip->family, aip->ipaddr, addr_out, sizeof(addr_out))) { + addr_out[0] = '?'; + addr_out[1] = '\0'; + } + printf("\t\t\t%s/%u\n", addr_out, aip->cidr_mask); +} + +/* Only printing public key in this demo. For better key formatting, + * use the constant-time implementation as found in wireguard-tools. + */ +static void print_peer_header(const struct wireguard_wgpeer *peer) +{ + unsigned int len = peer->_len.public_key; + uint8_t *key = peer->public_key; + unsigned int i; + + if (len != 32) + return; + printf("\tPeer "); + for (i = 0; i < len; i++) + printf("%02x", key[i]); + printf(":\n"); +} + +static void print_peer(const struct wireguard_wgpeer *peer) +{ + unsigned int i; + + print_peer_header(peer); + printf("\t\tData: rx: %llu / tx: %llu bytes\n", + peer->rx_bytes, peer->tx_bytes); + printf("\t\tAllowed IPs:\n"); + for (i = 0; i < peer->_count.allowedips; i++) + print_allowed_ip(&peer->allowedips[i]); +} + +static void build_request(struct wireguard_get_device_req *req, char *arg) +{ + char *endptr; + int ifindex; + + ifindex = strtol(arg, &endptr, 0); + if (endptr != arg + strlen(arg) || errno != 0) + ifindex = 0; + if (ifindex > 0) + wireguard_get_device_req_set_ifindex(req, ifindex); + else + wireguard_get_device_req_set_ifname(req, arg); +} + +int main(int argc, char **argv) +{ + struct wireguard_get_device_list *devs; + struct wireguard_get_device_req *req; + struct ynl_error yerr; + struct ynl_sock *ys; + + if (argc < 2) { + fprintf(stderr, "usage: %s <ifindex|ifname>\n", argv[0]); + return 1; + } + + ys = ynl_sock_create(&ynl_wireguard_family, &yerr); + if (!ys) { + fprintf(stderr, "YNL: %s\n", yerr.msg); + return 2; + } + + req = wireguard_get_device_req_alloc(); + build_request(req, argv[1]); + + devs = wireguard_get_device_dump(ys, req); + if (!devs) { + fprintf(stderr, "YNL (%d): %s\n", ys->err.code, ys->err.msg); + wireguard_get_device_req_free(req); + ynl_sock_destroy(ys); + return 3; + } + + ynl_dump_foreach(devs, d) { + unsigned int i; + + printf("Interface %d: %s\n", d->ifindex, d->ifname); + for (i = 0; i < d->_count.peers; i++) + print_peer(&d->peers[i]); + } + + wireguard_get_device_list_free(devs); + wireguard_get_device_req_free(req); + ynl_sock_destroy(ys); + + return 0; +} diff --git a/tools/net/ynl/tests/ynl_nsim_lib.sh b/tools/net/ynl/tests/ynl_nsim_lib.sh new file mode 100644 index 000000000..98cdce44a --- /dev/null +++ b/tools/net/ynl/tests/ynl_nsim_lib.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Shared netdevsim setup/cleanup for YNL C test wrappers + +NSIM_ID="1337" +NSIM_DEV="" +KSFT_SKIP=4 + +nsim_cleanup() { + echo "$NSIM_ID" > /sys/bus/netdevsim/del_device 2>/dev/null || true +} + +nsim_setup() { + modprobe netdevsim 2>/dev/null + if ! [ -f /sys/bus/netdevsim/new_device ]; then + echo "netdevsim module not available, skipping" >&2 + exit "$KSFT_SKIP" + fi + + trap nsim_cleanup EXIT + + echo "$NSIM_ID 1" > /sys/bus/netdevsim/new_device + udevadm settle + + NSIM_DEV=$(ls /sys/bus/netdevsim/devices/netdevsim${NSIM_ID}/net 2>/dev/null | head -1) + if [ -z "$NSIM_DEV" ]; then + echo "failed to find netdevsim device" >&2 + exit 1 + fi + + ip link set dev "$NSIM_DEV" name nsim0 + ip link set dev nsim0 up + ip addr add 192.168.1.1/24 dev nsim0 + ip addr add 2001:db8::1/64 dev nsim0 nodad +} diff --git a/tools/net/ynl/ynl-regen.sh b/tools/net/ynl/ynl-regen.sh new file mode 100755 index 000000000..d9809276d --- /dev/null +++ b/tools/net/ynl/ynl-regen.sh @@ -0,0 +1,37 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause + +TOOL=$(dirname $(realpath $0))/pyynl/ynl_gen_c.py + +force= +search= + +while [ ! -z "$1" ]; do + case "$1" in + -f ) force=yes; shift ;; + -p ) search=$2; shift 2 ;; + * ) echo "Unrecognized option '$1'"; exit 1 ;; + esac +done + +KDIR=$(dirname $(dirname $(dirname $(dirname $(realpath $0))))) +pushd ${search:-$KDIR} >>/dev/null + +files=$(git grep --files-with-matches '^/\* YNL-GEN \(kernel\|uapi\|user\)') +for f in $files; do + # params: 0 1 2 3 + # $YAML YNL-GEN kernel $mode + params=( $(git grep --no-line-number -B1 -h '/\* YNL-GEN' $f | sed 's@/\*\(.*\)\*/@\1@') ) + args=$(sed -n 's@/\* YNL-ARG \(.*\) \*/@\1@p' $f) + + if [ $f -nt ${params[0]} -a -z "$force" ]; then + echo -e "\tSKIP $f" + continue + fi + + echo -e "\tGEN ${params[2]}\t$f" + $TOOL --cmp-out --mode ${params[2]} --${params[3]} \ + --spec $KDIR/${params[0]} $args -o $f +done + +popd >>/dev/null diff --git a/tools/net/ynl/ynltool/.gitignore b/tools/net/ynl/ynltool/.gitignore new file mode 100644 index 000000000..690d399c9 --- /dev/null +++ b/tools/net/ynl/ynltool/.gitignore @@ -0,0 +1,2 @@ +ynltool +*.d diff --git a/tools/net/ynl/ynltool/Makefile b/tools/net/ynl/ynltool/Makefile new file mode 100644 index 000000000..48b0f3205 --- /dev/null +++ b/tools/net/ynl/ynltool/Makefile @@ -0,0 +1,55 @@ +# SPDX-License-Identifier: GPL-2.0-only + +include ../Makefile.deps + +INSTALL ?= install +prefix ?= /usr + +CC := gcc +CFLAGS := -Wall -Wextra -Werror -O2 +ifeq ("$(DEBUG)","1") + CFLAGS += -g -fsanitize=address -fsanitize=leak -static-libasan +endif +CFLAGS += -I../lib -I../generated -I../../../include/uapi/ + +SRC_VERSION := \ + $(shell make --no-print-directory -sC ../../../.. kernelversion 2>/dev/null || \ + echo "unknown") + +CFLAGS += -DSRC_VERSION='"$(SRC_VERSION)"' + +SRCS := $(wildcard *.c) +OBJS := $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) + +YNLTOOL := $(OUTPUT)ynltool + +include $(wildcard *.d) + +all: $(YNLTOOL) + +Q = @ + +$(YNLTOOL): ../libynl.a $(OBJS) + $(Q)echo -e "\tLINK $@" + $(Q)$(CC) $(CFLAGS) -o $@ $(OBJS) ../libynl.a -lm + +%.o: %.c ../libynl.a + $(Q)echo -e "\tCC $@" + $(Q)$(COMPILE.c) -MMD -c -o $@ $< + +../libynl.a: + $(Q)$(MAKE) -C ../ + +clean: + rm -f *.o *.d *~ + +distclean: clean + rm -f $(YNLTOOL) + +bindir ?= /usr/bin + +install: $(YNLTOOL) + $(INSTALL) -m 0755 $(YNLTOOL) $(DESTDIR)$(bindir)/$(YNLTOOL) + +.PHONY: all clean distclean +.DEFAULT_GOAL=all diff --git a/tools/net/ynl/ynltool/json_writer.c b/tools/net/ynl/ynltool/json_writer.c new file mode 100644 index 000000000..c8685e592 --- /dev/null +++ b/tools/net/ynl/ynltool/json_writer.c @@ -0,0 +1,288 @@ +// SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) +/* + * Simple streaming JSON writer + * + * This takes care of the annoying bits of JSON syntax like the commas + * after elements + * + * Authors: Stephen Hemminger <stephen@networkplumber.org> + */ + +#include <stdio.h> +#include <stdbool.h> +#include <stdarg.h> +#include <assert.h> +#include <malloc.h> +#include <inttypes.h> +#include <stdint.h> + +#include "json_writer.h" + +struct json_writer { + FILE *out; + unsigned depth; + bool pretty; + char sep; +}; + +static void jsonw_indent(json_writer_t *self) +{ + unsigned i; + for (i = 0; i < self->depth; ++i) + fputs(" ", self->out); +} + +static void jsonw_eol(json_writer_t *self) +{ + if (!self->pretty) + return; + + putc('\n', self->out); + jsonw_indent(self); +} + +static void jsonw_eor(json_writer_t *self) +{ + if (self->sep != '\0') + putc(self->sep, self->out); + self->sep = ','; +} + +static void jsonw_puts(json_writer_t *self, const char *str) +{ + putc('"', self->out); + for (; *str; ++str) + switch (*str) { + case '\t': + fputs("\\t", self->out); + break; + case '\n': + fputs("\\n", self->out); + break; + case '\r': + fputs("\\r", self->out); + break; + case '\f': + fputs("\\f", self->out); + break; + case '\b': + fputs("\\b", self->out); + break; + case '\\': + fputs("\\\\", self->out); + break; + case '"': + fputs("\\\"", self->out); + break; + default: + putc(*str, self->out); + } + putc('"', self->out); +} + +json_writer_t *jsonw_new(FILE *f) +{ + json_writer_t *self = malloc(sizeof(*self)); + if (self) { + self->out = f; + self->depth = 0; + self->pretty = false; + self->sep = '\0'; + } + return self; +} + +void jsonw_destroy(json_writer_t **self_p) +{ + json_writer_t *self = *self_p; + + assert(self->depth == 0); + fputs("\n", self->out); + fflush(self->out); + free(self); + *self_p = NULL; +} + +void jsonw_pretty(json_writer_t *self, bool on) +{ + self->pretty = on; +} + +void jsonw_reset(json_writer_t *self) +{ + assert(self->depth == 0); + self->sep = '\0'; +} + +static void jsonw_begin(json_writer_t *self, int c) +{ + jsonw_eor(self); + putc(c, self->out); + ++self->depth; + self->sep = '\0'; +} + +static void jsonw_end(json_writer_t *self, int c) +{ + assert(self->depth > 0); + + --self->depth; + if (self->sep != '\0') + jsonw_eol(self); + putc(c, self->out); + self->sep = ','; +} + +void jsonw_name(json_writer_t *self, const char *name) +{ + jsonw_eor(self); + jsonw_eol(self); + self->sep = '\0'; + jsonw_puts(self, name); + putc(':', self->out); + if (self->pretty) + putc(' ', self->out); +} + +void jsonw_vprintf_enquote(json_writer_t *self, const char *fmt, va_list ap) +{ + jsonw_eor(self); + putc('"', self->out); + vfprintf(self->out, fmt, ap); + putc('"', self->out); +} + +void jsonw_printf(json_writer_t *self, const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + jsonw_eor(self); + vfprintf(self->out, fmt, ap); + va_end(ap); +} + +void jsonw_start_object(json_writer_t *self) +{ + jsonw_begin(self, '{'); +} + +void jsonw_end_object(json_writer_t *self) +{ + jsonw_end(self, '}'); +} + +void jsonw_start_array(json_writer_t *self) +{ + jsonw_begin(self, '['); +} + +void jsonw_end_array(json_writer_t *self) +{ + jsonw_end(self, ']'); +} + +void jsonw_string(json_writer_t *self, const char *value) +{ + jsonw_eor(self); + jsonw_puts(self, value); +} + +void jsonw_bool(json_writer_t *self, bool val) +{ + jsonw_printf(self, "%s", val ? "true" : "false"); +} + +void jsonw_null(json_writer_t *self) +{ + jsonw_printf(self, "null"); +} + +void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num) +{ + jsonw_printf(self, fmt, num); +} + +void jsonw_float(json_writer_t *self, double num) +{ + jsonw_printf(self, "%g", num); +} + +void jsonw_hu(json_writer_t *self, unsigned short num) +{ + jsonw_printf(self, "%hu", num); +} + +void jsonw_uint(json_writer_t *self, uint64_t num) +{ + jsonw_printf(self, "%"PRIu64, num); +} + +void jsonw_lluint(json_writer_t *self, unsigned long long int num) +{ + jsonw_printf(self, "%llu", num); +} + +void jsonw_int(json_writer_t *self, int64_t num) +{ + jsonw_printf(self, "%"PRId64, num); +} + +void jsonw_string_field(json_writer_t *self, const char *prop, const char *val) +{ + jsonw_name(self, prop); + jsonw_string(self, val); +} + +void jsonw_bool_field(json_writer_t *self, const char *prop, bool val) +{ + jsonw_name(self, prop); + jsonw_bool(self, val); +} + +void jsonw_float_field(json_writer_t *self, const char *prop, double val) +{ + jsonw_name(self, prop); + jsonw_float(self, val); +} + +void jsonw_float_field_fmt(json_writer_t *self, + const char *prop, + const char *fmt, + double val) +{ + jsonw_name(self, prop); + jsonw_float_fmt(self, fmt, val); +} + +void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num) +{ + jsonw_name(self, prop); + jsonw_uint(self, num); +} + +void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num) +{ + jsonw_name(self, prop); + jsonw_hu(self, num); +} + +void jsonw_lluint_field(json_writer_t *self, + const char *prop, + unsigned long long int num) +{ + jsonw_name(self, prop); + jsonw_lluint(self, num); +} + +void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num) +{ + jsonw_name(self, prop); + jsonw_int(self, num); +} + +void jsonw_null_field(json_writer_t *self, const char *prop) +{ + jsonw_name(self, prop); + jsonw_null(self); +} diff --git a/tools/net/ynl/ynltool/json_writer.h b/tools/net/ynl/ynltool/json_writer.h new file mode 100644 index 000000000..0f1e63c88 --- /dev/null +++ b/tools/net/ynl/ynltool/json_writer.h @@ -0,0 +1,75 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */ +/* + * Simple streaming JSON writer + * + * This takes care of the annoying bits of JSON syntax like the commas + * after elements + * + * Authors: Stephen Hemminger <stephen@networkplumber.org> + */ + +#ifndef _JSON_WRITER_H_ +#define _JSON_WRITER_H_ + +#include <stdbool.h> +#include <stdint.h> +#include <stdarg.h> +#include <stdio.h> + +/* Opaque class structure */ +typedef struct json_writer json_writer_t; + +/* Create a new JSON stream */ +json_writer_t *jsonw_new(FILE *f); +/* End output to JSON stream */ +void jsonw_destroy(json_writer_t **self_p); + +/* Cause output to have pretty whitespace */ +void jsonw_pretty(json_writer_t *self, bool on); + +/* Reset separator to create new JSON */ +void jsonw_reset(json_writer_t *self); + +/* Add property name */ +void jsonw_name(json_writer_t *self, const char *name); + +/* Add value */ +void __attribute__((format(printf, 2, 0))) jsonw_vprintf_enquote(json_writer_t *self, + const char *fmt, + va_list ap); +void __attribute__((format(printf, 2, 3))) jsonw_printf(json_writer_t *self, + const char *fmt, ...); +void jsonw_string(json_writer_t *self, const char *value); +void jsonw_bool(json_writer_t *self, bool value); +void jsonw_float(json_writer_t *self, double number); +void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num); +void jsonw_uint(json_writer_t *self, uint64_t number); +void jsonw_hu(json_writer_t *self, unsigned short number); +void jsonw_int(json_writer_t *self, int64_t number); +void jsonw_null(json_writer_t *self); +void jsonw_lluint(json_writer_t *self, unsigned long long int num); + +/* Useful Combinations of name and value */ +void jsonw_string_field(json_writer_t *self, const char *prop, const char *val); +void jsonw_bool_field(json_writer_t *self, const char *prop, bool value); +void jsonw_float_field(json_writer_t *self, const char *prop, double num); +void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num); +void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num); +void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num); +void jsonw_null_field(json_writer_t *self, const char *prop); +void jsonw_lluint_field(json_writer_t *self, const char *prop, + unsigned long long int num); +void jsonw_float_field_fmt(json_writer_t *self, const char *prop, + const char *fmt, double val); + +/* Collections */ +void jsonw_start_object(json_writer_t *self); +void jsonw_end_object(json_writer_t *self); + +void jsonw_start_array(json_writer_t *self); +void jsonw_end_array(json_writer_t *self); + +/* Override default exception handling */ +typedef void (jsonw_err_handler_fn)(const char *); + +#endif /* _JSON_WRITER_H_ */ diff --git a/tools/net/ynl/ynltool/main.c b/tools/net/ynl/ynltool/main.c new file mode 100644 index 000000000..5d0f428ee --- /dev/null +++ b/tools/net/ynl/ynltool/main.c @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +/* Copyright (C) 2017-2018 Netronome Systems, Inc. */ +/* Copyright Meta Platforms, Inc. and affiliates */ + +#include <ctype.h> +#include <errno.h> +#include <getopt.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <stdarg.h> + +#include "main.h" + +const char *bin_name; +static int last_argc; +static char **last_argv; +static int (*last_do_help)(int argc, char **argv); +json_writer_t *json_wtr; +bool pretty_output; +bool json_output; + +static void __attribute__((noreturn)) clean_and_exit(int i) +{ + if (json_output) + jsonw_destroy(&json_wtr); + + exit(i); +} + +void usage(void) +{ + last_do_help(last_argc - 1, last_argv + 1); + + clean_and_exit(-1); +} + +static int do_help(int argc __attribute__((unused)), + char **argv __attribute__((unused))) +{ + if (json_output) { + jsonw_null(json_wtr); + return 0; + } + + fprintf(stderr, + "Usage: %s [OPTIONS] OBJECT { COMMAND | help }\n" + " %s version\n" + "\n" + " OBJECT := { page-pool | qstats }\n" + " " HELP_SPEC_OPTIONS "\n" + "", + bin_name, bin_name); + + return 0; +} + +static int do_version(int argc __attribute__((unused)), + char **argv __attribute__((unused))) +{ + if (json_output) { + jsonw_start_object(json_wtr); + jsonw_name(json_wtr, "version"); + jsonw_printf(json_wtr, SRC_VERSION); + jsonw_end_object(json_wtr); + } else { + printf("%s " SRC_VERSION "\n", bin_name); + } + return 0; +} + +static const struct cmd commands[] = { + { "help", do_help }, + { "page-pool", do_page_pool }, + { "qstats", do_qstats }, + { "version", do_version }, + { 0 } +}; + +int cmd_select(const struct cmd *cmds, int argc, char **argv, + int (*help)(int argc, char **argv)) +{ + unsigned int i; + + last_argc = argc; + last_argv = argv; + last_do_help = help; + + if (argc < 1 && cmds[0].func) + return cmds[0].func(argc, argv); + + for (i = 0; cmds[i].cmd; i++) { + if (is_prefix(*argv, cmds[i].cmd)) { + if (!cmds[i].func) { + p_err("command '%s' is not available", cmds[i].cmd); + return -1; + } + return cmds[i].func(argc - 1, argv + 1); + } + } + + help(argc - 1, argv + 1); + + return -1; +} + +bool is_prefix(const char *pfx, const char *str) +{ + if (!pfx) + return false; + if (strlen(str) < strlen(pfx)) + return false; + + return !memcmp(str, pfx, strlen(pfx)); +} + +/* Last argument MUST be NULL pointer */ +int detect_common_prefix(const char *arg, ...) +{ + unsigned int count = 0; + const char *ref; + char msg[256]; + va_list ap; + + snprintf(msg, sizeof(msg), "ambiguous prefix: '%s' could be '", arg); + va_start(ap, arg); + while ((ref = va_arg(ap, const char *))) { + if (!is_prefix(arg, ref)) + continue; + count++; + if (count > 1) + strncat(msg, "' or '", sizeof(msg) - strlen(msg) - 1); + strncat(msg, ref, sizeof(msg) - strlen(msg) - 1); + } + va_end(ap); + strncat(msg, "'", sizeof(msg) - strlen(msg) - 1); + + if (count >= 2) { + p_err("%s", msg); + return -1; + } + + return 0; +} + +void p_err(const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + if (json_output) { + jsonw_start_object(json_wtr); + jsonw_name(json_wtr, "error"); + jsonw_vprintf_enquote(json_wtr, fmt, ap); + jsonw_end_object(json_wtr); + } else { + fprintf(stderr, "Error: "); + vfprintf(stderr, fmt, ap); + fprintf(stderr, "\n"); + } + va_end(ap); +} + +void p_info(const char *fmt, ...) +{ + va_list ap; + + if (json_output) + return; + + va_start(ap, fmt); + vfprintf(stderr, fmt, ap); + fprintf(stderr, "\n"); + va_end(ap); +} + +int main(int argc, char **argv) +{ + static const struct option options[] = { + { "json", no_argument, NULL, 'j' }, + { "help", no_argument, NULL, 'h' }, + { "pretty", no_argument, NULL, 'p' }, + { "version", no_argument, NULL, 'V' }, + { 0 } + }; + bool version_requested = false; + int opt, ret; + + setlinebuf(stdout); + + last_do_help = do_help; + pretty_output = false; + json_output = false; + bin_name = "ynltool"; + + opterr = 0; + while ((opt = getopt_long(argc, argv, "Vhjp", + options, NULL)) >= 0) { + switch (opt) { + case 'V': + version_requested = true; + break; + case 'h': + return do_help(argc, argv); + case 'p': + pretty_output = true; + /* fall through */ + case 'j': + if (!json_output) { + json_wtr = jsonw_new(stdout); + if (!json_wtr) { + p_err("failed to create JSON writer"); + return -1; + } + json_output = true; + } + jsonw_pretty(json_wtr, pretty_output); + break; + default: + p_err("unrecognized option '%s'", argv[optind - 1]); + if (json_output) + clean_and_exit(-1); + else + usage(); + } + } + + argc -= optind; + argv += optind; + if (argc < 0) + usage(); + + if (version_requested) + ret = do_version(argc, argv); + else + ret = cmd_select(commands, argc, argv, do_help); + + if (json_output) + jsonw_destroy(&json_wtr); + + return ret; +} diff --git a/tools/net/ynl/ynltool/main.h b/tools/net/ynl/ynltool/main.h new file mode 100644 index 000000000..c7039f9ac --- /dev/null +++ b/tools/net/ynl/ynltool/main.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */ +/* Copyright (C) 2017-2018 Netronome Systems, Inc. */ +/* Copyright Meta Platforms, Inc. and affiliates */ + +#ifndef __YNLTOOL_H +#define __YNLTOOL_H + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif +#include <stdbool.h> +#include <stdio.h> +#include <stdlib.h> +#include <errno.h> +#include <string.h> + +#include "json_writer.h" + +#define NEXT_ARG() ({ argc--; argv++; if (argc < 0) usage(); }) +#define NEXT_ARGP() ({ (*argc)--; (*argv)++; if (*argc < 0) usage(); }) +#define BAD_ARG() ({ p_err("what is '%s'?", *argv); -1; }) +#define GET_ARG() ({ argc--; *argv++; }) +#define REQ_ARGS(cnt) \ + ({ \ + int _cnt = (cnt); \ + bool _res; \ + \ + if (argc < _cnt) { \ + p_err("'%s' needs at least %d arguments, %d found", \ + argv[-1], _cnt, argc); \ + _res = false; \ + } else { \ + _res = true; \ + } \ + _res; \ + }) + +#define HELP_SPEC_OPTIONS \ + "OPTIONS := { {-j|--json} [{-p|--pretty}] }" + +extern const char *bin_name; + +extern json_writer_t *json_wtr; +extern bool json_output; +extern bool pretty_output; + +void __attribute__((format(printf, 1, 2))) p_err(const char *fmt, ...); +void __attribute__((format(printf, 1, 2))) p_info(const char *fmt, ...); + +bool is_prefix(const char *pfx, const char *str); +int detect_common_prefix(const char *arg, ...); +void usage(void) __attribute__((noreturn)); + +struct cmd { + const char *cmd; + int (*func)(int argc, char **argv); +}; + +int cmd_select(const struct cmd *cmds, int argc, char **argv, + int (*help)(int argc, char **argv)); + +/* subcommands */ +int do_page_pool(int argc, char **argv); +int do_qstats(int argc, char **argv); + +#endif /* __YNLTOOL_H */ diff --git a/tools/net/ynl/ynltool/page-pool.c b/tools/net/ynl/ynltool/page-pool.c new file mode 100644 index 000000000..9487eda6b --- /dev/null +++ b/tools/net/ynl/ynltool/page-pool.c @@ -0,0 +1,463 @@ +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <errno.h> +#include <net/if.h> + +#include <ynl.h> +#include "netdev-user.h" + +#include "main.h" + +struct pp_stat { + unsigned int ifc; + + struct { + unsigned int cnt; + size_t refs, bytes; + } live[2]; + + size_t alloc_slow, alloc_fast, recycle_ring, recycle_cache; +}; + +struct pp_stats_array { + unsigned int i, max; + struct pp_stat *s; +}; + +static struct pp_stat *find_ifc(struct pp_stats_array *a, unsigned int ifindex) +{ + unsigned int i; + + for (i = 0; i < a->i; i++) { + if (a->s[i].ifc == ifindex) + return &a->s[i]; + } + + a->i++; + if (a->i == a->max) { + a->max *= 2; + a->s = reallocarray(a->s, a->max, sizeof(*a->s)); + } + a->s[i].ifc = ifindex; + return &a->s[i]; +} + +static void count_pool(struct pp_stat *s, unsigned int l, + struct netdev_page_pool_get_rsp *pp) +{ + s->live[l].cnt++; + if (pp->_present.inflight) + s->live[l].refs += pp->inflight; + if (pp->_present.inflight_mem) + s->live[l].bytes += pp->inflight_mem; +} + +/* We don't know how many pages are sitting in cache and ring + * so we will under-count the recycling rate a bit. + */ +static void print_json_recycling_stats(struct pp_stat *s) +{ + double recycle; + + if (s->alloc_fast + s->alloc_slow) { + recycle = (double)(s->recycle_ring + s->recycle_cache) / + (s->alloc_fast + s->alloc_slow) * 100; + jsonw_float_field(json_wtr, "recycling_pct", recycle); + } + + jsonw_name(json_wtr, "alloc"); + jsonw_start_object(json_wtr); + jsonw_uint_field(json_wtr, "slow", s->alloc_slow); + jsonw_uint_field(json_wtr, "fast", s->alloc_fast); + jsonw_end_object(json_wtr); + + jsonw_name(json_wtr, "recycle"); + jsonw_start_object(json_wtr); + jsonw_uint_field(json_wtr, "ring", s->recycle_ring); + jsonw_uint_field(json_wtr, "cache", s->recycle_cache); + jsonw_end_object(json_wtr); +} + +static void print_plain_recycling_stats(struct pp_stat *s) +{ + double recycle; + + if (s->alloc_fast + s->alloc_slow) { + recycle = (double)(s->recycle_ring + s->recycle_cache) / + (s->alloc_fast + s->alloc_slow) * 100; + printf("recycling: %.1lf%% (alloc: %zu:%zu recycle: %zu:%zu)", + recycle, s->alloc_slow, s->alloc_fast, + s->recycle_ring, s->recycle_cache); + } +} + +static void print_json_stats(struct pp_stats_array *a) +{ + jsonw_start_array(json_wtr); + + for (unsigned int i = 0; i < a->i; i++) { + char ifname[IF_NAMESIZE]; + struct pp_stat *s = &a->s[i]; + const char *name; + + jsonw_start_object(json_wtr); + + if (!s->ifc) { + jsonw_string_field(json_wtr, "ifname", "<orphan>"); + jsonw_uint_field(json_wtr, "ifindex", 0); + } else { + name = if_indextoname(s->ifc, ifname); + if (name) + jsonw_string_field(json_wtr, "ifname", name); + jsonw_uint_field(json_wtr, "ifindex", s->ifc); + } + + jsonw_uint_field(json_wtr, "page_pools", s->live[1].cnt); + jsonw_uint_field(json_wtr, "zombies", s->live[0].cnt); + + jsonw_name(json_wtr, "live"); + jsonw_start_object(json_wtr); + jsonw_uint_field(json_wtr, "refs", s->live[1].refs); + jsonw_uint_field(json_wtr, "bytes", s->live[1].bytes); + jsonw_end_object(json_wtr); + + jsonw_name(json_wtr, "zombie"); + jsonw_start_object(json_wtr); + jsonw_uint_field(json_wtr, "refs", s->live[0].refs); + jsonw_uint_field(json_wtr, "bytes", s->live[0].bytes); + jsonw_end_object(json_wtr); + + if (s->alloc_fast || s->alloc_slow) + print_json_recycling_stats(s); + + jsonw_end_object(json_wtr); + } + + jsonw_end_array(json_wtr); +} + +static void print_plain_stats(struct pp_stats_array *a) +{ + for (unsigned int i = 0; i < a->i; i++) { + char ifname[IF_NAMESIZE]; + struct pp_stat *s = &a->s[i]; + const char *name; + + if (!s->ifc) { + printf("<orphan>\t"); + } else { + name = if_indextoname(s->ifc, ifname); + if (name) + printf("%8s", name); + printf("[%u]\t", s->ifc); + } + + printf("page pools: %u (zombies: %u)\n", + s->live[1].cnt, s->live[0].cnt); + printf("\t\trefs: %zu bytes: %zu (refs: %zu bytes: %zu)\n", + s->live[1].refs, s->live[1].bytes, + s->live[0].refs, s->live[0].bytes); + + if (s->alloc_fast || s->alloc_slow) { + printf("\t\t"); + print_plain_recycling_stats(s); + printf("\n"); + } + } +} + +static bool +find_pool_stat_in_list(struct netdev_page_pool_stats_get_list *pp_stats, + __u64 pool_id, struct pp_stat *pstat) +{ + ynl_dump_foreach(pp_stats, pp) { + if (!pp->_present.info || !pp->info._present.id) + continue; + if (pp->info.id != pool_id) + continue; + + memset(pstat, 0, sizeof(*pstat)); + if (pp->_present.alloc_fast) + pstat->alloc_fast = pp->alloc_fast; + if (pp->_present.alloc_refill) + pstat->alloc_fast += pp->alloc_refill; + if (pp->_present.alloc_slow) + pstat->alloc_slow = pp->alloc_slow; + if (pp->_present.recycle_ring) + pstat->recycle_ring = pp->recycle_ring; + if (pp->_present.recycle_cached) + pstat->recycle_cache = pp->recycle_cached; + return true; + } + return false; +} + +static void +print_json_pool_list(struct netdev_page_pool_get_list *pools, + struct netdev_page_pool_stats_get_list *pp_stats, + bool zombies_only) +{ + jsonw_start_array(json_wtr); + + ynl_dump_foreach(pools, pp) { + char ifname[IF_NAMESIZE]; + struct pp_stat pstat; + const char *name; + + if (zombies_only && !pp->_present.detach_time) + continue; + + jsonw_start_object(json_wtr); + + jsonw_uint_field(json_wtr, "id", pp->id); + + if (pp->_present.ifindex) { + name = if_indextoname(pp->ifindex, ifname); + if (name) + jsonw_string_field(json_wtr, "ifname", name); + jsonw_uint_field(json_wtr, "ifindex", pp->ifindex); + } + + if (pp->_present.napi_id) + jsonw_uint_field(json_wtr, "napi_id", pp->napi_id); + + if (pp->_present.inflight) + jsonw_uint_field(json_wtr, "refs", pp->inflight); + + if (pp->_present.inflight_mem) + jsonw_uint_field(json_wtr, "bytes", pp->inflight_mem); + + if (pp->_present.detach_time) + jsonw_uint_field(json_wtr, "detach_time", pp->detach_time); + + if (pp->_present.dmabuf) + jsonw_uint_field(json_wtr, "dmabuf", pp->dmabuf); + + if (find_pool_stat_in_list(pp_stats, pp->id, &pstat) && + (pstat.alloc_fast || pstat.alloc_slow)) + print_json_recycling_stats(&pstat); + + jsonw_end_object(json_wtr); + } + + jsonw_end_array(json_wtr); +} + +static void +print_plain_pool_list(struct netdev_page_pool_get_list *pools, + struct netdev_page_pool_stats_get_list *pp_stats, + bool zombies_only) +{ + ynl_dump_foreach(pools, pp) { + char ifname[IF_NAMESIZE]; + struct pp_stat pstat; + const char *name; + + if (zombies_only && !pp->_present.detach_time) + continue; + + printf("pool id: %llu", pp->id); + + if (pp->_present.ifindex) { + name = if_indextoname(pp->ifindex, ifname); + if (name) + printf(" dev: %s", name); + printf("[%u]", pp->ifindex); + } + + if (pp->_present.napi_id) + printf(" napi: %llu", pp->napi_id); + + printf("\n"); + + if (pp->_present.inflight || pp->_present.inflight_mem) { + printf(" inflight:"); + if (pp->_present.inflight) + printf(" %llu pages", pp->inflight); + if (pp->_present.inflight_mem) + printf(" %llu bytes", pp->inflight_mem); + printf("\n"); + } + + if (pp->_present.detach_time) + printf(" detached: %llu\n", pp->detach_time); + + if (pp->_present.dmabuf) + printf(" dmabuf: %u\n", pp->dmabuf); + + if (find_pool_stat_in_list(pp_stats, pp->id, &pstat) && + (pstat.alloc_fast || pstat.alloc_slow)) { + printf(" "); + print_plain_recycling_stats(&pstat); + printf("\n"); + } + } +} + +static void aggregate_device_stats(struct pp_stats_array *a, + struct netdev_page_pool_get_list *pools, + struct netdev_page_pool_stats_get_list *pp_stats) +{ + ynl_dump_foreach(pools, pp) { + struct pp_stat *s = find_ifc(a, pp->ifindex); + + count_pool(s, 1, pp); + if (pp->_present.detach_time) + count_pool(s, 0, pp); + } + + ynl_dump_foreach(pp_stats, pp) { + struct pp_stat *s = find_ifc(a, pp->info.ifindex); + + if (pp->_present.alloc_fast) + s->alloc_fast += pp->alloc_fast; + if (pp->_present.alloc_refill) + s->alloc_fast += pp->alloc_refill; + if (pp->_present.alloc_slow) + s->alloc_slow += pp->alloc_slow; + if (pp->_present.recycle_ring) + s->recycle_ring += pp->recycle_ring; + if (pp->_present.recycle_cached) + s->recycle_cache += pp->recycle_cached; + } +} + +static int do_stats(int argc, char **argv) +{ + struct netdev_page_pool_stats_get_req_dump pp_stat_req = {}; + struct netdev_page_pool_stats_get_list *pp_stats; + struct netdev_page_pool_get_req_dump pp_req = {}; + struct netdev_page_pool_get_list *pools; + enum { + GROUP_BY_DEVICE, + GROUP_BY_POOL, + } group_by = GROUP_BY_DEVICE; + bool zombies_only = false; + struct pp_stats_array a = {}; + struct ynl_error yerr; + struct ynl_sock *ys; + int ret = 0; + + /* Parse options */ + while (argc > 0) { + if (is_prefix(*argv, "group-by")) { + NEXT_ARG(); + + if (!REQ_ARGS(1)) + return -1; + + if (is_prefix(*argv, "device")) { + group_by = GROUP_BY_DEVICE; + } else if (is_prefix(*argv, "pp") || + is_prefix(*argv, "page-pool") || + is_prefix(*argv, "none")) { + group_by = GROUP_BY_POOL; + } else { + p_err("invalid group-by value '%s'", *argv); + return -1; + } + NEXT_ARG(); + } else if (is_prefix(*argv, "zombies")) { + zombies_only = true; + group_by = GROUP_BY_POOL; + NEXT_ARG(); + } else { + p_err("unknown option '%s'", *argv); + return -1; + } + } + + ys = ynl_sock_create(&ynl_netdev_family, &yerr); + if (!ys) { + p_err("YNL: %s", yerr.msg); + return -1; + } + + pools = netdev_page_pool_get_dump(ys, &pp_req); + if (!pools) { + p_err("failed to get page pools: %s", ys->err.msg); + ret = -1; + goto exit_close; + } + + pp_stats = netdev_page_pool_stats_get_dump(ys, &pp_stat_req); + if (!pp_stats) { + p_err("failed to get page pool stats: %s", ys->err.msg); + ret = -1; + goto exit_free_pp_list; + } + + /* If grouping by pool, print individual pools */ + if (group_by == GROUP_BY_POOL) { + if (json_output) + print_json_pool_list(pools, pp_stats, zombies_only); + else + print_plain_pool_list(pools, pp_stats, zombies_only); + } else { + /* Aggregated stats mode (group-by device) */ + a.max = 64; + a.s = calloc(a.max, sizeof(*a.s)); + if (!a.s) { + p_err("failed to allocate stats array"); + ret = -1; + goto exit_free_stats_list; + } + + aggregate_device_stats(&a, pools, pp_stats); + + if (json_output) + print_json_stats(&a); + else + print_plain_stats(&a); + + free(a.s); + } + +exit_free_stats_list: + netdev_page_pool_stats_get_list_free(pp_stats); +exit_free_pp_list: + netdev_page_pool_get_list_free(pools); +exit_close: + ynl_sock_destroy(ys); + return ret; +} + +static int do_help(int argc __attribute__((unused)), + char **argv __attribute__((unused))) +{ + if (json_output) { + jsonw_null(json_wtr); + return 0; + } + + fprintf(stderr, + "Usage: %s page-pool { COMMAND | help }\n" + " %s page-pool stats [ OPTIONS ]\n" + "\n" + " OPTIONS := { group-by { device | page-pool | none } | zombies }\n" + "\n" + " stats - Display page pool statistics\n" + " stats group-by device - Group statistics by network device (default)\n" + " stats group-by page-pool | pp | none\n" + " - Show individual page pool details (no grouping)\n" + " stats zombies - Show only zombie page pools (detached but with\n" + " pages in flight). Implies group-by page-pool.\n" + "", + bin_name, bin_name); + + return 0; +} + +static const struct cmd page_pool_cmds[] = { + { "help", do_help }, + { "stats", do_stats }, + { 0 } +}; + +int do_page_pool(int argc, char **argv) +{ + return cmd_select(page_pool_cmds, argc, argv, do_help); +} diff --git a/tools/net/ynl/ynltool/qstats.c b/tools/net/ynl/ynltool/qstats.c new file mode 100644 index 000000000..a6c28ba4f --- /dev/null +++ b/tools/net/ynl/ynltool/qstats.c @@ -0,0 +1,674 @@ +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <errno.h> +#include <net/if.h> +#include <math.h> + +#include <ynl.h> +#include "netdev-user.h" + +#include "main.h" + +static enum netdev_qstats_scope scope; /* default - device */ + +struct queue_balance { + unsigned int ifindex; + enum netdev_queue_type type; + unsigned int queue_count; + __u64 *rx_packets; + __u64 *rx_bytes; + __u64 *tx_packets; + __u64 *tx_bytes; +}; + +static void print_json_qstats(struct netdev_qstats_get_list *qstats) +{ + jsonw_start_array(json_wtr); + + ynl_dump_foreach(qstats, qs) { + char ifname[IF_NAMESIZE]; + const char *name; + + jsonw_start_object(json_wtr); + + name = if_indextoname(qs->ifindex, ifname); + if (name) + jsonw_string_field(json_wtr, "ifname", name); + jsonw_uint_field(json_wtr, "ifindex", qs->ifindex); + + if (qs->_present.queue_type) + jsonw_string_field(json_wtr, "queue-type", + netdev_queue_type_str(qs->queue_type)); + if (qs->_present.queue_id) + jsonw_uint_field(json_wtr, "queue-id", qs->queue_id); + + if (qs->_present.rx_packets || qs->_present.rx_bytes || + qs->_present.rx_alloc_fail || qs->_present.rx_hw_drops || + qs->_present.rx_csum_complete || qs->_present.rx_hw_gro_packets) { + jsonw_name(json_wtr, "rx"); + jsonw_start_object(json_wtr); + if (qs->_present.rx_packets) + jsonw_uint_field(json_wtr, "packets", qs->rx_packets); + if (qs->_present.rx_bytes) + jsonw_uint_field(json_wtr, "bytes", qs->rx_bytes); + if (qs->_present.rx_alloc_fail) + jsonw_uint_field(json_wtr, "alloc-fail", qs->rx_alloc_fail); + if (qs->_present.rx_hw_drops) + jsonw_uint_field(json_wtr, "hw-drops", qs->rx_hw_drops); + if (qs->_present.rx_hw_drop_overruns) + jsonw_uint_field(json_wtr, "hw-drop-overruns", qs->rx_hw_drop_overruns); + if (qs->_present.rx_hw_drop_ratelimits) + jsonw_uint_field(json_wtr, "hw-drop-ratelimits", qs->rx_hw_drop_ratelimits); + if (qs->_present.rx_csum_complete) + jsonw_uint_field(json_wtr, "csum-complete", qs->rx_csum_complete); + if (qs->_present.rx_csum_unnecessary) + jsonw_uint_field(json_wtr, "csum-unnecessary", qs->rx_csum_unnecessary); + if (qs->_present.rx_csum_none) + jsonw_uint_field(json_wtr, "csum-none", qs->rx_csum_none); + if (qs->_present.rx_csum_bad) + jsonw_uint_field(json_wtr, "csum-bad", qs->rx_csum_bad); + if (qs->_present.rx_hw_gro_packets) + jsonw_uint_field(json_wtr, "hw-gro-packets", qs->rx_hw_gro_packets); + if (qs->_present.rx_hw_gro_bytes) + jsonw_uint_field(json_wtr, "hw-gro-bytes", qs->rx_hw_gro_bytes); + if (qs->_present.rx_hw_gro_wire_packets) + jsonw_uint_field(json_wtr, "hw-gro-wire-packets", qs->rx_hw_gro_wire_packets); + if (qs->_present.rx_hw_gro_wire_bytes) + jsonw_uint_field(json_wtr, "hw-gro-wire-bytes", qs->rx_hw_gro_wire_bytes); + jsonw_end_object(json_wtr); + } + + if (qs->_present.tx_packets || qs->_present.tx_bytes || + qs->_present.tx_hw_drops || qs->_present.tx_csum_none || + qs->_present.tx_hw_gso_packets) { + jsonw_name(json_wtr, "tx"); + jsonw_start_object(json_wtr); + if (qs->_present.tx_packets) + jsonw_uint_field(json_wtr, "packets", qs->tx_packets); + if (qs->_present.tx_bytes) + jsonw_uint_field(json_wtr, "bytes", qs->tx_bytes); + if (qs->_present.tx_hw_drops) + jsonw_uint_field(json_wtr, "hw-drops", qs->tx_hw_drops); + if (qs->_present.tx_hw_drop_errors) + jsonw_uint_field(json_wtr, "hw-drop-errors", qs->tx_hw_drop_errors); + if (qs->_present.tx_hw_drop_ratelimits) + jsonw_uint_field(json_wtr, "hw-drop-ratelimits", qs->tx_hw_drop_ratelimits); + if (qs->_present.tx_csum_none) + jsonw_uint_field(json_wtr, "csum-none", qs->tx_csum_none); + if (qs->_present.tx_needs_csum) + jsonw_uint_field(json_wtr, "needs-csum", qs->tx_needs_csum); + if (qs->_present.tx_hw_gso_packets) + jsonw_uint_field(json_wtr, "hw-gso-packets", qs->tx_hw_gso_packets); + if (qs->_present.tx_hw_gso_bytes) + jsonw_uint_field(json_wtr, "hw-gso-bytes", qs->tx_hw_gso_bytes); + if (qs->_present.tx_hw_gso_wire_packets) + jsonw_uint_field(json_wtr, "hw-gso-wire-packets", qs->tx_hw_gso_wire_packets); + if (qs->_present.tx_hw_gso_wire_bytes) + jsonw_uint_field(json_wtr, "hw-gso-wire-bytes", qs->tx_hw_gso_wire_bytes); + if (qs->_present.tx_stop) + jsonw_uint_field(json_wtr, "stop", qs->tx_stop); + if (qs->_present.tx_wake) + jsonw_uint_field(json_wtr, "wake", qs->tx_wake); + jsonw_end_object(json_wtr); + } + + jsonw_end_object(json_wtr); + } + + jsonw_end_array(json_wtr); +} + +static void print_one(bool present, const char *name, unsigned long long val, + int *line) +{ + if (!present) + return; + + if (!*line) { + printf(" "); + ++(*line); + } + + /* Don't waste space on tx- and rx- prefix, its implied by queue type */ + if (scope == NETDEV_QSTATS_SCOPE_QUEUE && + (name[0] == 'r' || name[0] == 't') && + name[1] == 'x' && name[2] == '-') + name += 3; + + printf(" %15s: %15llu", name, val); + + if (++(*line) == 3) { + printf("\n"); + *line = 0; + } +} + +static void print_plain_qstats(struct netdev_qstats_get_list *qstats) +{ + ynl_dump_foreach(qstats, qs) { + char ifname[IF_NAMESIZE]; + const char *name; + int n; + + name = if_indextoname(qs->ifindex, ifname); + if (name) + printf("%s", name); + else + printf("ifindex:%u", qs->ifindex); + + if (qs->_present.queue_type && qs->_present.queue_id) + printf("\t%s-%-3u", + netdev_queue_type_str(qs->queue_type), + qs->queue_id); + else + printf("\t "); + + n = 1; + + /* Basic counters */ + print_one(qs->_present.rx_packets, "rx-packets", qs->rx_packets, &n); + print_one(qs->_present.rx_bytes, "rx-bytes", qs->rx_bytes, &n); + print_one(qs->_present.tx_packets, "tx-packets", qs->tx_packets, &n); + print_one(qs->_present.tx_bytes, "tx-bytes", qs->tx_bytes, &n); + + /* RX error/drop counters */ + print_one(qs->_present.rx_alloc_fail, "rx-alloc-fail", + qs->rx_alloc_fail, &n); + print_one(qs->_present.rx_hw_drops, "rx-hw-drops", + qs->rx_hw_drops, &n); + print_one(qs->_present.rx_hw_drop_overruns, "rx-hw-drop-overruns", + qs->rx_hw_drop_overruns, &n); + print_one(qs->_present.rx_hw_drop_ratelimits, "rx-hw-drop-ratelimits", + qs->rx_hw_drop_ratelimits, &n); + + /* RX checksum counters */ + print_one(qs->_present.rx_csum_complete, "rx-csum-complete", + qs->rx_csum_complete, &n); + print_one(qs->_present.rx_csum_unnecessary, "rx-csum-unnecessary", + qs->rx_csum_unnecessary, &n); + print_one(qs->_present.rx_csum_none, "rx-csum-none", + qs->rx_csum_none, &n); + print_one(qs->_present.rx_csum_bad, "rx-csum-bad", + qs->rx_csum_bad, &n); + + /* RX GRO counters */ + print_one(qs->_present.rx_hw_gro_packets, "rx-hw-gro-packets", + qs->rx_hw_gro_packets, &n); + print_one(qs->_present.rx_hw_gro_bytes, "rx-hw-gro-bytes", + qs->rx_hw_gro_bytes, &n); + print_one(qs->_present.rx_hw_gro_wire_packets, "rx-hw-gro-wire-packets", + qs->rx_hw_gro_wire_packets, &n); + print_one(qs->_present.rx_hw_gro_wire_bytes, "rx-hw-gro-wire-bytes", + qs->rx_hw_gro_wire_bytes, &n); + + /* TX error/drop counters */ + print_one(qs->_present.tx_hw_drops, "tx-hw-drops", + qs->tx_hw_drops, &n); + print_one(qs->_present.tx_hw_drop_errors, "tx-hw-drop-errors", + qs->tx_hw_drop_errors, &n); + print_one(qs->_present.tx_hw_drop_ratelimits, "tx-hw-drop-ratelimits", + qs->tx_hw_drop_ratelimits, &n); + + /* TX checksum counters */ + print_one(qs->_present.tx_csum_none, "tx-csum-none", + qs->tx_csum_none, &n); + print_one(qs->_present.tx_needs_csum, "tx-needs-csum", + qs->tx_needs_csum, &n); + + /* TX GSO counters */ + print_one(qs->_present.tx_hw_gso_packets, "tx-hw-gso-packets", + qs->tx_hw_gso_packets, &n); + print_one(qs->_present.tx_hw_gso_bytes, "tx-hw-gso-bytes", + qs->tx_hw_gso_bytes, &n); + print_one(qs->_present.tx_hw_gso_wire_packets, "tx-hw-gso-wire-packets", + qs->tx_hw_gso_wire_packets, &n); + print_one(qs->_present.tx_hw_gso_wire_bytes, "tx-hw-gso-wire-bytes", + qs->tx_hw_gso_wire_bytes, &n); + + /* TX queue control */ + print_one(qs->_present.tx_stop, "tx-stop", qs->tx_stop, &n); + print_one(qs->_present.tx_wake, "tx-wake", qs->tx_wake, &n); + + if (n) + printf("\n"); + } +} + +static struct netdev_qstats_get_list * +qstats_dump(enum netdev_qstats_scope scope) +{ + struct netdev_qstats_get_list *qstats; + struct netdev_qstats_get_req *req; + struct ynl_error yerr; + struct ynl_sock *ys; + + ys = ynl_sock_create(&ynl_netdev_family, &yerr); + if (!ys) { + p_err("YNL: %s", yerr.msg); + return NULL; + } + + req = netdev_qstats_get_req_alloc(); + if (!req) { + p_err("failed to allocate qstats request"); + goto err_close; + } + + if (scope) + netdev_qstats_get_req_set_scope(req, scope); + + qstats = netdev_qstats_get_dump(ys, req); + netdev_qstats_get_req_free(req); + if (!qstats) { + p_err("failed to get queue stats: %s", ys->err.msg); + goto err_close; + } + + ynl_sock_destroy(ys); + return qstats; + +err_close: + ynl_sock_destroy(ys); + return NULL; +} + +static int do_show(int argc, char **argv) +{ + struct netdev_qstats_get_list *qstats; + + /* Parse options */ + while (argc > 0) { + if (is_prefix(*argv, "scope") || is_prefix(*argv, "group-by")) { + NEXT_ARG(); + + if (!REQ_ARGS(1)) + return -1; + + if (is_prefix(*argv, "queue")) { + scope = NETDEV_QSTATS_SCOPE_QUEUE; + } else if (is_prefix(*argv, "device")) { + scope = 0; + } else { + p_err("invalid scope value '%s'", *argv); + return -1; + } + NEXT_ARG(); + } else { + p_err("unknown option '%s'", *argv); + return -1; + } + } + + qstats = qstats_dump(scope); + if (!qstats) + return -1; + + /* Print the stats as returned by the kernel */ + if (json_output) + print_json_qstats(qstats); + else + print_plain_qstats(qstats); + + netdev_qstats_get_list_free(qstats); + return 0; +} + +static void compute_stats(__u64 *values, unsigned int count, + double *mean, double *stddev, __u64 *min, __u64 *max) +{ + double sum = 0.0, variance = 0.0; + unsigned int i; + + *min = ~0ULL; + *max = 0; + + if (count == 0) { + *mean = 0; + *stddev = 0; + *min = 0; + return; + } + + for (i = 0; i < count; i++) { + sum += values[i]; + if (values[i] < *min) + *min = values[i]; + if (values[i] > *max) + *max = values[i]; + } + + *mean = sum / count; + + if (count > 1) { + for (i = 0; i < count; i++) { + double diff = values[i] - *mean; + + variance += diff * diff; + } + *stddev = sqrt(variance / (count - 1)); + } else { + *stddev = 0; + } +} + +static void print_balance_stats(const char *name, enum netdev_queue_type type, + __u64 *values, unsigned int count) +{ + double mean, stddev, cv, ns; + __u64 min, max; + + if ((name[0] == 'r' && type != NETDEV_QUEUE_TYPE_RX) || + (name[0] == 't' && type != NETDEV_QUEUE_TYPE_TX)) + return; + + compute_stats(values, count, &mean, &stddev, &min, &max); + + cv = mean > 0 ? (stddev / mean) * 100.0 : 0.0; + ns = min + max > 0 ? (double)2 * (max - min) / (max + min) * 100 : 0.0; + + printf(" %-12s: cv=%.1f%% ns=%.1f%% stddev=%.0f\n", + name, cv, ns, stddev); + printf(" %-12s min=%llu max=%llu mean=%.0f\n", + "", min, max, mean); +} + +static void +print_balance_stats_json(const char *name, enum netdev_queue_type type, + __u64 *values, unsigned int count) +{ + double mean, stddev, cv, ns; + __u64 min, max; + + if ((name[0] == 'r' && type != NETDEV_QUEUE_TYPE_RX) || + (name[0] == 't' && type != NETDEV_QUEUE_TYPE_TX)) + return; + + compute_stats(values, count, &mean, &stddev, &min, &max); + + cv = mean > 0 ? (stddev / mean) * 100.0 : 0.0; + ns = min + max > 0 ? (double)2 * (max - min) / (max + min) * 100 : 0.0; + + jsonw_name(json_wtr, name); + jsonw_start_object(json_wtr); + jsonw_uint_field(json_wtr, "queue-count", count); + jsonw_uint_field(json_wtr, "min", min); + jsonw_uint_field(json_wtr, "max", max); + jsonw_float_field(json_wtr, "mean", mean); + jsonw_float_field(json_wtr, "stddev", stddev); + jsonw_float_field(json_wtr, "coefficient-of-variation", cv); + jsonw_float_field(json_wtr, "normalized-spread", ns); + jsonw_end_object(json_wtr); +} + +static int cmp_ifindex_type(const void *a, const void *b) +{ + const struct netdev_qstats_get_rsp *qa = a; + const struct netdev_qstats_get_rsp *qb = b; + + if (qa->ifindex != qb->ifindex) + return qa->ifindex - qb->ifindex; + if (qa->queue_type != qb->queue_type) + return qa->queue_type - qb->queue_type; + return qa->queue_id - qb->queue_id; +} + +static int do_balance(int argc, char **argv __attribute__((unused))) +{ + struct netdev_qstats_get_list *qstats; + struct netdev_qstats_get_rsp **sorted; + unsigned int count = 0; + unsigned int i, j; + int ret = 0; + + if (argc > 0) { + p_err("balance command takes no arguments"); + return -1; + } + + qstats = qstats_dump(NETDEV_QSTATS_SCOPE_QUEUE); + if (!qstats) + return -1; + + /* Count and sort queues */ + ynl_dump_foreach(qstats, qs) + count++; + + if (count == 0) { + if (json_output) + jsonw_start_array(json_wtr); + else + printf("No queue statistics available\n"); + goto exit_free_qstats; + } + + sorted = calloc(count, sizeof(*sorted)); + if (!sorted) { + p_err("failed to allocate sorted array"); + ret = -1; + goto exit_free_qstats; + } + + i = 0; + ynl_dump_foreach(qstats, qs) + sorted[i++] = qs; + + qsort(sorted, count, sizeof(*sorted), cmp_ifindex_type); + + if (json_output) + jsonw_start_array(json_wtr); + + /* Process each device/queue-type combination */ + i = 0; + while (i < count) { + __u64 *rx_packets, *rx_bytes, *tx_packets, *tx_bytes; + enum netdev_queue_type type = sorted[i]->queue_type; + unsigned int ifindex = sorted[i]->ifindex; + unsigned int queue_count = 0; + char ifname[IF_NAMESIZE]; + const char *name; + + /* Count queues for this device/type */ + for (j = i; j < count && sorted[j]->ifindex == ifindex && + sorted[j]->queue_type == type; j++) + queue_count++; + + /* Skip if no packets/bytes (inactive queues) */ + if (!sorted[i]->_present.rx_packets && + !sorted[i]->_present.rx_bytes && + !sorted[i]->_present.tx_packets && + !sorted[i]->_present.tx_bytes) + goto next_ifc; + + /* Allocate arrays for statistics */ + rx_packets = calloc(queue_count, sizeof(*rx_packets)); + rx_bytes = calloc(queue_count, sizeof(*rx_bytes)); + tx_packets = calloc(queue_count, sizeof(*tx_packets)); + tx_bytes = calloc(queue_count, sizeof(*tx_bytes)); + + if (!rx_packets || !rx_bytes || !tx_packets || !tx_bytes) { + p_err("failed to allocate statistics arrays"); + free(rx_packets); + free(rx_bytes); + free(tx_packets); + free(tx_bytes); + ret = -1; + goto exit_free_sorted; + } + + /* Collect statistics */ + for (j = 0; j < queue_count; j++) { + rx_packets[j] = sorted[i + j]->_present.rx_packets ? + sorted[i + j]->rx_packets : 0; + rx_bytes[j] = sorted[i + j]->_present.rx_bytes ? + sorted[i + j]->rx_bytes : 0; + tx_packets[j] = sorted[i + j]->_present.tx_packets ? + sorted[i + j]->tx_packets : 0; + tx_bytes[j] = sorted[i + j]->_present.tx_bytes ? + sorted[i + j]->tx_bytes : 0; + } + + name = if_indextoname(ifindex, ifname); + + if (json_output) { + jsonw_start_object(json_wtr); + if (name) + jsonw_string_field(json_wtr, "ifname", name); + jsonw_uint_field(json_wtr, "ifindex", ifindex); + jsonw_string_field(json_wtr, "queue-type", + netdev_queue_type_str(type)); + + print_balance_stats_json("rx-packets", type, + rx_packets, queue_count); + print_balance_stats_json("rx-bytes", type, + rx_bytes, queue_count); + print_balance_stats_json("tx-packets", type, + tx_packets, queue_count); + print_balance_stats_json("tx-bytes", type, + tx_bytes, queue_count); + + jsonw_end_object(json_wtr); + } else { + if (name) + printf("%s", name); + else + printf("ifindex:%u", ifindex); + printf(" %s %d queues:\n", + netdev_queue_type_str(type), queue_count); + + print_balance_stats("rx-packets", type, + rx_packets, queue_count); + print_balance_stats("rx-bytes", type, + rx_bytes, queue_count); + print_balance_stats("tx-packets", type, + tx_packets, queue_count); + print_balance_stats("tx-bytes", type, + tx_bytes, queue_count); + printf("\n"); + } + + free(rx_packets); + free(rx_bytes); + free(tx_packets); + free(tx_bytes); + +next_ifc: + i += queue_count; + } + + if (json_output) + jsonw_end_array(json_wtr); + +exit_free_sorted: + free(sorted); +exit_free_qstats: + netdev_qstats_get_list_free(qstats); + return ret; +} + +static int do_hw_gro(int argc, char **argv __attribute__((unused))) +{ + struct netdev_qstats_get_list *qstats; + + if (argc > 0) { + p_err("hw-gro command takes no arguments"); + return -1; + } + + qstats = qstats_dump(0); + if (!qstats) + return -1; + + if (json_output) + jsonw_start_array(json_wtr); + + ynl_dump_foreach(qstats, qs) { + char ifname[IF_NAMESIZE]; + const char *name; + double savings; + + if (!qs->_present.rx_packets || + !qs->_present.rx_hw_gro_packets || + !qs->_present.rx_hw_gro_wire_packets) + continue; + + if (!qs->rx_packets) + continue; + + /* How many skbs did we avoid allocating thanks to HW GRO */ + savings = (double)(qs->rx_hw_gro_wire_packets - + qs->rx_hw_gro_packets) / + qs->rx_packets * 100.0; + + name = if_indextoname(qs->ifindex, ifname); + + if (json_output) { + jsonw_start_object(json_wtr); + jsonw_uint_field(json_wtr, "ifindex", qs->ifindex); + if (name) + jsonw_string_field(json_wtr, "ifname", name); + jsonw_float_field(json_wtr, "savings", savings); + jsonw_end_object(json_wtr); + } else { + if (name) + printf("%s", name); + else + printf("ifindex:%u", qs->ifindex); + printf(": %.1f%% savings\n", savings); + } + } + + if (json_output) + jsonw_end_array(json_wtr); + + netdev_qstats_get_list_free(qstats); + return 0; +} + +static int do_help(int argc __attribute__((unused)), + char **argv __attribute__((unused))) +{ + if (json_output) { + jsonw_null(json_wtr); + return 0; + } + + fprintf(stderr, + "Usage: %1$s qstats { COMMAND | help }\n" + " %1$s qstats [ show ] [ OPTIONS ]\n" + " %1$s qstats balance\n" + " %1$s qstats hw-gro\n" + "\n" + " OPTIONS := { scope queue | group-by { device | queue } }\n" + "\n" + " show - Display queue statistics (default)\n" + " Statistics are aggregated for the entire device.\n" + " show scope queue - Display per-queue statistics\n" + " show group-by device - Display device-aggregated statistics (default)\n" + " show group-by queue - Display per-queue statistics\n" + "\n" + " Analysis:\n" + " balance - Traffic distribution between queues.\n" + " hw-gro - HW GRO effectiveness analysis\n" + " - savings - delta between packets received\n" + " on the wire and packets seen by the kernel.\n" + "", + bin_name); + + return 0; +} + +static const struct cmd qstats_cmds[] = { + { "show", do_show }, + { "balance", do_balance }, + { "hw-gro", do_hw_gro }, + { "help", do_help }, + { 0 } +}; + +int do_qstats(int argc, char **argv) +{ + return cmd_select(qstats_cmds, argc, argv, do_help); +} |
