1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
|
.. SPDX-License-Identifier: GPL-2.0
=========
S/390 PCI
=========
Authors:
- Pierre Morel
- Niklas Schnelle
Copyright, IBM Corp. 2020
Command line parameters and debugfs entries
===========================================
Command line parameters
-----------------------
* nomio
Do not use PCI Mapped I/O (MIO) instructions.
* norid
Ignore the RID field and force use of one PCI domain per PCI function.
debugfs entries
---------------
The S/390 debug feature (s390dbf) generates views to hold various debug results
in sysfs directories of the form:
* /sys/kernel/debug/s390dbf/pci_*/
For example:
- /sys/kernel/debug/s390dbf/pci_msg/sprintf
holds messages from the processing of PCI events, like machine check handling
and setting of global functionality, like UID checking.
Change the level of logging to be more or less verbose by piping
a number between 0 and 6 to /sys/kernel/debug/s390dbf/pci_*/level. For
details, see the documentation on the S/390 debug feature at
Documentation/arch/s390/s390dbf.rst.
Sysfs entries
=============
Entries specific to zPCI functions and entries that hold zPCI information.
* /sys/bus/pci/slots/XXXXXXXX:
The slot entries are set up using the function identifier (FID) of the PCI
function as slot name. The format depicted as XXXXXXXX above is 8 hexadecimal
digits with 0 padding and lower case hexadecimal digits.
- /sys/bus/pci/slots/XXXXXXXX/power
In addition to using the FID as the name of the slot, the slot directory
also contains the following s390-specific slot attributes.
- uid:
The User-defined identifier (UID) of the function which may be configured
by this slot. See also the corresponding attribute of the device.
A physical function that currently supports a virtual function cannot be
powered off until all virtual functions are removed with:
echo 0 > /sys/bus/pci/devices/DDDD:BB:dd.f/sriov_numvfs
* /sys/bus/pci/devices/DDDD:BB:dd.f/:
- function_id:
The zPCI function identifier (FID) is a 32-bit hexadecimal value that
uniquely identifies the PCI function. Unless the hypervisor provides
a virtual FID e.g. on KVM this identifier is unique across the machine even
between different partitions.
- function_handle:
This 32-bit hexadecimal value is a low-level identifier used for a PCI
function. Note that the function handle may be changed and become invalid
on PCI events and when enabling/disabling the PCI function.
- pchid:
This 16-bit hexadecimal value encodes a model-dependent location for
the PCI function.
- pfgid:
PCI function group ID; functions that share identical functionality
use a common identifier.
A PCI group defines interrupts, IOMMU, IOTLB, and DMA specifics.
- vfn:
The virtual function number, from 1 to N for virtual functions,
0 for physical functions.
- pft:
The PCI function type is an s390-specific type attribute. It indicates
a more general, usage oriented, type than PCI Specification
class/vendor/device identifiers. That is PCI functions with the same pft
value may be backed by different hardware implementations. At the same time
apart from unclassified functions (pft is 0x00) the same pft value
generally implies a similar usage model. At the same time the same
PCI hardware device may appear with different pft values when in a
different usage model. For example NETD and NETH VFs may be implemented
by the same PCI hardware device but in NETD the parent Physical Function
is user managed while with NETH it is platform managed.
Currently the following PFT values are defined:
- 0x00 (UNC): Unclassified
- 0x02 (ROCE): RoCE Express
- 0x05 (ISM): Internal Shared Memory
- 0x0a (ROC2): RoCE Express 2
- 0x0b (NVMe): NVMe
- 0x0c (NETH): Network Express hybrid
- 0x0d (CNW): Cloud Network Adapter
- 0x0f (NETD): Network Express direct
- port:
The port is a decimal value corresponding to the physical port the function
is attached to. Virtual Functions (VFs) share the port with their parent
Physical Function (PF). A value of 0 indicates that the port attribute is
not applicable for that PCI function type.
- uid:
The user-defined identifier (UID) for a PCI function is a 32-bit
hexadecimal value. It is defined on a per instance basis as part of the
partition, KVM guest, or z/VM guest configuration. If UID Checking is
enabled the platform ensures that the UID is unique within that instance
and no two PCI functions with the same UID will be visible to the instance.
Independent of this guarantee and unlike the function ID (FID) the UID may
be the same in different partitions within the same machine. This allows to
create PCI configurations in multiple partitions to be identical in the
UID-namespace.
- uid_is_unique:
A 0 or 1 flag indicating whether the user-defined identifier (UID) is
guaranteed to be and remain unique within this Linux instance. This
platform feature is called UID Checking.
- pfip/segmentX:
The segments determine the isolation of a function.
They correspond to the physical path to the function.
The more the segments are different, the more the functions are isolated.
- fidparm:
Contains an 8-bit-per-PCI function parameter field in hexadecimal provided
by the platform. The meaning of this field is PCI function type specific.
For NETH VFs a value of 0x01 indicates that the function supports
promiscuous mode.
* /sys/firmware/clp/uid_checking:
In addition to the per-device uid_is_unique attribute this presents a
global indication of whether UID Checking is enabled. This allows users
to check for UID Checking even when no PCI functions are configured.
Enumeration and hotplug
=======================
The PCI address consists of four parts: domain, bus, device and function,
and is of this form: DDDD:BB:dd.f.
* For a PCI function for which the platform does not expose the RID, the
pci=norid kernel parameter is used, or a so-called isolated Virtual Function
which does have RID information but is used without its parent Physical
Function being part of the same PCI configuration:
- There is only one function per domain.
- The domain is set from the zPCI function's UID if UID Checking is on;
otherwise the domain ID is generated dynamically and is not stable
across reboots or hot plug.
* For a PCI function for which the platform exposes the RID and which
is not an Isolated Virtual Function:
- There is still only one bus per domain.
- There can be up to 256 PCI functions per bus.
- The domain part of the address of all functions within the same topology is
that of the configured PCI function with the lowest devfn within that
topology.
- Virtual Functions generated by an SR-IOV capable Physical Function only
become visible once SR-IOV is enabled.
|
