Skip to content
/ linux Public

Commit fcdc8ce

Browse files
Jiri Pirkodavem330
Jiri Pirko
authored and
davem330
committed
devlink: implement line card provisioning
In order to be able to configure all needed stuff on a port/netdevice of a line card without the line card being present, introduce line card provisioning. Basically by setting a type, provisioning process will start and driver is supposed to create a placeholder for instances (ports/netdevices) for a line card type. Allow the user to query the supported line card types over line card get command. Then implement two netlink command SET to allow user to set/unset the card type. On the driver API side, add provision/unprovision ops and supported types array to be advertised. Upon provision op call, the driver should take care of creating the instances for the particular line card type. Introduce provision_set/clear() functions to be called by the driver once the provisioning/unprovisioning is done on its side. These helpers are not to be called directly due to the async nature of provisioning. Example: $ devlink port # No ports are listed $ devlink lc pci/0000:01:00.0: lc 1 state unprovisioned supported_types: 16x100G lc 2 state unprovisioned supported_types: 16x100G lc 3 state unprovisioned supported_types: 16x100G lc 4 state unprovisioned supported_types: 16x100G lc 5 state unprovisioned supported_types: 16x100G lc 6 state unprovisioned supported_types: 16x100G lc 7 state unprovisioned supported_types: 16x100G lc 8 state unprovisioned supported_types: 16x100G $ devlink lc set pci/0000:01:00.0 lc 8 type 16x100G $ devlink lc show pci/0000:01:00.0 lc 8 pci/0000:01:00.0: lc 8 state active type 16x100G supported_types: 16x100G $ devlink port pci/0000:01:00.0/0: type notset flavour cpu port 0 splittable false pci/0000:01:00.0/53: type eth netdev enp1s0nl8p1 flavour physical lc 8 port 1 splittable true lanes 4 pci/0000:01:00.0/54: type eth netdev enp1s0nl8p2 flavour physical lc 8 port 2 splittable true lanes 4 pci/0000:01:00.0/55: type eth netdev enp1s0nl8p3 flavour physical lc 8 port 3 splittable true lanes 4 pci/0000:01:00.0/56: type eth netdev enp1s0nl8p4 flavour physical lc 8 port 4 splittable true lanes 4 pci/0000:01:00.0/57: type eth netdev enp1s0nl8p5 flavour physical lc 8 port 5 splittable true lanes 4 pci/0000:01:00.0/58: type eth netdev enp1s0nl8p6 flavour physical lc 8 port 6 splittable true lanes 4 pci/0000:01:00.0/59: type eth netdev enp1s0nl8p7 flavour physical lc 8 port 7 splittable true lanes 4 pci/0000:01:00.0/60: type eth netdev enp1s0nl8p8 flavour physical lc 8 port 8 splittable true lanes 4 pci/0000:01:00.0/61: type eth netdev enp1s0nl8p9 flavour physical lc 8 port 9 splittable true lanes 4 pci/0000:01:00.0/62: type eth netdev enp1s0nl8p10 flavour physical lc 8 port 10 splittable true lanes 4 pci/0000:01:00.0/63: type eth netdev enp1s0nl8p11 flavour physical lc 8 port 11 splittable true lanes 4 pci/0000:01:00.0/64: type eth netdev enp1s0nl8p12 flavour physical lc 8 port 12 splittable true lanes 4 pci/0000:01:00.0/125: type eth netdev enp1s0nl8p13 flavour physical lc 8 port 13 splittable true lanes 4 pci/0000:01:00.0/126: type eth netdev enp1s0nl8p14 flavour physical lc 8 port 14 splittable true lanes 4 pci/0000:01:00.0/127: type eth netdev enp1s0nl8p15 flavour physical lc 8 port 15 splittable true lanes 4 pci/0000:01:00.0/128: type eth netdev enp1s0nl8p16 flavour physical lc 8 port 16 splittable true lanes 4 $ devlink lc set pci/0000:01:00.0 lc 8 notype Signed-off-by: Jiri Pirko <jiri@nvidia.com> Signed-off-by: Ido Schimmel <idosch@nvidia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
1 parent c246f9b commit fcdc8ce

File tree

5 files changed

+497
-5
lines changed

5 files changed

+497
-5
lines changed

‎Documentation/networking/devlink/devlink-linecard.rst‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
.. SPDX-License-Identifier: GPL-2.0
2+
3+
=================
4+
Devlink Line card
5+
=================
6+
7+
Background
8+
==========
9+
10+
The ``devlink-linecard`` mechanism is targeted for manipulation of
11+
line cards that serve as a detachable PHY modules for modular switch
12+
system. Following operations are provided:
13+
14+
* Get a list of supported line card types.
15+
* Provision of a slot with specific line card type.
16+
* Get and monitor of line card state and its change.
17+
18+
Line card according to the type may contain one or more gearboxes
19+
to mux the lanes with certain speed to multiple ports with lanes
20+
of different speed. Line card ensures N:M mapping between
21+
the switch ASIC modules and physical front panel ports.
22+
23+
Overview
24+
========
25+
26+
Each line card devlink object is created by device driver,
27+
according to the physical line card slots available on the device.
28+
29+
Similar to splitter cable, where the device might have no way
30+
of detection of the splitter cable geometry, the device
31+
might not have a way to detect line card type. For that devices,
32+
concept of provisioning is introduced. It allows the user to:
33+
34+
* Provision a line card slot with certain line card type
35+
36+
- Device driver would instruct the ASIC to prepare all
37+
resources accordingly. The device driver would
38+
create all instances, namely devlink port and netdevices
39+
that reside on the line card, according to the line card type
40+
* Manipulate of line card entities even without line card
41+
being physically connected or powered-up
42+
* Setup splitter cable on line card ports
43+
44+
- As on the ordinary ports, user may provision a splitter
45+
cable of a certain type, without the need to
46+
be physically connected to the port
47+
* Configure devlink ports and netdevices
48+
49+
Netdevice carrier is decided as follows:
50+
51+
* Line card is not inserted or powered-down
52+
53+
- The carrier is always down
54+
* Line card is inserted and powered up
55+
56+
- The carrier is decided as for ordinary port netdevice
57+
58+
Line card state
59+
===============
60+
61+
The ``devlink-linecard`` mechanism supports the following line card states:
62+
63+
* ``unprovisioned``: Line card is not provisioned on the slot.
64+
* ``unprovisioning``: Line card slot is currently being unprovisioned.
65+
* ``provisioning``: Line card slot is currently in a process of being provisioned
66+
with a line card type.
67+
* ``provisioning_failed``: Provisioning was not successful.
68+
* ``provisioned``: Line card slot is provisioned with a type.
69+
70+
The following diagram provides a general overview of ``devlink-linecard``
71+
state transitions::
72+
73+
+-------------------------+
74+
| |
75+
+----------------------------------> unprovisioned |
76+
| | |
77+
| +--------|-------^--------+
78+
| | |
79+
| | |
80+
| +--------v-------|--------+
81+
| | |
82+
| | provisioning |
83+
| | |
84+
| +------------|------------+
85+
| |
86+
| +-----------------------------+
87+
| | |
88+
| +------------v------------+ +------------v------------+
89+
| | | | |
90+
+----- provisioning_failed | | provisioned |
91+
| | | | |
92+
| +------------^------------+ +------------|------------+
93+
| | |
94+
| | |
95+
| | +------------v------------+
96+
| | | |
97+
| | | unprovisioning |
98+
| | | |
99+
| | +------------|------------+
100+
| | |
101+
| +-----------------------------+
102+
| |
103+
+-----------------------------------------------+
104+
105+
106+
Example usage
107+
=============
108+
109+
.. code:: shell
110+
111+
$ devlink lc show [ DEV [ lc LC_INDEX ] ]
112+
$ devlink lc set DEV lc LC_INDEX [ { type LC_TYPE | notype } ]
113+
114+
# Show current line card configuration and status for all slots:
115+
$ devlink lc
116+
117+
# Set slot 8 to be provisioned with type "16x100G":
118+
$ devlink lc set pci/0000:01:00.0 lc 8 type 16x100G
119+
120+
# Set slot 8 to be unprovisioned:
121+
$ devlink lc set pci/0000:01:00.0 lc 8 notype

‎Documentation/networking/devlink/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ general.
3939
devlink-resource
4040
devlink-reload
4141
devlink-trap
42+
devlink-linecard
4243

4344
Driver-specific documentation
4445
-----------------------------

‎include/net/devlink.h‎

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,40 @@ struct devlink_port_new_attrs {
149149
sfnum_valid:1;
150150
};
151151

152+
/**
153+
* struct devlink_linecard_ops - Linecard operations
154+
* @provision: callback to provision the linecard slot with certain
155+
* type of linecard. As a result of this operation,
156+
* driver is expected to eventually (could be after
157+
* the function call returns) call one of:
158+
* devlink_linecard_provision_set()
159+
* devlink_linecard_provision_fail()
160+
* @unprovision: callback to unprovision the linecard slot. As a result
161+
* of this operation, driver is expected to eventually
162+
* (could be after the function call returns) call
163+
* devlink_linecard_provision_clear()
164+
* devlink_linecard_provision_fail()
165+
* @same_provision: callback to ask the driver if linecard is already
166+
* provisioned in the same way user asks this linecard to be
167+
* provisioned.
168+
* @types_count: callback to get number of supported types
169+
* @types_get: callback to get next type in list
170+
*/
171+
struct devlink_linecard_ops {
172+
int (*provision)(struct devlink_linecard *linecard, void *priv,
173+
const char *type, const void *type_priv,
174+
struct netlink_ext_ack *extack);
175+
int (*unprovision)(struct devlink_linecard *linecard, void *priv,
176+
struct netlink_ext_ack *extack);
177+
bool (*same_provision)(struct devlink_linecard *linecard, void *priv,
178+
const char *type, const void *type_priv);
179+
unsigned int (*types_count)(struct devlink_linecard *linecard,
180+
void *priv);
181+
void (*types_get)(struct devlink_linecard *linecard,
182+
void *priv, unsigned int index, const char **type,
183+
const void **type_priv);
184+
};
185+
152186
struct devlink_sb_pool_info {
153187
enum devlink_sb_pool_type pool_type;
154188
u32 size;
@@ -1537,9 +1571,14 @@ void devlink_port_attrs_pci_sf_set(struct devlink_port *devlink_port,
15371571
int devlink_rate_leaf_create(struct devlink_port *port, void *priv);
15381572
void devlink_rate_leaf_destroy(struct devlink_port *devlink_port);
15391573
void devlink_rate_nodes_destroy(struct devlink *devlink);
1540-
struct devlink_linecard *devlink_linecard_create(struct devlink *devlink,
1541-
unsigned int linecard_index);
1574+
struct devlink_linecard *
1575+
devlink_linecard_create(struct devlink *devlink, unsigned int linecard_index,
1576+
const struct devlink_linecard_ops *ops, void *priv);
15421577
void devlink_linecard_destroy(struct devlink_linecard *linecard);
1578+
void devlink_linecard_provision_set(struct devlink_linecard *linecard,
1579+
const char *type);
1580+
void devlink_linecard_provision_clear(struct devlink_linecard *linecard);
1581+
void devlink_linecard_provision_fail(struct devlink_linecard *linecard);
15431582
int devlink_sb_register(struct devlink *devlink, unsigned int sb_index,
15441583
u32 size, u16 ingress_pools_count,
15451584
u16 egress_pools_count, u16 ingress_tc_count,

‎include/uapi/linux/devlink.h‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,18 @@ enum devlink_reload_limit {
343343

344344
#define DEVLINK_RELOAD_LIMITS_VALID_MASK (_BITUL(__DEVLINK_RELOAD_LIMIT_MAX) - 1)
345345

346+
enum devlink_linecard_state {
347+
DEVLINK_LINECARD_STATE_UNSPEC,
348+
DEVLINK_LINECARD_STATE_UNPROVISIONED,
349+
DEVLINK_LINECARD_STATE_UNPROVISIONING,
350+
DEVLINK_LINECARD_STATE_PROVISIONING,
351+
DEVLINK_LINECARD_STATE_PROVISIONING_FAILED,
352+
DEVLINK_LINECARD_STATE_PROVISIONED,
353+
354+
__DEVLINK_LINECARD_STATE_MAX,
355+
DEVLINK_LINECARD_STATE_MAX = __DEVLINK_LINECARD_STATE_MAX - 1
356+
};
357+
346358
enum devlink_attr {
347359
/* don't change the order or add anything between, this is ABI! */
348360
DEVLINK_ATTR_UNSPEC,
@@ -559,6 +571,9 @@ enum devlink_attr {
559571
DEVLINK_ATTR_REGION_MAX_SNAPSHOTS, /* u32 */
560572

561573
DEVLINK_ATTR_LINECARD_INDEX, /* u32 */
574+
DEVLINK_ATTR_LINECARD_STATE, /* u8 */
575+
DEVLINK_ATTR_LINECARD_TYPE, /* string */
576+
DEVLINK_ATTR_LINECARD_SUPPORTED_TYPES, /* nested */
562577

563578
/* add new attributes above here, update the policy in devlink.c */
564579

0 commit comments

Comments
 (0)