Ethernet¶
Overview¶
NetModule OEM Linux Distribution comes with built-in support for a number of Ethernet devices. This includes support for hardware solutions as simple as single port standard Ethernet PHY and very complex hardware architecture involving cascading Ethernet switches with external PHYs.
Supported scenarios¶
Basic setup¶
Single network for internal (umnet0) end external (lan0, broadr0 and broadr1) interfaces. Main IP address assigned to eth0. Common broadcast domain.
Supported hardware¶
NMHW21¶
- TI Common Platform Ethernet Switch (CPSW)
- NXP SJA1105TEL Five- Ports AVB & TSN Automotive Ethernet Switch
- NXP TJA1100 100BASE-T1 PHY for Automotive Ethernet
- SMSC LAN8710/LAN8720 PHY
+-----------------------------------------------+ +---------------------------+
| NXP SJA1105TEL switchdev driver | | TI CPSW Switch driver |
| | | |
| umnet0 broadr0 broadr1 lan0 eth0* | | eth0 <eth1> [cpu] |
+---+---------+---------+---------+---------+---+ +---+---------+---------+---+
| | | | | | | |
+---+---+ +---+---+ +---+---+ +---+---+ +---+---+ +---+---+ +---+---+ +---+---+
| sw1p0 | | sw1p1 | | sw1p2 | | sw1p3 | | sw1p4 | | sw0p0 | | sw0p1 | | sw0p2 |
+---+---+ +---+---+ +---+---+ +---+---+ +---+---+ +---+---+ +-------+ +---+---+
| | | | | | |
| | | | +------------+ |
| | | | |
+---+---+ +---+---+ +---+---+ +---+---+ +-----------------------+---+
| User | | phy6 | | phy7 | | phy1 | | TI AM335x |
| Modu- | +---+---+ +---+---+ +---+---+ | ARM Cortex A8 |
| le | | | | | |
| | +-----------------+ +-----------------+ | |
| | | NXP TJA110X | | SMSC LAN87X | | |
| | | driver | | driver | | |
+-------+ +-----------------+ +-----------------+ +---------------------------+
NXP SJA1105TEL switchdev driver¶
The SJA1105TEL switchdev driver functionality: - Ports are available in user space as netdevs (e.g. broadr1, lan0) - Hardware statistics and standard information about devices (ethtool) - Rx/Tx and error statistics (ifconfig) - Hardware offloading (bridge fdb) - VLAN tagging/bridging support (brctl, bridge vlan)
The SJA1105TEL switchdev driver limitations: - No VLAN double tagging
Hardware offloading¶
The idea is to offload the L2 data forwarding (switching) path from the kernel to the switch device by mirroring bridge FDB entries down to the device. An FDB entry is the {port, MAC, VLAN} tuple forwarding destination.
Static bridge FDB entries are installed, for example, using bridge command:
$ bridge fdb add ADDR dev DEV [vlan VID] [self]
The SJA1105TEL switch device has 1024 entries in the L2 address lookup table. Parts of the table can be statically configured, e.g.:
$ bridge fdb add 00:1b:21:2f:fa:1f dev lan0
Some of the entries are dynamically learned during operation. Show all valid FDB entries:
$ bridge fdb show
Loaded entries never time-out and cannot be replaced by learned entries. They have to be removed manually, e.g.:
$ bridge fdb del 00:1b:21:2f:fa:1f dev lan0
VLAN tagging/bridging¶
Each port of the SJA1105TEL switch has assigned a PVID in MAC Configuration table. The respective entry in the VLAN Lookup table must be loaded and the flag of the port set to indicate VLAN membership.
A Port VLAN ID (PVID) is a default VLAN ID used to tag untagged incoming frames. In addition to the PVID mandatory definition, a port can be assigned to other VLANs. In this case tagged incoming frames are expected.
E.g. isolate ports 1 (broadr0) and 2 (broadr1) for external traffic by creating a bridge with PVID 10:
$ brctl addbr bridge0
$ ip link set dev bridge0 type bridge vlan_default_pvid 10
$ brctl addif bridge0 broadr0
$ brctl addif bridge0 broadr1
$ bridge vlan
port vlan ids
bridge0 10 PVID Egress Untagged
broadr0 10 PVID Egress Untagged
broadr1 10 PVID Egress Untagged
E.g. enable VLAN filtering on bridge0 (untagged incoming frames are dropped):
$ ip link set dev bridge0 type bridge vlan_filtering 1
E.g. add VLAN for tagged incoming frames and tagged egress:
$ bridge vlan add vid 12 tagged dev broadr0
$ bridge vlan add vid 12 tagged dev broadr1
$ bridge vlan
port vlan ids
bridge0 10 PVID Egress Untagged
12
broadr0 10 PVID Egress Untagged
12
broadr1 10 PVID Egress Untagged
NXP TJA110X PHY driver¶
The driver exposes configuration options through sysfs.
Checking link status of a TJA1100 PHY.
E.g. broadr0, cable connected:
# cat /sys/bus/mdio_bus/devices/4a101000.mdio\:02/configuration/link_status
up
E.g. broadr1, cable disconnected:
# cat /sys/bus/mdio_bus/devices/4a101000.mdio\:03/configuration/link_status
down
Both TJA110X PHYs are configured as slaves by default.
E.g. check master/slave configuration of broadr0, default configuration:
# cat /sys/bus/mdio_bus/devices/4a101000.mdio\:02/configuration/link_status
slave
E.g. set broadr1 as master (a counterpart has to be set as slave):
# echo master > /sys/bus/mdio_bus/devices/4a101000.mdio\:03/configuration/master_cfg
# cat /sys/bus/mdio_bus/devices/4a101000.mdio\:03/configuration/link_status
master
The TJA1100 PHY driver can give SNR class of a connection.
E.g. check connection quality of broadr0 to a counterpart over short good quality cable:
# cat /sys/bus/mdio_bus/devices/4a101000.mdio\:02/configuration/snr_class
Class G SQI (very good link)
E.g. check connection quality of broadr1 to a counterpart over long poor quality cable:
# cat /sys/bus/mdio_bus/devices/4a101000.mdio\:03/configuration/snr_class
Class B SQI (unstable link)
User space tools and configuration¶
Check available interfaces:
$ ip link show
List IP addresses:
$ ip address show
List routes:
$ ip route show
Deactivate a link layer device:
$ ip link set dev devicename down
Activate a device:
$ ip link set dev devicename up
Print current settings of the specified device
$ ethtool devname
Print statistics of the specified device
$ ethtool -S devname
Multiple isolated interfaces example¶
Here are the steps necessary to create 3 IP interfaces able to connect to 3 isolated networks:
- Internal, 192.168.1.0/24 - connection between User Module and Linux
- Ethernet LAN, 172.16.71.0/24 - standard Ethernet connection to Linux
- BroadR LAN, 172.17.1.0/24 - BroadR connection to Linux
- Set 0.0.0.0 ipv4 address for eth0
# nmcli -p con show ethernet
...
ipv4.addresses: 0.0.0.0/32
...
- Add vlan aware interfaces on top of eth0 using nmcli (routing added automatically)
# nmcli con add type vlan con-name eth0.16 dev eth0 id 16 ip4 172.16.71.1/24
# nmcli con add type vlan con-name eth0.17 dev eth0 id 17 ip4 172.17.1.10/24
# nmcli con add type vlan con-name eth0.19 dev eth0 id 19 ip4 192.128.1.1/24
- Create bridge with eth0* as trunk port and separated access ports for all networks
# brctl addbr trunk0
# brctl addif trunk0 umnet0
# brctl addif trunk0 broadr0
# brctl addif trunk0 broadr1
# brctl addif trunk0 lan0
# brctl addif trunk0 eth0*
- Update VLAN IDs
# bridge vlan add dev eth0* vid 16 tagged
# bridge vlan add dev lan0 vid 16 pvid untagged
# bridge vlan add dev eth0* vid 17 tagged
# bridge vlan add dev broadr0 vid 17 pvid untagged
# bridge vlan add dev broadr1 vid 17 pvid untagged
# bridge vlan add dev eth0* vid 19 tagged
# bridge vlan add dev umnet0 vid 19 pvid untagged
- Clean up default PVID 1
# bridge vlan del dev umnet0 vid 1
# bridge vlan del dev broadr0 vid 1
# bridge vlan del dev broadr1 vid 1
# bridge vlan del dev lan0 vid 1
- Verify
# bridge vlan
port vlan ids
umnet0 19 PVID Egress Untagged
broadr0 17 PVID Egress Untagged
broadr1 17 PVID Egress Untagged
lan0 16 PVID Egress Untagged
eth0* 1 PVID Egress Untagged
16
17
19
trunk0 1 PVID Egress Untagged
- Analyse traces from eth0 (all traffic should be tagged with VLAN ID assigned to a port)
# tcpdump -nnei eth0 -vvv