wiki:venice/ethernet

Version 19 (modified by Ryan Erbstoesser, 4 weeks ago) ( diff )

specify loading part for switch

Venice Ethernet

Venice Ethernet device mapping:

Board Connector Silkscreen U-Boot Linux MAC/Phy Notes
GW740x J21 Eth#0 eth0 eth0 IMX8MP EQOS (0x30bf0000) RGMII GPY111 802.3at class5 PoE
- - eth1 eth1 IMX8MP FEC (0x30be0000) RGMII KSZ9897STXI Port6 cpu uplink, uses KSZ9477 SW driver
J22 Eth#1 lan1 lan1 KSZ9477 Port1 Passive 8-60V PoE
J22 Eth#2 lan2 lan2 KSZ9477 Port2
J23 Eth#3 lan3 lan3 KSZ9477 Port3
J23 Eth#4 lan4 lan4 KSZ9477 Port4
J24 Eth#5 lan5 lan5 KSZ9477 Port5
GW730x J20 Eth#0 eth0 eth0 IMX8MM FEC RGMII GPY111 802.3at class4 PoE
J21 Eth#1 eth1 eth1 / enp6s0 PCI LAN7430 or 88E8057 Passive 8-60V PoE
GW720x J17 eth0 eth0 IMX8MM FEC RGMII GPY111 802.3at class4 PoE
J17 eth1 eth1 / enp5s0 PCI LAN7430 or 88E8057 Passive 8-60V PoE
GW710x J16 Ethernet eth0 eth0 IMX8MM FEC RGMII GPY111 Passive 8-60V PoE
  • Port ordering above is from leftmost RJ45 on board when viewing the front-panel
  • If 'net.ifnames=0' kernel parameter is specified network interfaces on enumerated busses such as pci/usb are numbered sequentially (ie eth1 instead of enp6s0). See Predictable Interface Names for details
    • Bootloader variable example:
      u-boot=> print bootargs 
      bootargs=net.ifnames=0
      u-boot=> 
      
  • For boards like the GW740x which have a multi-port GbE switch, bandwidth is shared across the downstream ports

IEEE 1588 PTP Linux TimeSync

see timesync

GW740x Ethernet Notes

The GW740x uses the Microchip KSZ9897STXI part.

This part uses the KSZ9477 driver.

The GW740x has 6 RJ45's on the front panel. When looking at the board from the front with the user LED and barrel jack on the left these ports are:

eth0 (IMX8MM EQOS MAC + GPY111 RGMII PHY)
lan1 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan2 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan3 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan4 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan5 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)

In U-Boot use the 'net list' command to show the available network devices as well as the 'eth' interface name and the device name that can be used to specify the active device in U-Boot. Either name can be set to the 'ethact' env variable to specify the active device and only 1 device is active at a time for U-Boot network commands.

Note that from the CPU's perspective 'eth1' is the CPU uplink device to the KSZ9897STXI/KSZ9477 switch so it is never used by itself.

Example:
# list network devs in U-Boot
u-boot=> net list
eth2 : lan1 00:d0:12:d3:f7:8e
eth3 : lan2 00:d0:12:d3:f7:8f
eth4 : lan3 00:d0:12:d3:f7:90
eth5 : lan4 00:d0:12:d3:f7:91
eth6 : lan5 00:d0:12:d3:f7:92
eth1 : ethernet@30be0000 00:d0:12:d3:f7:8d
eth0 : ethernet@30bf0000 00:d0:12:d3:f7:8c active
# specify eth0 (the 1st RJ45) (this is also the default)
setenv ethact eth0
# specify lan1 (the 2nd RJ45)
setenv ethact lan1 # or eth2 if you want but that is a U-Boot only name
# save env if you want this to persist in U-Boot across boots
saveenv

For Linux, specifically our kernel and BSP you will see the device naming above:

eth0 (IMX8MM EQOS MAC + GPY111 RGMII PHY)
lan1 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan2 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan3 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan4 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)
lan5 (KSZ9477 switch port from IMX8MM FEC MAC (eth1) CPU uplink)

In Linux 5.12+ you do not need to bring up the CPU uplink port to the switch (eth1) as it will be brought up whenever any of the downstream ports are brought up. Modern Linux DSA (Distributed Switch Architecture) provides more or less full interface devices for the ports on the switch thus you can have IP configuration, VLAN etc, on the individual ports.

Additionally our BSP is pre-configured to bring up eth0 with DHCP so if you have the first RJ45 plugged into your network you should see 'eth0' properly configured for networking. If you want to use the 2nd RJ45 it would be 'lan1', etc.

Linux 5.12 DSA drivers will bring up the cpu port interface automatically which is why one may see eth1 up after bringing up any of the lan ports.

GW7400 Switch

The GW74xx switched ports utilize a KSZ9897STXI (SW driver KSZ9477) switch by Microchip.

This is a Distributed Switch Architecture (DSA) setup. Thus, this is not a 'basic hub' that has everything hardwired together.

The switch needs to be configured in software to behave properly for each application.

More information about DSA can be found here:

IMX8M FEC Ethernet Interrupt Coalesence

The FEC Ethernet network interface controller on the IMX8MM and IMX8MP features interrupt coalesence which is enabled by default and increases packet latency for small numbers of packets.

The purpose of the interrupt coalescing is to reduce the number of interrupts generated by the MAC so as to reduce the CPU loading.

The default is to delay an RX and TX interrupt by 1ms or until 200 frames have occurred:

# ethtool --show-coalesce eth0
Coalesce parameters for eth0:
Adaptive RX: off  TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 1000
rx-frames: 200
rx-usecs-irq: 0
rx-frames-irq: 0

tx-usecs: 1000
tx-frames: 200
tx-usecs-irq: 0
tx-frames-irq: 0

rx-usecs-low: 0
rx-frames-low: 0
tx-usecs-low: 0
tx-frames-low: 0

rx-usecs-high: 0
rx-frames-high: 0
tx-usecs-high: 0
tx-frames-high: 0

This would cause a packet latency of >1ms for light network activity.

This can be modified using the userspace 'ethtool':

# default is 1000us 200 frames, change to 1/1
ethtool --coalesce eth0 tx-usecs 1
ethtool --coalesce eth0 tx-frames 1
ethtool --coalesce eth0 rx-usecs 1
ethtool --coalesce eth0 rx-frames 1
  • ping times now will be < 0.5ms

And if you wanted to change it back to defaults for example:

# change back to default
ethtool --coalesce eth0 tx-usecs 1000
ethtool --coalesce eth0 tx-frames 200
ethtool --coalesce eth0 rx-usecs 1000
ethtool --coalesce eth0 rx-frames 200
  • ping times now will be >1ms
Note: See TracWiki for help on using the wiki.