|
|
Brussels: Uniform Interface to Driver Administration Through dladm(1m)The Driver Configuration Framework component of Project Brussels became available in Solaris Nevada snv_83 build and will be included in the OpenSolaris 2008.05 release. The Framework component provides users the following benefits when configuring drivers that use the GLDv3 framework:
This article elaborates on the the administrative features provided by Brussels. Introduction and Problem DescriptionPrior to the Brussels implementation, configuring data–link parameters on a machine running Solaris was needlessly painful. Commands and interfaces were cryptic and inconsistent across the drivers, and the process often required service interruption to all connected networks, regardless of whether the connecting link was actually being impacted by the change or not. An example of the deficiencies in these administrative methods is the tuning of interface MTU typically done to get Jumbo Frame MTU. The complications are evident as described in this inquiry in the Sun Managers Mailing List. Customizing NIC Properties in the GLDv3 FrameworkOverview of NIC Driver PropertiesDriver properties configurable with the dladm command fall into one of 2 categories:
Properties of a NIC driver are typically set with default values. However, certain networking scenarios might require you to change specific property settings of a NIC. These property settings can either be public or private properties. For example, a NIC might be communicating to an old switch that does not properly perform autonegotiation. Or, a switch might have been configured to support Jumbo frames. Or, driver–specific properties that regulate packet transmission or receiving need to be modified for the given driver. With the implementation of the Brussels project, these settings can now be reset by a single administrative tool. dladm Commands to Administer NIC PropertiesFor NIC drivers that have been converted to the GLDv3 framework, properties are configured with the dladm command. This command allows you to configure the properties dynamically without causing any network disruption on other NICs of similar types. The values that you set are stored into a dladm repository and therefore persist even after you reboot the system or unplumb the interface. Therefore, you should use dladm as the preferred command to configure NICs, instead of the ndd command. To administer NIC drivers, you use the following dladm commands:
For more information about these commands, see the dladm(1M) man page. Note - Customizing NIC properties with the dladm command is supported only in network drivers that have been converted to the GLDv3 framework, such as the following:
Work continues to make other drivers become supported in the Brussels implementation. To confirm whether your specific driver supports this feature, refer to the driver's man page. The following section provides sample procedures to set certain NIC driver properties. The selected properties are public and common to all NIC drivers. A separate section describes driver–specific properties and also provides sample procedures to configure selected private properties of the e1000g driver.. How to Enable Support for Jumbo FramesEnabling support for Jumbo frames in a network setup is a common task for most network scenarios. Support for Jumbo frames requires increasing the size of a data link's maximum transmission unit (MTU). The following procedure includes the use of customized names to identify data links. a feature that is introduced by project Clearview. This overview document introduces the concept of customized link names.
The following example builds on the following scenario:
# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 ether up 100Mb full bge0 itops1 ether up 100Mb full qfe3 web1 ether up 100Mb full bge1 # dladm show-linkprop -p mtu web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 mtu 1500 1500 -- # ifconfig web1 unplumb # dladm set-linkprop -p mtu=9000 web1 # ifconfig web1 plumb 10.10.1.2/24 up # dladm show-link web1 LINK CLASS MTU STATE OVER web1 phys 9000 up -- Notice that the MTU value is now 9000. In this example, the dladm command allowed you to change web1's MTU size directly. The previous method would have required you to unplumb net0 as well, which would have unnecessarily disrupted the primary interface's operations. How to Change Link Speed ParametersMost network setups consist of a combination of systems with varying speed capabilities. For example, between an older system and a newer system, the advertised speed between two machines might need to be changed to a lower setting to allow communication. By default, all the speed and duplex capabilities of a NIC card are advertised. This procedure shows the steps to turn off the gigabit capabilities and advertise only the megabit capabilities.
This example shows how you can prevent the link web1 from advertising gigabit capabilities. # dladm show-linkprop -p env_1000fdx_cap web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 adv_1000fdx_cap 1 -- 1,0 # dladm show-linkprop -p env_1000hdx_cap web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 adv_1000hdx_cap 1 -- 1,0 The properties that advertise the link's gigabit capabilities are adv_1000fdx_cap and adv_1000hdx_cap. To disable these properties from being advertised, you would to type the following commands: # dladm set-linkprop -p env_1000fdx_cap=0 web1 # dladm set-linkprop -p env_1000hdx_cap=0 web1 Listing the ethernet parameter settings would display the following output: # dladm show-ether web1 LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE web1 current up yes 1G-f both How to Obtain Status Information About NIC PropertiesYou can obtain information about the NIC driver's properties by displaying either the ethernet parameter settings or the link properties.
This examples displays an extended list of parameter information of a specified link. # dladm show-ether -x web1 LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE web1 current up yes 1G-f both -- capable -- yes 1G-fh,100M-fh,10M-fh both -- adv -- yes 100M-fh,10M-fh both -- peeradv -- yes 100M-f,10M-f both With the -x option, the command also displays the built-in capabilities of the specific link, as well as the capabilities that are currently advertised between the host and the peer link. The following information is displayed:
This example lists all the properties of a link. If you want to display only a specific property, you use the -p option together with the specific property that you want to monitor. # dladm show-linkprop web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 autopush -- -- -- web1 zone -- -- -- web1 duplex full full -- web1 speed 1000 1000 -- web1 state up up up,down web1 adv_autoneg_cap 0 1 1,0 web1 mtu 9000 1500 -- web1 flowctrl bi bi no,rx,tx,bi web1 adv_1000fdx_cap 1 -- -- web1 en_1000fdx_cap 1 -- 1,0 web1 adv_1000hdx_cap 1 -- 1,0 web1 en_1000hdx_cap 1 -- 1,0 web1 adv_100fdx_cap 1 -- 1,0 web1 en_100fdx_cap 1 -- 1,0 web1 adv_100hdx_cap 0 -- 1,0 web1 en_100hdx_cap 0 -- 1,0 web1 adv_10fdx_cap 0 -- 1,0 web1 en_10fdx_cap 0 -- 1,0 web1 adv_10hdx_cap 0 -- 1,0 web1 en_10hdx_cap 0 -- 1,0 The settings for the speed and duplex capabilities of the link are manually configured on the enabled-speed properties, which are labeled en_*_cap. See, for example, en_1000fdx_cap for the gigabit full-duplex capability, or en_100hdx_cap for the 100 Mbits half-duplex capability. The settings of these enabled-speed properties are advertised between the host and its peer link by corresponding advertised-speed properties, which are labeled adv_*_cap such as adv_1000fdx_cap and adv_100hdx_cap. Normally, the settings between a given enabled-speed property and the corresponding advertised-property are identical. However, if a NIC supports some advanced features such as Power Management, those features might set limits on the bits that are actually advertised between the host and its peer link on a given instant. For example, with Power Management, the values of the adv_*_cap might only be a subset of the values of the en_*_cap properties. For more details about the enabled and advertised speed properties, see the dladm(1M) man page. Configuring Properties Specific to the e1000g DriverBrussels support for the e1000g driver is available in the Solaris Nevada snv_88 build. The driver is widely used on many important Sun platforms to support the Intel PRO/1000 Gigabit NICs, such as the Sun FireTM x4200 server, the Sun NetraTM x4450 server, and the Sun Fire T2000 server. Previously, the e1000g driver, like many other NIC drivers, was configured by both using ndd command and defining property settings in the driver's e1000g.conf configuration file. This driver is now converted to the GLDv3 framework. Thus, its configuration can be uniformly performed just as other similarly-converted drivers with the dladm command. The same command can be used to fine tune parameters that are specific to the e1000g driver. For example, you might need to configure the driver to use Direct Memory Access (DMA) binding for transmission instead of the bcopy mode. Likewise, you might need to reset certain interrupt parameters to improve the performance of the driver. See the sample procedures that show how you configure these parameters. Note - The nxge driver has become Brussels-supported in the Solaris Nevada snv_88 build. The driver, which is widely used on many platforms, also has nxge-specific properties. The following how–to's about configuring private properties apply to the e1000g driver. However, the general procedures can also apply to the nxge driver. How to Set the e1000g Driver to Use Direct Memory Access BindingBulk traffic, such as file transfers, normally involves negotiation of large packets across the network. In such a case, you can obtain better performance from the e1000g driver by configuring it to automatically use DMA binding. In the following procedure, a threshold is defined for packet fragment sizes. If a fragment size surpasses the threshold, then DMA binding is used for transmitting. If a fragment size is within the threshold, then bcopy mode is used, where the fragment data is copied to the pre–allocated transmit buffer. For more details about this threshold and other tunables, see the e1000 tunables list. To set the threshold, perform the following:
How to Manually Set the Interrupt RateParameters that regulate the rate at which interrupts are delivered by the e1000g driver also affect network and system performance. Typically network packets are delivered to the upper layer of the stack by generating an interrupt for every packet. In turn the interrupt rate, by default, is automatically tuned by the e1000g driver. However, this mode might not be desirable in all network traffic conditions. For a discussion of this issue, refer to this document. Thus, in certain circumstances, setting the interrupt rate manually becomes necessary to obtain better performance. To define the interrupt rate, you set the following parameters:
This example uses an x86–based system with an e1000g NIC. The driver is configured with a threshold setting toggle between using DMA binding or the bcopy mode for transmitting packets. The setting for the interrupt throttling rate is also modified. Further, the e1000g data link has been renamed with a customized name. Therefore the configuration is performed on the data link by referring to the customized name, public0. # dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE public0 ether up 100Mb full e1000g0 # dladm show-linkprop -p _tx_bcopy_threshold public0 LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _tx_bcopy_threshold 512 512 -- # dladm show-linkprop -p _intr-throttling_rate LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _intr-throttling_rate 260 260 -- # ifconfig public0 unplumb # dladm set-linkprop -p _tx_bcopy_threshold=1024 public0 # dladm set-linkprop -p _intr_adaptive=0 public0 # dladm set-linkprop -p _intr-throttling_rate=1024 public0 # ifconfig public0 plumb 10.10.1.2/24 up # dladm show-linkprop -p _tx_bocopy_threshold=1024 public0 LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _tx_bcopy_threshold 1024 512 -- # dladm show-linkprop -p _intr_adaptive public0 LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _intr-adaptive 0 1 -- # dladm show-linkprop -p _intr-throttling_rate LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _intr-throttling_rate 1024 260 -- Future WorkThe Brussels ndd compat component, which will appear in the next OpenSolaris release, will provide additional functionality in the Framework for legacy support of the ndd(1m) commands so that drivers will not have to include complex and undocumented interfaces to support the ndd command. Also, the Brussels Persistence component will add additional functionality to allow the property settings to be automatically applied across reboot and driver restart. Future articles will describe the details of the Framework, as well as the methods used to provide ndd compatibility and persistent property settings. Work is also ongoing to provide a GUI to administer data links. A prototype of the GUI is available at the OpenSolaris project page. The prototype is an interactive GUI for demonstration purposes. Please send feedback to the Brussels discussion list. Access to the Brussels-supported command line interfaces is available in OpenSolaris 2008.05 upon its release. |