leo-iot
/
vendor


			
				
					
						
						
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304
							<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>Npcap internals</title><meta name="generator" content="DocBook XSL Stylesheets V1.79.2"><meta name="description" content="Describes the internal structure and interfaces of Npcap: the NPF driver and Packet.dll"><link rel="home" href="index.html" title="Npcap Reference Guide"><link rel="up" href="index.html" title="Npcap Reference Guide"><link rel="prev" href="npcap-tutorial.html" title="Npcap Development Tutorial"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Npcap internals</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="npcap-tutorial.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> </td></tr></table><hr></div><div class="sect1"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="npcap-internals"></a>Npcap internals</h2></div><div><div class="abstract"><p class="title"><b>Abstract</b></p>
      <p>Describes the internal structure and interfaces of Npcap: the NPF
        driver and Packet.dll</p>
    </div></div></div></div>
  
  
  <p>This portion of the manual describes the internal structure and
    interfaces of Npcap, starting from the lowest-level module. It is targeted
    at people who must extend or modify this software, or to the ones
    interested in how it works. Therefore, developers who just want to use
    Npcap in their software don't need to read it.</p>

  <div class="sect2"><div class="titlepage"><div><div><h3 class="title"><a name="npcap-structure"></a>Npcap structure</h3></div></div></div>
    

    <p>Npcap is an architecture for packet capture and network analysis for the
      Win32 platforms. It includes a kernel-level packet filter, a
      low-level dynamic link library (packet.dll), and a high-level and
      system-independent library (wpcap.dll).</p>

    <p>Why do we use the term <em class="wordasword">architecture</em> rather
      than <em class="wordasword">library</em>? Because packet capture is a low
      level mechanism that requires a strict interaction with the network
      adapter and with the operating system, in particular with its networking
      implementation, so a simple library is not sufficient.</p>

    
    <div class="sect3"><div class="titlepage"><div><div><h4 class="title"><a name="id582422"></a>Main components of Npcap.</h4></div></div></div>
      

      <p>First, a capture system needs to bypass the operating systems's
        protocol stack in order to access the raw data transiting on the
        network. This requires a portion running inside the kernel of OS,
        interacting directly with the network interface drivers. This portion
        is very system dependent, and in our solution it is realized as a
        device driver, called Netgroup Packet Filter (NPF); This driver offers
        basic features like packet capture and injection, as well as more
        advanced ones like a programmable filtering system and a monitoring
        engine. The filtering system can be used to restrict a capture session
        to a subset of the network traffic (e.g. it is possible to capture only
        the ftp traffic generated by a particular host); the monitoring engine
        provides a powerful but simple to use mechanism to obtain statistics on
        the traffic (e.g. it is possible to obtain the network load or the
        amount of data exchanged between two hosts).</p>

      <p>Second, the capture system must export an interface that user-level
        applications will use to take advantage of the features provided by the
        kernel driver. Npcap provides two different libraries:
        <code class="filename">packet.dll</code> and
        <code class="filename">wpcap.dll</code>.</p>

      <p> Packet.dll offers a low-level API that can be used to directly
        access the functions of the driver, with a programming interface
        independent from the Microsoft OS.</p>

      
      <p>Wpcap.dll exports a more powerful set of high level capture
        primitives that are compatible with libpcap, the well known Unix
        capture library. These functions enable packet capture in a manner that
        is independent of the underlying network hardware and operating
        system.</p>
    </div>
  </div>

  <div class="sect2"><div class="titlepage"><div><div><h3 class="title"><a name="npcap-internals-driver"></a>Npcap driver internals</h3></div></div></div>
    

    <p>This section documents the internals of the Netgroup Packet Filter
      (NPF), the kernel portion of Npcap. Normal users are probably interested
      in how to use Npcap and not in its internal structure. Therefore the
      information present in this module is destined mainly to Npcap developers
      and maintainers, or to the people interested in how the driver works. In
      particular, a good knowledge of OSes, networking and Windows kernel
      programming and device drivers development is required to profitably read
      this section.</p>

    <p>NPF is the Npcap component that does the hard work, processing the
      packets that transit on the network and exporting capture, injection and
      analysis capabilities to user-level.</p>

    <p>The following paragraphs will describe the interaction of NPF with
      the OS and its basic structure.</p>

    <div class="sect3"><div class="titlepage"><div><div><h4 class="title"><a name="npcap-internals-driver-ndis"></a>NPF and NDIS</h4></div></div></div>
      

      <p>NDIS (Network Driver Interface Specification) is a standard that
        defines the communication between a network adapter (or, better, the
        driver that manages it) and the protocol drivers (that implement for
        example TCP/IP). Main NDIS purpose is to act as a wrapper that allows
        protocol drivers to send and receive packets onto a network (LAN or
        WAN) without caring either the particular adapter or the particular
        Win32 operating system.</p>

      <p>NDIS supports four types of network drivers:</p>

      <div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem">
          <p><span class="emphasis"><em>Miniport drivers</em></span>. Miniport drivers
            directly manage network interface cards, referred to as NICs. The
            miniport drivers interface directly to the hardware at their lower
            edge and at their upper edge present an interface to allow upper
            layers to send packets on the network, to handle interrupts, to
            reset the NIC, to halt the NIC and to query and set the operational
            characteristics of the driver.</p>
          <p>Miniport drivers implement only the hardware-specific
            operations necessary to manage a NIC, including sending and
            receiving data on the NIC. Operations common to all lowest level
            NIC drivers, such as synchronization, is provided by NDIS.
            Miniports do not call operating system routines directly; their
            interface to the operating system is NDIS.</p>
          <p>A miniport does not keep track of bindings. It merely passes
            packets up to NDIS and NDIS makes sure that these packets are
            passed to the correct protocols.</p>
        </li><li class="listitem">
          <p><span class="emphasis"><em>Intermediate drivers</em></span>. Intermediate drivers
            interface between an upper-level driver such as a protocol driver
            and a miniport. To the upper-level driver, an intermediate driver
            looks like a miniport. To a miniport, the intermediate driver looks
            like a protocol driver. An intermediate protocol driver can layer
            on top of another intermediate driver although such layering could
            have a negative effect on system performance. A typical reason for
            developing an intermediate driver is to perform media translation
            between an existing legacy protocol driver and a miniport that
            manages a NIC for a new media type unknown to the protocol driver.
            For instance, an intermediate driver could translate from LAN
            protocol to ATM protocol. An intermediate driver cannot communicate
            with user-mode applications, but only with other NDIS drivers.</p>
        </li><li class="listitem">
          <p><span class="emphasis"><em>Filter drivers</em></span>. Filter drivers can monitor
            and modify traffic between protocol drivers and miniport drivers
            like an intermediate driver, but are much simpler. They have less
            processing overhead than intermediate drivers.</p>
        </li><li class="listitem">
          <p><span class="emphasis"><em>Transport drivers or protocol drivers</em></span>. A
            protocol driver implements a network protocol stack such as IPX/SPX
            or TCP/IP, offering its services over one or more network interface
            cards. A protocol driver services application-layer clients at its
            upper edge and connects to one or more NIC driver(s) or
            intermediate NDIS driver(s) at its lower edge.</p>
        </li></ol></div>

      <p>NPF is implemented as a filter driver. In order to provide complete
        access to the raw traffic and allow injection of packets, it is
        registered as a modifying filter driver in the compression
        <code class="literal">FilterClass</code>.</p>

      <p>Notice that the various Windows operating systems have different
        versions of NDIS: NPF is NDIS 6.0 compliant, and so requires a Windows
        OS that supports NDIS 6.0: Windows Vista or later.</p>

      
      
    </div>
    <div class="sect3"><div class="titlepage"><div><div><h4 class="title"><a name="npcap-internals-structure"></a>NPF structure basics</h4></div></div></div>
      

      

      <p>NPF is able to perform a number of different operations: capture,
        monitoring, packet injection. The following paragraphs
        will describe shortly each of these operations.</p>
      <div class="sect4"><div class="titlepage"><div><div><h5 class="title"><a name="npcap-internals-capture"></a>Packet Capture</h5></div></div></div>
        

        <p>The most important operation of NPF is packet capture.  During a
          capture, the driver sniffs the packets using a network interface and
          delivers them intact to the user-level applications.</p>
        <p>The capture process relies on two main components:</p>
        <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>A packet filter that decides if an incoming packet
              has to be accepted and copied to the listening application.  Most
              applications using NPF reject far more packets than those
              accepted, therefore a versatile and efficient packet filter is
              critical for good over-all performance. A packet filter is a
              function with boolean output that is applied to a packet. If the
              value of the function is true the capture driver copies the
              packet to the application; if it is false the packet is
              discarded. NPF packet filter is a bit more complex, because it
              determines not only if the packet should be kept, but also the
              amount of bytes to keep. The filtering system adopted by NPF
              derives from the <span class="emphasis"><em>BSD Packet Filter</em></span> (BPF), a
              virtual processor able to execute filtering programs expressed in
              a pseudo-assembler and created at user level. The application
              takes a user-defined filter (e.g. <span class="quote">&#8220;<span class="quote">pick up all UDP
                packets</span>&#8221;</span>) and, using wpcap.dll, compiles them into a BPF
              program (e.g.  <span class="quote">&#8220;<span class="quote">if the packet is IP and the
                <code class="literal">protocol type</code>  field is equal to 17, then
                return true</span>&#8221;</span>). Then, the application uses the
              <code class="literal">BIOCSETF</code> IOCTL to inject the filter in the
              kernel. At this point, the program is executed for every incoming
              packet, and only the conformant packets are accepted. Unlike
              traditional solutions, NPF does not
              <span class="emphasis"><em>interpret</em></span> the filters, but it
              <span class="emphasis"><em>executes</em></span> them. For performance reasons,
              before using the filter NPF feeds it to a JIT compiler that
              translates it into a native 80x86 function. When a packet is
              captured, NPF calls this native function instead of invoking the
              filter interpreter, and this makes the process very fast.  The
              concept behind this optimization is very similar to the one of
              Java jitters.</p>
          </li><li class="listitem">
            <p>A circular buffer to store the packets and avoid loss. A
              packet is stored in the buffer with a header that maintains
              information like the timestamp and the size of the packet.
              Moreover, an alignment padding is inserted between the packets in
              order to speed-up the access to their data by the applications.
              Groups of packets can be copied with a single operation from the
              NPF buffer to the applications. This improves performances
              because it minimizes the number of reads. If the buffer is full
              when a new packet arrives, the packet is discarded and hence it's
              lost. Both kernel and user buffer can be changed at runtime for
              maximum versatility: packet.dll and wpcap.dll provide functions
              for this purpose.</p>
          </li></ul></div>

        <p>The size of the user buffer is very important because it determines
          the <span class="emphasis"><em>maximum</em></span> amount of data that can be copied from
          kernel space to user space within a single system call. On the other
          hand, it can be noticed that also the <span class="emphasis"><em>minimum</em></span>
          amount of data that can be copied in a single call is extremely
          important. In presence of a large value for this variable, the kernel
          waits for the arrival of several packets before copying the data to the
          user. This guarantees a low number of system calls, i.e. low processor
          usage, which is a good setting for applications like sniffers. On the
          other side, a small value means that the kernel will copy the packets
          as soon as the application is ready to receive them. This is excellent
          for real time applications (like, for example, ARP redirectors or
          bridges) that need the better responsiveness from the kernel.  From
          this point of view, NPF has a configurable behavior, that allows users
          to choose between best efficiency or best responsiveness (or any
          intermediate situation).</p>

        <p>The wpcap library includes a couple of system calls that can be
          used both to set the timeout after which a read expires and the minimum
          amount of data that can be transferred to the application. By default,
          the read timeout is 1 second, and the minimum amount of data copied
          between the kernel and the application is 16K.</p>
      </div>
      <div class="sect4"><div class="titlepage"><div><div><h5 class="title"><a name="npcap-internals-injection"></a>Packet injection</h5></div></div></div>
        

        <p>NPF allows to write raw packets to the network. To send data, a
          user-level application performs a WriteFile() system call on the NPF
          device file. The data is sent to the network as is, without
          encapsulating it in any protocol, therefore the application will have
          to build the various headers for each packet. The application usually
          does not need to generate the FCS because it is calculated by the
          network adapter hardware and it is attached automatically at the end of
          a packet before sending it to the network.</p>

        <p>In normal situations, the sending rate of the packets to the
          network is not very high because of the need of a system call for each
          packet. For this reason, the possibility to send a single packet more
          than once with a single write system call has been added. The
          user-level application can set, with an IOCTL call
          (<code class="literal">BIOCSWRITEREP</code>), the number of times a single packet
          will be repeated: for example, if this value is set to 1000, every raw
          packet written by the application on the driver's device file will be
          sent 1000 times.  This feature can be used to generate high speed
          traffic for testing purposes: the overload of context switches is no
          longer present, so performance is remarkably better.</p>
      </div>

      <div class="sect4"><div class="titlepage"><div><div><h5 class="title"><a name="npcap-internals-monitoring"></a>Network monitoring</h5></div></div></div>
        

        <p>Npcap offers a kernel-level programmable monitoring module, able to
          calculate simple statistics on the network traffic.  Statistics can be
          gathered without the need to copy the packets to the application, that
          simply receives and displays the results obtained from the monitoring
          engine.  This allows to avoid great part of the capture overhead in
          terms of memory and CPU clocks.</p>

        <p>The monitoring engine is made of a <span class="emphasis"><em>classifier</em></span>
          followed by a <span class="emphasis"><em>counter</em></span>. The packets are classified
          using the filtering engine of NPF, that provides a configurable way to
          select a subset of the traffic. The data that pass the filter go to the
          counter, that keeps some variables like the number of packets and the
          amount of bytes accepted by the filter and updates them with the data
          of the incoming packets. These variables are passed to the user-level
          application at regular intervals whose period can be configured by the
          user. No buffers are allocated at kernel and user level.</p>
      </div>

    </div>
  </div>
  <div class="sect2"><div class="titlepage"><div><div><h3 class="title"><a name="npcap-internals-references"></a>Further reading</h3></div></div></div>
    
    <p>The structure of NPF and its filtering engine derive directly from
      the one of the BSD Packet Filter (BPF), so if you are interested the
      subject you can read the following papers:</p>
    <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>S. McCanne and V. Jacobson, <a class="ulink" href="ftp://ftp.ee.lbl.gov/papers/bpf-usenix93.ps.Z" target="_top">The BSD Packet
            Filter: A New Architecture for User-level Packet Capture</a>.
          Proceedings of the 1993 Winter USENIX Technical Conference (San
          Diego, CA, Jan.  1993), USENIX.</p>
      </li><li class="listitem"><p>A. Begel, S. McCanne, S.L.Graham, BPF+: <a class="ulink" href="http://www.acm.org/pubs/articles/proceedings/comm/316188/p123-begel/p123-begel.pdf" target="_top">Exploiting
            Global Data-flow Optimization in a Generalized Packet Filter
            Architecture</a>, Proceedings of ACM SIGCOMM '99, pages 123-134,
          Conference on Applications, technologies, architectures, and
          protocols for computer communications, August 30 - September 3, 1999,
          Cambridge, USA</p>
      </li></ul></div>
  </div>

</div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="npcap-tutorial.html">Prev</a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> </td></tr><tr><td width="40%" align="left" valign="top">Npcap Development Tutorial </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> </td></tr></table></div></body></html>