-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

   _   _ _
  | \ | | |_ ___  _ __
  |  \| | __/ _ \| '_ \
  | |\  | || (_) | |_) |
  |_| \_|\__\___/| .__/
                 |_|

             Network Top

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
                        Fully revised for ntop 3.0

       (If you are reading this before 3.0 is really released, e.g.
          3.0pre1 or some such, consider it "Back to the Future")
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Compiled by Burton M. Strauss III <Burton@ntopsupport.com> - comments to him!
In this document, that's whom "I" or "My" refers to.

      |\      _,,,---,,_       "If I purr, you must have fed me.
      /,`.-'`'    -.  ;-;;,_    If the network purrs, it must
    <|,4-  ) )-,_..;\ (  `'-'   be ntop!"   -----Pixel
      '---''(_/--'  `-'\_)

As usual, entries are in no particular sequence.  However, there is an
overall flow, starting with very general questions, then questions about
specific features, then questions about specific platforms.

What structure there is in this FAQ is that of a dialog with a bright 7 year
old.  The response to every answer is "Why".  Keep reading until you get
too detailed for your needs.

If you actually read through this FAQ and don't find yourself saying "Oh,
Bother", at least twice, and then go read Winnie the Pooh.

At the very end are "HowTo Ask For Help" and a "GDB ultraMini-tutorial".
Read them.  Before posting to the mailing list.


Note that some of this FAQ is based on the entries people contributed
to the community FAQ formerly at http://snapshot.ntop.org.  Lots more are 
based on messages to the ntop and ntop-dev mailing lists.

Post 3.0 release entries will have a author/date stamp, at the end:

(Updated/Added ddmmmyyyy by x)


--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Where can I find ntop?
A. The official website can be found at http://www.ntop.org/.

Q. Where do I get the source?
A. SourceForge -- http://sourceforge.net/projects/ntop/

   There is also a cvs (current development) maintained at cvs.ntop.org.
   (instructions are on the download page of www.ntop.org)

     DO NOT (under ANY circumstances) USE THE SOURCE FILES FROM 
     http://snapshot.ntop.org!!!!! They are broken, incomplete and
     WILL NOT WORK. (See 'snapshot, cvs and the Wiki' below)


Q. What is ntop?
A. ntop is an open source network top - it monitors a network and collects
   information about the protocols and hosts for display.


Q. Um, so it's like mrtg (http://people.ee.ethz.ch/~oetiker/webtools/mrtg/)?
A. Yes and no...

   Yes in that both are analyzers of network packets.

   Yes in that both display information about your network.

   No in that they take very different approaches to collecting information.

   No in that they display different types of information.


Q. So mrtg...
A. mrtg creates a picture of the network centered on the device on the link
   between devices (and aggregations of devices and links).

   Tobias (Tobi) Oetiker describe mrtg as:

       "The Multi Router Traffic Grapher (MRTG) is a tool to monitor
        the traffic load on network-links. MRTG generates HTML pages
        containing graphical images which provide a LIVE visual
        representation of this traffic."


Q. And mrtg works how?
A. mrtg reads the counters maintained by various devices such as routers, using
   a protocol called 'snmp' (Simple Network Monitoring Protocol).  The
   management information bases (MIBs) read using snmp, contain incredibly
   detailed information about the packets the device has seen and what it has
   done with them.

   Again, quoting Tobi:

   "MRTG ... uses SNMP to read the traffic counters of your routers and
    ... which logs the traffic data and creates beautiful graphs representing
    the traffic on the monitored network connection."

   mrtg is focused on 'layer 2' (the tcp/ip and other low level protocol).

Q. And ntop?
A. ntop doesn't use snmp - it's not a device centric view of the network.
   Instead ntop actually processes the network packets directly.


Q. What's wrong with snmp?
A. Nothing.

   snmp is a pull protocol, that is a monitoring tool has to pull the
   information from the device.  snmp has 'traps' that is alert messages which
   can be sent out, but the MIB data has to be read by the monitoring tool from
   the device.

   snmp is an older protocol, from the dawn of the network era and it has some
   minor issues of security and complexity and verbosity.  But it's a critical
   network protocol, used successfully by 1000s of ISPs to monitor AND CONTROL
   vast networks.

   From our perspective, the problems with snmp are minor -

       It can use a lot of bandwidth - especially if you're reading from devices
                                       on the far side of slow links.

       Pulling data out of MIBs is a complex process. MIBs can be specified in
       an RFC, or be unique to a vendor/device.

       An snmp-based tool either has to restrict itself to the common MIBs,
       or (most often for vendor tools) it be updated whenever a new device
       must be supported or a MIB changes.

       This makes snmp-based tools complex, because data may be unavailable
       in certain versions of seemingly similar devices, etc.  For example,
       http://www.cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml
       is a page which links to the specific MIBs supported by various
       Cisco devices.


Q. So why hasn't ntop 'won'?
A. mrtg excels at monitoring 100s or 1000s of network devices (routers,
   switches, etc.) and presenting that information over long periods of time.

   ntop doesn't do a good job of showing multiple 'networks' - it's really
   focused on aggregating a picture of a single network.  And for drilling down
   into that picture or presenting it over long periods of time.

   The processing of packets requires a lot more computer resources than just
   reading counters from devices.  On the plus side, this gives much more
   detailed information - for example ntop sees the actual web server request
   instead of just that there was traffic on port 80.  On the minus side, it's
   pretty easy to exceed the processing power of the low end machine typically
   available for ntop.  An ISP using ntop to monitor a couple of T3s needs a
   FAST computer and A LOT of memory.

   ntop also requires access to the physical network (either directly via a
   network card or indirectly via a netFlow/sFlow probe).  This limits ntop's
   (usefullness|ability) to work across sites.

   Once you learn what they do (mrtg and ntop), you'll probably discover that
   You need both.


Q. What's this 'layer' crud?
A. Network layers come from a widely cited but never implemented model, the OSI
   (Open System Interconnect ) networking model from the ISO (International
   Standards Organization).

   Google for it - for example
       http://www.ussg.iu.edu/usail/network/nfs/network_layers.html


Q. So ntop is like netFlow (Cisco), sFlow (RFC 3176, http://www.sflow.org) or
   RMON (HP)?
A. Not really, actually those are all protocols for sending and receiving
   information about the network.

   ntop has lots in common with the tools that USE those protocols.

   And there are lots of tools - some proprietary, many open source.

   The devices/programs that collect the information and send it out in netFlow,
   sFlow or RMON format are usually called (by me) 'probes'.  The devices and/or
   programs that receive the netFlow, sFlow or RMON formatted information and do
   things with it are called 'collectors' (if they process it and forward it on)
   or called 'displays'.

   In fact, ntop can receive netFlow packets and both send and receive sFlow 
   packets.  It can be a 'probe' or a 'display'. (ntop used to be able to send
   netFlow packets - that was removed 2004-03 by Luca).


Q. So ntop is like Nagios or Ipswitch's - WhatsUp Gold?.
A. Nope - those are layer 4 and higher (application) monitoring programs.


Q. So it's like ...
A. Enough already - if you search Freshmeat.net,
   http://freshmeat.net/search/?q=network+monitor&section=projects
   you will find (as of 18Aug2003):


       Topic :: System :: Networking :: Monitoring (654 projects)

   We'll be here all day.  ntop is ntop.


Q. Ok, so ntop is a unique TCP/IP analyzer.
A. Not exactly.

   First off, ntop pretty much doesn't care about the lowest (layer 1 or wire)
   layer.  It leaves dealing with that to a library, libpcap, which hides most
   of that.

   ntop is designed as a hybrid packet analyzer, not a pure Ethernet analyzer
   (layer 2) nor a pure TCP/IP analyzer (layer 3).

   ntop gets the data at the layer 2 (frame) level, which could be Ethernet
   or another protocol.  Beyond Ethernet, ntop has minimal smarts about FDDI,
   PPP, RAW and TOKEN-RING frames.  That is, at least enough for some basic
   counts or to extract the (layer 3) TCP/IP data in side.

   ntop 3.0 adds TWO (three?) HUGE areas of new protocol support.

   ntop 3.0 supports IPv6, thanks to code contributed by Olivier Festor
   <Olivier.Festor@loria.fr> and Abdelkader Lahmadi <Abdelkader.Lahmadi@loria.fr>
   of the MADYNES Research Time (Managing DYnamic NEtworks and Services), see
   http://madynes.loria.fr/.

   ntop 3.0 has more than a fair amount of smarts about FibreChannel and SCSI
   thanks to code contributed by Dinesh G Dutt <ddutt@cisco.com> of Cisco. 

   However, since most of ntop's displayed counts are at the TCP/IP level, it
   confuses people into thinking ntop is purely a TCP/IP analyzer.

   ntop is a traffic monitor with it's own network interfaces, which monitors
   what it sees (or is told about through netFlow or sFlow probes).



Q. ntop runs under?
A. ntop is known to work under Linux, Mac OS X, FreeBSD, OpenBSD (3.4, nothing
   earlier) NetBSD (with issues) and Win32.

   ntop development is done primarily on RedHat Linux and Mac OS.

   Luca also does a port to Win32 (MS Visual C++) and works in the Sun
   Solaris environment.

   I (Burton) usually work under RedHat Linux, but have a multi-boot system
   for testing that runs various Linux distros, FreeBSD (4.9 and 5.2),
   NetBSD, Solaris 8 i86 and OpenBSD.

   During the ntop 3.0 development cycle, we did some development and
   testing on various platforms.  Here's data from the version.xml log
   showing what people were running while testing 2.2.98 and 2.2.99:

          count OS      Version        
        ------- ------- -------------- 
            492	Linux                  
             90	Windows WinNT/2K/XP    
             11	FreeBSD 4.9            
              9	Solaris 9              
              7	Darwin  7.2.0          
              6	FreeBSD 5.2            
              6	FreeBSD 5.1            

        and <5 records for FreeBSD 4.6.2, 4.8, 5.0, 5.2.1; OpenBSD 3.4; 
        and Solaris 8.

   So:

      Solaris 8 / 9 work, 2.6 and 7 probably would...
      FreeBSD 4.6.2/4.8/4.9 and 5.1/5.2
      OpenBSD 3.4

   NetBSD doesn't seem to have been tested.

   Lots of Different Linuxes... of the ones we recognize:

          count Distro    Release
        ------- --------- --------
            141	redhat    9
             42	fedora    1
             39	slackware 9.1.0
             38	redhat    7.3
             26	redhat    8.0
             15	debian          (probably 3.0)
             11	mandrake  9.2
             10	suse      9.0
             10	slackware 9.0.0
              9	redhat    7.2
              8	debian    3.0
              7	redhat    2.1AS
              6	mandrake  9.1
              5	suse      8.2
              5	redhat    3

   and < 5 records for arch 0.6; aurora 1.0; aurox 9.2; fedora 0.95; 
   gentoo 1.4.2.9, 1.4.3.10 and 1.4.3.12; immunix 7.0; mandrake 8.1 and 8.2; 
   pld 1.1; redhat 1, 2.1, 6.2, 7.0, 7.1; slackware 7.1.0, 8.1 and 9.0-beta; 
   suse 6.3, 7.0, 8.0 and 8.1; and yellowdog 3.0.

 Running under pretty much any *nix is at least theoretically possible.
 But it takes interested party/parties and access to resources - some of the
 things that ntop does such as libpcap and loading plugins are tied tighter
 to the OS than you might like.

 There are sections below about each specific OS.


--------------------------------------------------------------------------------
-----Features-------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. What determines the features of ntop?
A. <laugh>Whatever Luca wants</laugh>


Q. Why did you do this "x" instead of feature "y"?
A. Don't know.  I could guess...

   Imagine you are the network manager for a large University network and have
   to crack down on users who are illegally exchanging copyrighted files or
   using University resources to run a business without paying for the resources
   being consumed.

   or

   You are a major vendor of infrastructure, whose customers are using networks
   for new features such as storage area networks and you want to give them
   the ability to monitor these.

   or

   You have a really cool technology that you've just donated to the community
   via an RFC and wish to jump-start adoption of it.

   or

   You are moving heavily into IPv6 and need to be able to monitor your 'new'
   network just like you monitored the IPv4 one.

   or (my favorite)

   You really, really, really hate that ntop generates such lousy html code
   and you decide to scratch that itch.

   or (what should happen)

   A major corporation, with out resources, time, skills and/or inclination
   to do it themselves sponsors the development of a feature that's critical
   to him/her.

   or

   Then again, it could just be because it's cool...


Q. Could ntop do "x"
A. Probably - as long as it doesn't move the tool away from it's purpose and
   it's strengths, almost anything is possible - especially as a plugin.


Q. Will you do "x"
A. Maybe - if it's of interest to a developer, or you provide the code such
   that it can be merged in, or if you're willing to sponsor the development
   effort (contact us through http://www.ntop.org/consultancy.html).


--------------------------------------------------------------------------------
-----Documentation--------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Why isn't there (any)(more)(better) documentation.
A. (A personal peeve from Burton...)

   I get real tired of people complaining that there isn't any documentation
   and then being unwilling to contribute even the simplest stuff.  I've said
   I'll edit and assemble whatever people send me... and since I started working
   with ntop in November 2001, I've received maybe six pages of stuff.

   I'm trying to get people - who aren't coders - to contribute to ntop the
   project.  The contribution that ANYONE can make is "documentation".  A task-
   specific HOWTO... some sample screen shots... An FAQ entry...

   I've tried being nice.
   I've tried asking.
   I've tried shaming people into it.

   What have I gotten?    Zip.

   Nasty is all that's left...  This is your fair warning.  If you show up on
   the ntop mailing lists and complain about documentation, you will get
   blasted.

   -----Burton


Q. Ok, where can I find what does exist.
A. http://www.ntop.org has pointers and some (dated) documents.

   The documentation in the docs/ directory, the Documentation files at SourceForge
   and some at http://www.ntopsupport.com are basically all that there is.

   Search the ntop mailing lists at gmane, http://search.gmane.org. The lists
   are called gmane.linux.ntop.general and gmane.linux.ntop.devel.

   Please contribute to the ntop community by writing things up for inclusion
   in this FAQ or other documents!

   You can even do this all by yourself at the ntop Wiki, 
   http://www.burtonstrauss.com/pmwiki/pmwiki.php?pagename=Ntop.HomePage
   (But note that whether it remains available depends on whether people 
   contribute)


Q. I can't find a file at SourceForge!
A. You can reach the archives through any SourceForge mirror, e.g.

      http://telia.dl.sourceforge.net/sourceforge/ntop/
             ^^^^^  change that qualifier to one that's close to you.

   It has ALL the files unless we explicitly delete them...


--------------------------------------------------------------------------------
-----Problems-------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. I have a problem...
A. Make sure it's a supported release!

   We support only the current versions of ntop.  This is either:

       * the last release, e.g. v3.0.
       * the current cvs (cvs.ntop.org)
       * the latest development version posted at SourceForge (if one)

   If you use a port/package and the latest version available for your
   OS is some release candidate from a year ago, sorry.  Contact the
   packager and ask them to get current.

   Luca usually asks people to try the latest cvs (development) version
   because problems are frequently already fixed in there.


Q. I'm running something supported and I've tried the latest & greatest.
   Still, I have a problem.
A. Read the ntop mailing lists.

   The ntop mailing lists are at
      http://listgateway.unipi.it/mailman/listinfo/ntop
   and
      http://listgateway.unipi.it/mailman/listinfo/ntop-dev

   If you're having non-user problems OR you are using the cvs, you should be
   reading and posting ntop-dev (for example, the cvs commit messages are posted
   there).

   You can read the lists through gmane (or other gateways) if you don't
   want to subscribe, but only subscribers can post.

   You can download the older messages in large chunks from the mailing list
   subscription pages.  Look for "To see the collection of prior postings to
   the list".


Q. I looked and I didn't find my problem.
A. Join the mailing list(s) and ask for help.

   Before you post, check the "HowTo Ask for Help" at the end of this FAQ.

   Please, if at all possible, use the built in PR_ form (the little 'bug' icon
   on the 'About' tab).

   Guidelines for asking questions:

      ONE and only ONE problem / issue / question per message.

      With a meaningful subject.

         The goal is that if you're asking a common question, the
         subject would have allowed you to find it in the back
         traffic for the mailing list.

      Post the information about your environment we ask for.

         We STRONGLY suggest you use the automatically generated "Problem
         Report" form that since it contains much of the necessary information.

      Make sure you're in a supported environment (./configure --showoses).

         If it's an unsupported environment, we're interested in your efforts to
         make ntop work, but we don't have the time, resources, knowledge and/or
         interest to do it ourselves.

      For software 'crashes', please run ntop under the gdb debugger and capture
      the full failure information.

         Brief instructions on using gdb are at the end of this file (docs/FAQ).


Q. I posted to the list and nobody answered me.
A. ntop is open source, and the lists are a community resource.  If nobody
   answered your question, then nobody knew the answers off-hand and nobody
   wanted to spend THEIR time solving YOUR problem.


Q. Do you offer paid support?
A. Yes - contact us through http://www.ntop.org/consultancy.html


--------------------------------------------------------------------------------
-----Configuring ntop-----------------------------------------------------------
--------------------------------------------------------------------------------

Q. What does ./configure do?
A. ./configure checks for the tools installed on your system - configuring
   ntop to compile with the ones you have and skip the ones you don't (or
   to tell you if you're missing something critical).


Q. Why bother - just compile the code.
A. Then you would have to have a machine configured EXACTLY like Luca's.
   Nothing else would work.  Various OSes and Linux distributions package
   the files in different ways and put them in different places.  Plus some
   packages put files into directories with release information in them, etc.


Q. OK, so ./configure
A. Is how you tell ntop where to find things.  A lot of stuff it can figure
   out on it's own, but if things get put in 'strange' places, ntop's
   ./configure has switches you use to tell where to find things.


Q. And the list is?
A. ./configure --help shows the whole list.  It's a bit confusing because
   there are standard options and ntop options mixed in there.

A. So, first let's look at the 'where are things' options.  There are two
   types of files ntop is looking for, '.h' files and libxxxx files.

   .h files are also called 'includes' and libxxxx files are called libraries
   or lib files.

   .h files are the C source for functions provided by the OS or by libraries.
   They are typically in a directory named /usr/include, but they can be
   placed ANYWHERE.

   lib files are compiled libraries of these functions (the .h tells ntop
   how to call something, the lib file is the actual code).  Their names
   usually begin libxxxx (so the library gd is named libgd).  

   By convention, libraries end in .so or .a.  A .so library is a shared
   library (Windows DLL), where one copy of the library is used by all
   programs that want it's functions.  A .a library is a non-shared or
   static library, which must be merged (the technical term is linked)
   with the code.

   ntop uses both - the myrrd library is a static, .a library.  When it
   comes to things like libpcap or libgd, we use shared (.so) libraries.

   Library files are typically placed in /usr/lib, where the gnu linker
   (ld), 'knows' automatically how to find them.  However, from OS to 
   OS and distribution to distribution, there are many other common places.
   Some OSes even have a file telling ld all the places to look.


Q. So ntop looks for these .h and library thingies in a couple of places.
   What if it doesn't find them?
A. If a basic ./configure can't find something, you'll have to tell ntop 
   where to look.

   It's complex and OS/distro dependent. For example, if you install libgd
   from the Sun Freeware site on to a Solaris machine, the files get put
   into /usr/local/include and /usr/local/lib, which are not on the lists
   of 'standard' places for Solaris' versions of gcc (the Gnu c compiler)
   or ld.  So to compile ntop, you have to tell gcc to look in these
   additional locations.

   The things ntop might be looking for are in this part of the
   ./configure --help output:

       +-External-source-locations:-------------------------------------------------+
       --with-pcap-root=DIR        LBNL pcap located in DIR
       --with-pcap-lib=DIR          or libpcap located in DIR
       --with-pcap-include=DIR      or pcap.h located in DIR
       --with-gdbm-root=DIR        gdbm located in DIR
       --with-gdbm-lib=DIR          or libgdbm located in DIR
       --with-gdbm-include=DIR      or gdbm.h located in DIR
       --with-zlib-root=DIR        zlib located in DIR
       --with-zlib-lib=DIR          or libz located in DIR
       --with-zlib-include=DIR      or zlib.h located in DIR
       --with-gd-root=DIR          gd located in DIR
       --with-gd-lib=DIR            or libgd located in DIR
       --with-gd-include=DIR        or gd.h located in DIR
       --with-libpng-root=DIR      libpng located in DIR
       --with-libpng-lib=DIR        or libpng located in DIR
       --with-libpng-include=DIR    or png.h located in DIR
       --with-ossl-root=DIR        openSSL located in DIR
       --with-ossl-lib=DIR          or libssl located in DIR
       --with-ossl-include=DIR      or ssl.h located in DIR
       --with-localedir=DIR        LOCALE files located in DIR (i18n)
       +-----------------------------------------------------------------------+
     
   You can see that there is a pattern, pretty much every --with-xxxxx-root
   has a --with-xxxxx-include and --with-xxxxx-lib option.

   So, if ntop tells you it can't find something, do this - first look for the
   File on your system:

     $ locate pcap.h
     /usr/include/pcap/pcap.h

   (If you don't have locate, this works too:
     $ find / -type f -name "pcap.h"
   )

   And then tell ./configure via --with-pcap-include=/usr/include/pcap

   (Some OSes are smart enough to look in a subdirectory of the standard
   location, but others aren't).

Q. Ok, but why three options?
A. You use either the --with-xxxxx-root option OR either/both of the others
   at a time.  But ntop really only looks at the  --with-xxxxx-include and
   --with-xxxxx-lib options.

   Internally, --with-xxxxx-root=/a/b/c is translated into 
   --with-xxxxx-lib=/a/b/c/lib and --with-xxxxx-include=/a/b/c/include
   (that's the usual pattern).

   Now sometimes libraries are installed logically - if the pcap.h file is in
   /usr/local/pcap/include, the library (libpcap.so) is probably in
   /usr/local/pcap/lib.  Sometimes they are not logical and you will have
   to use the split options.

   The --with-pcap-root=/usr/local/pcap is shorthand for the two options,
   --with-pcap-include=/usr/local/pcap/include and
   --with-pcap-lib=/usr/local/pcap/lib.


Q. Oh Ghu - aren't there any short cuts.
A. For the first time you try ./configure, there's a script on SourceForge in
   The user contributed area that will try to build the ./configure line for
   you.


Q. And the OTHER options?
A. There is a set that tells ntop where to install stuff.  For simplicity,
   the two you might want to change are:

       --prefix=PREFIX         install architecture-independent files in PREFIX
                               [/usr/local]
       --datadir=DIR           read-only architecture-independent data
                               [PREFIX/share]


   --prefix tells ntop where to install the various files.  The default value
   is /usr/local, which is where most non-OS software normal goes.

   A common choice for libraries (such as pcap) is --prefix=/usr, which puts
   things like .h files in places easier to automatically find (/usr/include).
   --prefix=/usr certainly works for ntop.  --prefix=/opt is another choice.

   The 'proper' choice turns on which model of file organization the OS and
   sysadmin prefer.  That's a fight I'm staying out of.

   --datadir tells ntop where to put its databases and output files.  The
   default is /usr/share/ntop, but that will give some sysadmin's agita.
   Another popular choice is --datadir=/var, which puts all the files in
   /var/ntop.  That may be attractive especially if you make /var/ntop
   a separate partition, so the rrd files don't eat all your disk space.


Q. What's --enable-iknowbetter        Override WILLFAIL
A. There are some error messages where it's possible that thing work (now)
   that didn't used to, or you're doing development and don't want ntop
   to stop you from doing something, or there's an error message that you
   have skipped before without getting bitten.

   --enable-iknowbetter will print the message but not stop ntop from
   finishing ./configure.

   Use it at your own risk.


Q. What are the --enable and --disable options?
A. These (and the with/without options) pretty much do what you think - they
   enable or disable large chunks of ntop functionality:

       +--ntop-specific:-------------------------------------------------------+
       --disable-mt                disable multithread support [default=enabled]
       --enable-sslv3              enable ssl v3 support [default=disabled]
       --enable-sslwatchdog        enable Watchdog for ssl hang-ups
                                   [default=disabled]
       --disable-plugins           disable compilation of plugins
                                   [default=enabled]
       --enable-static-plugins     Enable static linked plugins sntop,
                                   default=dynamic]
       --enable-ignoresigpipe      Ignore SIGPIPE errors [default=do not ignore]
       --enable-i18n               Enable (limited) internationalization
                                   [default=disabled]
       --enable-showoses           display OS Support information.
       --disable-ipv6              use IPv6 [default=enabled]
       +-----------------------------------------------------------------------+

       +--external-packages:---------------------------------------------------+
       --without-ssl               disable HTPPS support [default=enabled]
       --without-zlib              disable zlib [default=enabled]
       --with-tcpwrap              enable use of TCP Wrapper [default=disabled]

Q. What ELSE?
A. There are some so called 'environment variables' you can set that change
   things too.  The common ones are:
        CC          C compiler command
        CFLAGS      C compiler flags
        LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
                    nonstandard directory <lib dir>
        CPPFLAGS    C/C++ preprocessor flags, e.g. -I<include dir> if you have
                    headers in a nonstandard directory <include dir>
        CPP         C preprocessor
      
   You would use these variables to override the choices made by ./configure
   or to help it to find libraries and programs with truly nonstandard
   names/locations.

   The best place to look for examples of these environment variables are in
   the OS/distribution files in the configureextra directory.  For example,
   (again picking on Solaris), we use LDFLAGS to tell ld to look in some
   common Solaris locations for libraries via this:

     LDFLAGS="-L/usr/local/lib -R/usr/local/lib ${LDFLAGS}"

   What that does is add /usr/local/lib to the locations that ld will check.


Q. ./configure dies in some strange horrible way complaining about version
   conflicts, lunar phase or whatever.
A. The process of creating portable cross-platform scripts for building
   software is ugly and hard and prone to failure.  There are tools
   (released by gnu) to help, named automake, autoconf and libtool.
   (Collectively the auto* tools).

   These tools take basic files and generate more complex files based
   on a series of rules, conventions and macros (much of which is poorly
   or undocumented).  Other processes (e.g. ./configure) take these files
   and generate more complex files based on a different series of rules,
   conventions and macros.  This ultimately produces the 'Makefile', which
   a program called make uses - based on (...wait for it...) yet another
   series of rules, conventions and macros - to actually compile the ntop
   source.

   There are interdependencies among the tools, partial support for older
   versions in some releases, but not in other releases and many more
   problems.

   Different OSes and different Linux distros ship with wildly different
   versions.  Some even have scripts that attempt to analyze the file and
   pick the correct version (which just means that trying to code a file
   with multiple version support confuses the tool).

   So if you have a basic, total failure of ./configure, it's usually
   the auto* tools.

   ntop used to ship with a manual script that rebuilt the generated files
   according to the version(s) of the tools installed on the system you were
   using to build ntop.  Thus the standard answer was 'run ./autogen.sh -1'.

   But, this meant that you had to have these 'developer' tools installed
   and caused much problems and gnashing of teeth.

   So we spent 100s of hours rebuilding the scripts to be totally independent
   of having the tools installed on your system - only to run into problems
   because the xyz 1.6.1 tool installed by default on OS A isn't quite
   compatible with the 1.6.3 version on OS B.

   So we put technology in place to automatically detect tool versions and
   rebuild the generated files if necessary.  That meant you had to have
   these 'developer' tools installed and caused much problems.

   So we rebuilt the scripts AGAIN and AGAIN, dropped support for old versions
   of the tools and finally reached a point where it works for most reasonably
   current platforms.  This is a compromise:

       Systems that don't have the tools installed usually work.
       Systems that have bleeding edge versions of these tools may break.
       Systems with very old versions also may break.

   The technology to detect versions and rebuild the script files if
   appropriate is still in there, but it's disabled from normal use.

   If you have the auto* tools installed and have ./configure problems, you
   can activate the automatic rebuild feature via:

      $ export NTOPAUTOREBUILD=yes
      $ ./configure ...

   If the rebuild fixes it, that's great.  Regardless, please report the
   problem to the ntop mailing list.  Please don't paraphrase the messages
   cut & paste the ACTUAL MESSAGES into your report.  Also, please let us
   know the version(s) of the tools installed on your system:

      $ aclocal --version
      $ autoheader --version
      $ autoconf --version
      $ automake --version
      $ libtool --version

Q. But how does it all hang together?
A. There's a vsd (Visio)/pdf in the docs directory - ntop-autotools.pdf.


--------------------------------------------------------------------------------
-----Compiling ntop-------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Which packages/libraries do I need to compile ntop:

    Note: In some cases the minimal header files for a tool will be in
          one "package" and the execution library in another. ntop needs
          both so that the ./configure test finds the tool. It's usually
          safest to install both the tool and development packages!

         (Some packages will have additional packages as pre-requisites)

       gdbm (We have people using 1.7.3, 1.8.0, 1.8.2 and 1.8.3)

       libpcap
          We recommend 0.7.2, but ntop works with most 0.6.x versions. 
          The newest version, 0.8.x works too - there are ntop fixes since 
          late 2.2.9x releases for this.

          STAY AWAY from older versions, especially under Linux.

          Note: Building libpcap requires: bison/flex


       gd (1.8.x - stay away from 2.x development versions)
          http://www.boutell.com/gd/gd.html

       libpng (1.2.x - although ntop works with 1.0.x versions, the two libpng
              versions are not compatible and will often cause version conflicts
              and crashes).

       glibc

       cpp

       gcc
          Note: Both gcc 2.95/2.96 and 3.2/3.3 are reported to work.  Across a lot
                of software, gcc 3.0 had problems - no clue re ntop.
          For you folks still running 2.94, I think I fixed all the compile errors,
          but upgrade for ghu's sake!

       libtool  1.4+ (distribution with 1.4.2)
                     (there are successes reported with 1.3.4 or 1.3.5 -
                      use --enable-iknowbetter)

       gawk/mawk
          Note: If all you have is nawk, then --enable-showoses will not work.
                If all you have is original awk, then ./configure will not work.

Q. Will the "xyz" compiler work instead of gcc?
A. We explicitly REQUIRE gcc.  

   We know that under many systems, a compiler called cc is available. On some,
   it's a symbolic link to gcc, BUT, when invoked as cc, it often triggers 'old'
   behaviors for cc compatibility.  On others, cc is a retarded compiler just good
   enough for compiling the kernel.

   The one exception is Sun's cc, which - since Luca used to do a lot of development
   on Solaris - was pretty well supported.


Q. The auto* tools?
A. Only if you are going to rebuild the distributed script files:

           autoconf 2.5+ (distribution with 2.57)
           automake 1.6+ (distribution with 1.6.1)


Q. Optional libraries?
A. Lots.  The most common is:

       openssl (for https:// support)
          Note: For security reasons you should only have the most current
                Version of openssl installed.


Q. ntop doesn't compile.
A. Sure, it does - for me :-)


Q. Right smarty-pants...
A. First, check if you're using a supported OS.

   The long and short of it is that ntop works under Linux, Mac OS X and
   FreeBSD.

   Luca distributes a Win32 version through http://shop.ntop.org but charges a
   convenience fee.  Or you can fire up Microsoft's Visual C++ 6.0 and compile
   the Win32 version that way.  We don't recommend Visual C++ .Net as the
   internals of the memory management caused problems.

   It works (albeit with severe limitations) on NetBSD.

   It works on OpenBSD 3.4 (only).

   It used to work under MinGW, but that broke after 2.1.3 and we need somebody
   with some technical skills and an interest in MinGW to fix it.

   Anything else, you are in untested waters.  Some cases we know there are
   problems, others we just haven't tested.


Q. But that was as of 18Aug2003?
A. Yes.  For the latest scoop, run ./configure --enable-showoses

   ntop will perform the basic ./configure tests (make sure you have a c
   compiler that works, etc.) and then output a table.

   This is from a pretty complete 3.0 release (at least as far as OS support
   goes):

     ntop OS Support status extract for version 2.2.91
         Run Mon Aug 18 10:00:58 CDT 2003

     config.guess value     compiler OS name    Status
     ------------------    --------- ---------  ---------------
     mingw32*                    gcc MINGW             UNTESTED
     cygwin*                       * CYGWIN            WILLFAIL
     linux*                     *gcc LINUX      SUPPORTED
     solaris2.9*                   * SOLARIS           UNTESTED
     solaris2.8*                   * SOLARIS    SUPPORTED
     solaris2.5*                   * SOLARIS           UNTESTED
     solaris*                      * SOLARIS           UNTESTED
     darwin*                       * DARWIN     SUPPORTED
     freebsd[45]*               *gcc FREEBSD    SUPPORTED
     freebsd*                      * FREEBSD           UNTESTED
     openbsd*                      * OPENBSD           WILLFAIL
     netbsd*1.5*                 gcc NETBSD
     netbsd*                       * NETBSD            UNTESTED
     hpux11*                     gcc HPUX              WILLFAIL
     hpux10*                     gcc HPUX
     aix*                          * AIX               UNTESTED

   Quoting from the ./configure --enable-showoses output:

     Status of 'SUPPORTED' means you are:

          Building ntop for a supported platform
          This means we expect ntop to work without major issues


     Status of 'UNTESTED' means you are:

          Building ntop for an untested platform

     Untested means that:

          1. A previous version of this OS is supported.
          or 2. This version of this OS was supported by a
          previous version of ntop.

          ./configure applies the same configuration options,
          special parameters, etc. as for the prior release.

          For OS .0 releases that is probably a bad bet...

          It could just be that you are compiling an older version
          of ntop on a new OS release - check the http://www.ntop.org
          site for an update.

     Status of 'WILLFAIL' means you are:

          Attempting to build ntop for an unsupported platform

          >>> This is unsupported.
          >>> It will almost certainly fail
          (that is why it is listed as 'unsupported' - doh!)

     Status of blank means that the support level depends
     upon ntop settings and/or other factors.


Q. Ok, it's a supported release and it still won't compile.
A. Did you run ./configure?  And did it complete successfully?

   Usually 'compile' problems for supported platforms are a missing
   (critical) library or header file, but the user ignored (didn't see)
   the error/warning message and tried running make anyway.

   ./configure checks a lot of things.  When it's looking for
   headers and libraries, ntop will report KEY information and PROBLEMS
   in large, set-off, lines:

     *******************************************************************
     *
     * NOTE: Building ntop for a supported platform
     *       This means we expect ntop to work without major issues
     *
     *            'i686-pc-linux-gnu'
     *
     *    Please keep the ntop-dev mailing list updated with any
     *    successes you have or problems you encounter...
     *
     *   Support for this platform was most recently verified for
     *
     *     RedHat7.2 w/ updates           ntop 2.1.51   on 2002-10-21
     *     Suse i686, 2.4.18-4GB-SMP      ntop 2.1.51   on 2002-10-24
     *
     *******************************************************************

   READ THESE BOXES.  Even if you don't read the rest of the output, read
   the boxes.  ntop can work around a lot of problems (missing libraries)
   by disabling features that need them.  If, for example, you don't have
   zlib, ntop will compile a version that doesn't output compressed html
   pages.  If you don't read the boxes, you will never know.

   READ THE MESSAGE BOXES!

     *******************************************************************
     *                                                                 *
     * NOTICE:  I know you're used to ignoring output from ./configure *
     *                                                                 *
     *          ntop has a lot of complexity and interdependences.     *
     *                                                                 *
     *          Please, please AT LEAST read the stuff in these        *
     *          boxes!                                                 *
     *                                                                 *
     *>>> The ACTION taken is shown prefixed '>>>'                     *
     *                                                                 *
     *    If the ACTION is unacceptable,                               *
     *??? The REMEDIATION plan is shown prefixed with '???'            *
     *                                                                 *
     *******************************************************************

   The box will tell you what's wrong, what ntop did and often how to fix it
   if you don't like ntop's fix.

   READ THE MESSAGE BOXES!

   Hint: It may sometimes be that you're missing the header files (often those
   are in a -devel rpm if you're running RedHat)

   If you see a message box and don't understand why ("I'm sure that
   header file is present"), then look at a file called config.log.  Search
   for the specific header or library reported in the message box and you will
   see the detailed compiler/linker error messages.

   For example, ./configure reports:

        checking for linux/if_pppox.h... no

   The first thing to so is check if it's on your system:

     $ locate linux/if_pppox.h
     /usr/include/linux/if_pppox.h

   (If you don't have locate, this works too:
     $ find / -type f -name "ethertype.h"
   )

   Open up config.log and look for if_pppox.h:

     configure:13086: checking for linux/if_pppox.h
     configure:13103: gcc -c -g -O2 -Wshadow -Wpointer-arith
      -Wmissing-prototypes -Wmissing-declarations -Wnested-externs  -fPIC
      -DLINUX conftest.c >&5
     configure:13158: parse error before '/' token
     In file included from configure:13160:
     /usr/include/linux/if_pppox.h:38: `ETH_ALEN' undeclared here (not in a
         function)
     /usr/include/linux/if_pppox.h:39: `IFNAMSIZ' undeclared here (not in a
         function)
     /usr/include/linux/if_pppox.h:40: confused by earlier errors, bailing out
     configure:13106: $? = 1
     configure: failed program was:
     | #line 13091 "configure"
     | /* confdefs.h.  */
     |
     | #define PACKAGE_NAME "ntop"
     | #define PACKAGE_TARNAME "ntop"
     | #define PACKAGE_VERSION "2.2.91"
     | #define PACKAGE_STRING "ntop 2.2.91"
     | #define PACKAGE_BUGREPORT ""
     | #define PACKAGE "ntop"
     | #define VERSION "2.2.91"
     | #define STDC_HEADERS 1
     ...
     | #define HAVE_NETINET_UDP_H 1
     | /* end confdefs.h.  */
     | net/if.h netinet/if_ether.h
     |
     | #include <linux/if_pppox.h>
     configure:13123: result: no

   You see the actual test program, the actual compile line and the error
   messages.

   Before reporting it to us, chase down where those missing items are declared:

     $ grep -R 'ETH_ALEN' /usr/include/* | grep '#define'
     /usr/include/linux/if_ether.h:#define ETH_ALEN  6
     /usr/include/net/ethernet.h:#define     ETHER_ADDR_LEN  ETH_ALEN

   And post that information with your error report.  The reason is that these
   field definitions are often placed in very different places in different
   OSes and even in different distributions.

   FWIW, if you look in the older configure.in:

      AC_CHECK_HEADERS([linux/if_pppox.h], [], [], [net/if.h
         netinet/if_ether.h])

   should have been

      AC_CHECK_HEADERS([linux/if_pppox.h], [], [], [#include <net/if.h>
      #include <netinet/if_ether.h>])

   because the macro doesn't do the #include automatically.


Q. Nope, it's not ./configure...
A. If it's not the configuration, then it's usually a problem with your specific
   system, either:

        - A new release of a supported OS.
        - An uncommon option/configuration of a supported OS.

   In other words, something is different from what we've seen or expected.

   Review the output from make.  The error message will usually give you a
   Somewhat cryptic description of what's wrong and where.  Look for messages
   about missing files.  Post as much information as you can - do locates for
   the missing files, etc.  The more you give us the less we will have to ask
   you to provide.

   Remember, we can't see your box - all that the people on the list see is the
   information you give in your message.


Q. Compile dies because it's missing depcomp
A. automake/autoconf issue.  The problem should have been fixed. If not, just
   Copy the missing file (or make a symbolic link) into the ntop source
   directory.

   It's in /usr/share/automake on my Linux boxes.  Another user reports it is in
   /usr/local/share/automake in sun8.

   If you have automake installed, this will do it automatically:

       $ automake --add-missing --gnu -c


Q. Make fails with a message about being unable to create a .deps file.
A. Check the permissions on the (hidden) .deps (and .libs) directories - if root
   owns them your non-root userid may not be able to create files in there.


Q. Make fails with a message about a missing .deps file
A. Basically, it's a automake 1.5 bug, related to dependency tracking.

   If you don't have automake installed, you shouldn't see this problem.

   ntop requires automake 1.6+ - that dependency is EXPLICT in the Makefile.am!

   If you do have automake installed, it's possible to hit this problem -
   especially if it's one of those Linux distributions that have three versions
   of automake installed and a (broken) script that's supposed to figure out
   which version to actually run.

   Since we distribute ntop with scripts generated from 1.6.3, you would *think*
   they should work, regardless of what version of automake is installed.

   That's not the case.  The problem occurs because automake gets invoked by
   ./configure to copy the missing gnu files such as depcomp.  If you have 1.5
   installed, it then remakes the plugins/Makefile as a 1.5 version, which
   fails.

   The problem should be trapped and worked around, however, so report the
   problem to the list.


Q. Why is the .deps problem mostly happening under FreeBSD?
A. Because the FreeBSD ports tree only has 1.5, but that's a FreeBSD ports
   problem, not ntop's.  If you search the FreeBSD lists on Google, there's lots
   of traffic about a 1.6 version for FreeBSD, but it doesn't seem to be in the
   tree.  What's there is:

     ./devel/automake
        -- which is 1.5
     ./devel/automake14
     ./devel/automake17
        -- which does NOT work


Q. So how do I work around the problem?
A. Install 1.6.3.  It's quite easy, does NOT require root.  The steps are listed
   in the ./configure message, repeated below:

      Download automake 1.6.3 from gnu
          $ wget http://ftp.gnu.org/gnu/automake/automake-1.6.3.tar.gz

      Untar it
          $ tar xfvz automake-1.6.3.tar.gz

      Make it
          $ cd automake-1.6.3
          $ ./configure --prefix=/home/<whatever>/automake163
          $ make
          $ make install

      Add it to your path (this is bash, but other shells, can do it too)

          $ PATH=/home/<whatever>automake163/bin:$PATH
          $ export PATH

      And then untar, ./configure and make ntop.


Q. Make fails with a message like this:
     /bin/ld: Warning: size of symbol `pcap_open_dead' changed from 100 to 67
         in pcap.o
     collect2: ld returned 1 exit status
     make[2]: *** [libntop.la] Error 1
     make[2]: Leaving directory `/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3'
     make[1]: *** [all-recursive] Error 1
     make[1]: Leaving directory `/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3'
     make: *** [all] Error 2
A. Ah yes, size of symbol...changed.  It means you have a conflict between 
   the version of the shared library (be it libpcap, libgd, whatever) that
   was used to compile ntop with the version that was used to link ntop.

Q. 'splain please...
A. If you break down a typical link line:

      gcc -shared  address.lo ... ntop_darwin.lo
          -L/usr/local/include
          -L/usr/local/lib
          -L/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3/myrrd
          -lpthread -lresolv -lnsl -lz -lc -lssl
          -lcrypto -lgd -lpng -lpcap /usr/lib/libgdbm.so
          -lmyrrd
          -Wl,-soname -Wl,libntop-2.2.3.so
          -o .libs/libntop-2.2.3.so

   You see the mix of -L and -l parameters.  The -L parameters ADD 
   additional places to look for the shared libraries, which are in 
   addition to the 'standard locations' for the system.  The -l
   parameters tell which libraries to include.

   Read the man pages (man ld, man ld.so, etc.)

   The 'standard locations' are very system dependent, but usually
   include /usr/lib and /lib.  PLUS whatever is (under Linux) in the
   ld.so.conf file, for example

      $ cat /etc/ld.so.conf
      /usr/kerberos/lib
      /usr/X11R6/lib
      /usr/lib/qt-3.0.5/lib
      /usr/local/lib

   So, on this system, to resolve a library, ld looks in the -L values:

      1. /usr/local/include
      2. /usr/local/lib
      3. /linuxadmin/ntop/ntop2.2.3/ntop-2.2.3/myrrd

      And then the 'standard' places:

      4. /usr/kerberos/lib
      5. /usr/X11R6/lib
      6. /usr/lib/qt-3.0.5/lib
      7. /usr/local/lib
      8. /lib
      9. /usr/lib

   Similarly, if you break apart the gcc COMPILE line and scrap the dups,
   you'll have a different set of places where gcc looks for the .h files.
   For those, it's the -I parameters plus whatever is 'standard' on your
   system, which is dependent on the specific gcc port:

   /bin/sh ./libtool --mode=compile gcc -DHAVE_CONFIG_H  -DLINUX
       -I. -I/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3/myrrd 
       -I/usr/local/include
      -g -O2 -fPIC -g -O2 -c
      -Wshadow ... -Wnested-externs
       -o ntop_darwin.lo

   Again, read the gcc man page.

   A big problem is that - unlike .so files, it's not real clear what
   directories ARE searched for .h files, only that there is a set.

   With me so far?  Go back to the message... what it means is that:

      * From one set of locations, at compile time, the size of 
        the parameter list for pcap_open_dead was 100 bytes.
      * From the other set of locations, at link (ld) time, the
        library expects the parameter list to be of size 67.

   Danger, Will Robinson...

   The most likely cause of this problem is when you tell ntop to look at
   one location to resolve either the .h or .so and it finds the other,
   'automatically' in a 'standard' location, but the two are actually from
   incompatible versions.

   Say you have libpcap 0.6.2 installed by the os in /usr/lib and 
   /usr/include, but you also install libpcap 0.7.2 in /usr/local/lib and 
   /usr/local/include.

      $ locate pcap.h
      /usr/include/pcap.h
      /usr/local/include/pcap.h

   and

      $ locate libpcap.so
      /usr/lib/libpcap.so.0
      /usr/lib/libpcap.so
      /usr/lib/libpcap.so.0.6.2
      /usr/lib/libpcap.so.0.6
      /usr/local/lib/libpcap.so.0
      /usr/local/lib/libpcap.so
      /usr/local/lib/libpcap.so.0.7.2
      /usr/local/lib/libpcap.so.0.7

   If all you give ./configure is --with-pcap-header=/usr/local/lib

      * At compile time, it finds the pcap.h in /usr/local/lib (0.7.2)
      * At link time, it finds the libpcap.so in the 'default', 
        /usr/lib/libpcap.so which is actually libpcap.so.0.6.2


   ntop did EXACTLY what you told it to do.  The fact that it makes no
   sense is a problem, so you get the error message.

Q. So the real solution is?
A. Give ntop pointers to consistent sets of header and library files, and maybe
   don't have multiple versions of the same library installed at once.


Q. I'm done compiling and it works/doesn't work, what do I do?
A. make install (You'll typically need to be root for this to work)

Q. Anything else?
A. Yes.  Run the "census" - this will send us an email telling us what's 
   working and what's not working.  The census sends me the basics - uname 
   information, autotool versions, version.c.  The domain name is blinded, 
   and the script will show you the file before it's sent (you'll have 10 
   seconds to press control-c to abort).

   If things work ok, then send the census information via

    make census-ok

   If you have problems, the create the census information via

    make census-fail

   Edit it to describe the problem and send it to census@ntopsupport.com


Q. I'm seeing a lot of "changing search order for system directory" messages?
A. It's not a problem, just confusing.  This warning, which appears if you 
   redefine the system include directory, causes lots of confusion.  Supposedly,
   according to this http://gcc.gnu.org/ml/gcc-bugs/2002-10/msg01080.html,
   it wasn't fixed in gcc 3.2.0, but should be fixed in gcc 3.2.1.

   According to this: http://gcc.gnu.org/onlinedocs/cpp/Invocation.html,
   it's truly harmless on systems which already have /usr/local/include as
   a standard system library:

      -I dir 
      Add the directory dir to the list of directories to be searched for
      header files. See Search Path. Directories named by -I are searched
      before the standard system include directories. If the directory dir
      is a standard system include directory, the option is ignored to ensure
      that the default search order for system directories and the special
      treatment of system headers are not defeated (see System Headers) . 


--------------------------------------------------------------------------------
-----Other ./configure features-------------------------------------------------
--------------------------------------------------------------------------------

Q. How do I update the Vendor Table (MAC address prefixes)?
A. ntop has (in Makefile), a rule to automatically download the latest vendor
   information table from the IEEE, the oui.txt file ntop reads.

   If you are seeing unknown MAC address prefixes (the 1st three units), try
   the full IEEE table.  To rebuild it:

     # make dnvt

   and then copy the new oui.txt over the one installed by ntop originally.

   Also note that the table changes over time - there are almost 600
   Modifications and/or new assignments between the version shipped with
   ntop 2.0 and the version on the IEEE site in February 2002.


Q. How do I update the passive ethernet fingerprint database?
A. ntop has (in Makefile), a rule to automatically download the latest
   ettercap file from SourceForge:

     # make dnetter


   It will also compress the file and tell you how big the old and new files
   were.


Q. How do I update the IP to Country Code table?
A. Again, ntop has (in Makefile), a rule to automatically download the
   latest data and build the file.

     # make p2c

   Note that this is actually a fairly complex script (utils/p2c) and is
   dependent upon data posted to supposedly well known locations by the various
   internet number authorities (APNIC, RIPE, etc.)

   It's a LOT of data to download so don't do this on a site with a slow
   link to the internet.  Also, read the output carefully and DO NOT apply a
   file if there are messages you don't understand.


Q. But the IP -> CC data is wrong.
A. Yeah.  Most people don't care :-(... it's enough to see various pretty
   flags.

   Sadly, the registries often post files with errors in them.  Some Tier
   1 ISPs post their own data to elaborate on the registry data.  Most ISPs
   don't post data.  Some entries in the files make 'logical' sense, but not
   physical sense.

   If you find another set of data, let us know - the shell script is pretty
   easy to follow to add another data source.

   The user who used to care about this (Mr. Anon E. Mouse) used to send me
   a file of corrections to apply to the file we posted with ntop.

   If such a file exists, it would be posted @ SourceForge in the user
   contributed area.


--------------------------------------------------------------------------------
-----Running - Startup----------------------------------------------------------
--------------------------------------------------------------------------------

Q. ntop won't start - I get this message:
  ** FATAL ERROR** ... open of /var/ntop/addressQueue.db failed:
       File open error
A. It's either permissions (the ntop -u userid doesn't have read/write access
   to that file - common if you're upgrading from 2.2 to 3.0).

   The other cause is that there's still an instance of ntop running. Make
   sure you shutdown ntop before starting it.


Q. What is the function of the 'ntop' script in the build directory - should I
   call it or /usr/local/bin/ntop ?
A. (from the comments in the script):

    # ntop - temporary wrapper script for .libs/ntop
    # Generated by ltmain.sh - GNU libtool 1.4 (1.920 2001/04/24 23:26:18)
    #
    # The ntop program cannot be directly executed until all the libtool
    # libraries that it depends on are installed.
    #
    # This wrapper script should never be moved out of the build directory.
    # If it is, it will not operate correctly.

   It allows you to run ntop out of the build directory before doing a "make
   install" by doing all the necessary linkage magic - such as forcing a relink
   if it didn't succeed originally - to the files in .libs.

   Think of it as simulating make install, but not moving stuff to /usr/local
   or wherever.

   Don't use it - it just causes problems...


Q. Which libraries do I need?
A. To run ntop: glibc, gdbm, libpcap
      For https://, add openssl.
      For other tools and compile options, add the appropriate libraries.


Q. ntop seems to run, but the web server isn't up.
A. Set the password - see docs/1STRUN.TXT


Q. How do you reset Admin password if we lost it?
A. Delete ntop_pw.db and follow the procedure in docs/1STRUN.txt


Q. I'm being prompted to set a password, what do I enter.
A. Anything you want, of 5 or more characters.  If ntop can't find the value
   in it's database, it will prompt you to set on.  If this happens every
   time you start ntop, check the permissions on the ntop_pw.db file.


Q. How does the @filename option work e.g. /usr/bin/ntop ... @filename ...
A. The text of 'filename', is copied - ignoring line breaks and
   comments (anything following a #) - into the command line.

   ntop behaves as if all of the text had simply been typed directly 
   on the command line.  Multiple @s are permitted in the command
   line, nesting is not.  @s in the file will cause an error.

   Both are displayed on the info.html report, the "Started as"
   shows the actual command line ntop was given and the 
   "Resolved to" shows what ntop processed.

   Started as /usr/bin/ntop -i eth0,eth1 @/root/ntop_parms -d -L

   with /root/ntop_parms containing:

      -p /usr/share/ntop/protocol.list
      -P /usr/share/ntop
      -w 192.168.42.38:3000
      # -W 192.168.42.38:3001
      -u ntop
      --trace-level 3
      -m 12.239.0.0/16,10.113.0.0/16
      -E
      -K

   becomes:

   Resolved to /usr/bin/ntop -i eth0,eth1 -p /usr/share/ntop/protocol.list
               -P /usr/share/ntop -w 192.168.42.38 -u ntop --trace-level 3
               -m 12.239.0.0/16,10.113.0.0/16 -E -K -d -L

   Remember, most ntop options are "sticky", that is they just set an
   internal flag. Invoking them multiple times doesn't change ntop's
   behavior. However, options that set a value, such as --trace-level,
   will use the LAST value given: --trace-level 2 --trace-level 3 will
   run as --trace-level 3.

   It is recommended that you use FULL pathnames for @filename, since
   ntop may have different effective directories when run in different
   ways.  However, you may wish to use relative pathnames to take
   advantage of the different effective directories (say cron vs.
   command line).  Just know where you're starting.


Q. ntop seems to run but I don't see any traffic.
A. Make sure you aren't running against the loopback (127.0.0.1) interface.
   lo shouldn't see much traffic, only that originating on the host destined
   for it (e.g. ping 127.0.0.1).


Q. ntop is unable to open its database file.  Specifically:
   I have following messages while running ntop

     wait please: ntop is coming up...
     24/Jul/2003 15:15:23 Initializing IP services...
     <snip />
     24/Jul/2003 15:15:23 Initializing GDBM...
     24/Jul/2003 15:15:23 Database '/var/ntop/addressCache.db' open failed: File
         open error
     24/Jul/2003 15:15:23 Possible solution: please use '-P <directory>'
A. Multiple possible choices...
    1. The directory /var/ntop doesn't exist.  Create it or, as the message
       says, use the -P parameter to point ntop at another directory.
    2. You many not have read/write rights in /var/ntop - if you're running in
       non-promiscuous mode from a user other than root.
    3. Another instance of ntop may already be running, so it has the file open
       and locked.

Q. What's inside the .db files and what's their format?
A. The files are stored in GDBM format. You can dump their content using tools like
   this: http://tboudet.free.fr/dumpgdbm/. In principle you should not be
   interested in the file content as they are temporary caches and contain the
   configuration as well as the MD5 hash of the web passwords.

   I posted a trivial dumpgdbm.c to gmane - search for it.

  
Q. ntop starts up with this:
   WARNING: Discarded network 172.20.0.0/16: this is the local network.
A. No worries.  The message means exactly what it says - it's a warning that
   you gave the local network as one of the parameter(s) to -m.  Since the
   local networks are always local, ntop doesn't need to make them pseudo-local.


Q. What are ntop's options?
A. There are a couple of options that appear only if they're not compiled in,
   and a few that depend on various external libraries, e.g. openSSL.

   The best way to see what is actually available is to run ntop with the -h or
   --help options and see.   Also read man ntop.

   Here is the FULL set of real, working options as of February 2004

    -4             | --ipv4
                             Use IPv4 connections (for the ntop web server)
    -6             | --ipv6
                             Use IPv6 connections (for the ntop web server)

    -a <file>      | --access-log-file <file>
                             File for ntop web server access log

    -b             | --disable-decoders
                             Disable protocol decoders

    -c             | --sticky-hosts
                             Idle hosts are not purged from memory

    -d             | --daemon
                             Run ntop in daemon mode

    -e <number>    | --max-table-rows <number>
                             Maximum number of table rows to report

    -f <file>      | --traffic-dump-file <file>
                             Traffic dump file (see tcpdump)

    -g             | --track-local-hosts
                             Track only local hosts

    -h             | --help
                             Display help and exit

    -i <name>      | --interface <name>
                             Interface name or names to monitor

    -j             | --create-other-packets
                             Create file ntop-other-pkts.XXX.pcap file

    -o             | --no-mac
                             ntop will trust just IP addresses (no MACs)

    -k             | --filter-expression-in-extra-frame
                             Show kernel filter expression in extra frame

    -l <path>      | --pcap-log <path>
                             Dump packets captured to a file (debug only!)

    -m <addresses> | --local-subnets <addresses>
                             Local subnetwork(s) (see man page)

    -n             | --numeric-ip-addresses
                             Numeric IP addresses - no DNS resolution

    -p <list>      | --protocols <list>
                             List of IP protocols to monitor (see man page)

    -q             | --create-suspicious-packets
                             Create file ntop-suspicious-pkts.XXX.pcap file

    -r <number>    | --refresh-time <number>
                             Refresh time in seconds, default is 120

    -s             | --no-promiscuous
                             Disable promiscuous mode

    -t <number>    | --trace-level <number>
                             Trace level 0-6

    -u <user>      | --user <user>
                             Userid/name to run ntop under (see man page)

    -x <max num hash entries>
                             Max num. hash entries ntop can handle (default 4294967295)

    -w <port>      | --http-server <port>
                             Web server (http:) port (or address:port) to listen on

    -z             | --disable-sessions
                             Disable TCP session tracking

    -A
                             Ask admin user password and exit

    --set-admin-password=<pass>
                             Set password for the admin user to <pass>

    --w3c
                             Add extra headers to make better html

    -B <filter>    | --filter-expression
                             Packet filter expression, like tcpdump

    -D <name>      | --domain <name>
                             Internet domain name

    -F <spec>      | --flow-spec <specs>
                             Flow specs (see man page)

    -K             | --enable-debug
                             Enable debug mode

    -L
                             Do logging via syslog

    --use-syslog=<facility>
                             Do logging via syslog, facility ('=' is REQUIRED)

    -M             | --no-interface-merge
                             Don't merge network interfaces (see man page)

    -N             | --wwn-map
                             Map file providing map of WWN to FCID/VSAN

    -O <path>      | --pcap-file-path <path>
                             Path for log files in pcap format

    -P <path>      | --db-file-path <path>
                             Path for ntop internal database files

    -U <URL>       | --mapper <URL>
                             URL (mapper.pl) for displaying host location

    -V             | --version
                             Output version information and exit

    -X <max num TCP sessions>
                             Max num. TCP sessions ntop can handle (default 4294967295)

    -W <port>      | --https-server <port>
                             Web server (https:) port (or address:port) to listen on

    --disable-instantsessionpurge
                             Disable instant FIN session purge

    --disable-mutexextrainfo
                             Disable extra mutex info

    --disable-schedyield
                             Turn off sched_yield() calls, if ntop is deadlocking on them

    --disable-stopcap
                             Capture packets even if there's no memory left

    --fc-only
                             Display only Fibre Channel statistics

    --no-fc
                             Disable processing & Display of Fibre Channel

    --no-invalid-lun
                             Don't display Invalid LUN information

    --p3p-cp
                             Set return value for p3p compact policy, header

    --p3p-uri
                             Set return value for p3p policyref header

    --set-pcap-nonblocking
                             Call pcap_setnonblock

    --skip-version-check
                             Skip ntop version check

    --ssl-watchdog
                             Use ssl watchdog (NS6 problem)


Q. -4 and -6?
A. The default for the ntop web server is to connect to any address and any family
   of addresses, so if the NIC has both IPv4 and IPv6 addresses it should respond 
   to both.  Use these to restrict the web server.


Q. --trace-level?
A. The settings 0..4 control the amount of information logged or displayed.
   5 adds a [MSGIDnnnn] field and 6 adds the file:line where the message originated.
   -t 5 can be useful with a log watch type package, -t 6 is mostly for debugging.

   -t 5:
      [MSGID8757584] OSFP: scanFingerprintLoop() checked 1, resolved 1

   -t 6:
      [MSGID8757584] [ntop:698] OSFP: scanFingerprintLoop() checked 1, resolved 1


Q. ntop starts up find and then seems to die.
A. Are the ntop threads still there?  Use ps to check, for example:
   # ps -C ntop -m -o "uid,pid,ppid,stat,time,command"

     UID   PID  PPID STAT     TIME COMMAND
     501 18327     1 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18330 18327 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18331 18330 S    00:00:05 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18332 18330 S    00:00:45 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18333 18330 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18334 18330 S    00:00:06 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18335 18330 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18336 18330 R    00:00:32 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18337 18330 S    00:00:41 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...

   The 'PID' numbers match the THREADMGMT: numbers in the log:

      ntop[18326]:   INIT: Parent process is exiting (this is normal)
           ^^^^^ is the thread that issued the message

      ntop[18327]:   INIT: Bye bye: I'm becoming a daemon...
               (18327 is the 'master' thread created as ntop becomes a daemon)

      ntop[18327]: THREADMGMT: Started thread (16386) for network packet analyser 
               A message is issued for each thread started.
                                               ^^^^^ is the 'internal' thread #.
                                                     Ignore it
      ntop[18331]: THREADMGMT: Packet processor thread running... 
           ^^^^^ is the child thread.  Each one issues a message as it starts up

      ntop[18327]: THREADMGMT: Started thread (32771) for idle hosts detection 
      ntop[18332]: THREADMGMT: Idle host scan thread running... 
      ntop[18327]: THREADMGMT: Started thread (49156) for DNS address resolution 
      ntop[18333]: THREADMGMT: Address resolution thread running... 
      ntop[18334]: THREADMGMT: rrd thread (65541) started 
      ntop[18327]: THREADMGMT: Started thread (81926) for web server 
      ntop[18335]: THREADMGMT: web connections thread (18335) started... 
      ntop[18327]: THREADMGMT: Started thread (98311) for network packet sniffing on eth1 
      ntop[18336]: THREADMGMT: pcap dispatch thread running... 
      ntop[18327]: THREADMGMT: Started thread (114696) for network packet sniffing on eth2 
      ntop[18337]: THREADMGMT: pcap dispatch thread running... 

   So:

     UID   PID  PPID STAT     TIME is
     501 18327     1 S    00:00:00 Master
     501 18330 18327 S    00:00:00 (internal used by POSIX)
     501 18331 18330 S    00:00:05 Packet processor
     501 18332 18330 S    00:00:45 Idle Host Scan
     501 18333 18330 S    00:00:00 Address Resolution
     501 18334 18330 S    00:00:06 rrd
     501 18335 18330 S    00:00:00 Web Server
     501 18336 18330 R    00:00:32 eth1
     501 18337 18330 S    00:00:41 eth2
i
   If you connect to the running ntop via gdb, you can see this:

   $ gdb /usr/bin/ntop 18327
   (gdb) info threads
         ^^^^^^^^^^^^ Shows all the threads and their current state

     9 Thread 114696 (LWP 18337)  0x40255c68 in recvfrom () from
            /lib/i686/libpthread.so.0
     8 Thread 98311 (LWP 18336)  0x40255c68 in recvfrom () from
            /lib/i686/libpthread.so.0
     7 Thread 81926 (LWP 18335)  0x420dcc31 in select () from
            /lib/i686/libc.so.6
     6 Thread 65541 (LWP 18334)  0x420b0226 in nanosleep () from
            /lib/i686/libc.so.6
     5 Thread 49156 (LWP 18333)  0x40252a35 in __pthread_sigsuspend ()
            from /lib/i686/libpthread.so.0
     4 Thread 32771 (LWP 18332)  0x420b0226 in nanosleep () from
            /lib/i686/libc.so.6
     3 Thread 16386 (LWP 18331)  0x40252a35 in __pthread_sigsuspend ()
            from /lib/i686/libpthread.so.0
     2 Thread 32769 (LWP 18330)  0x420db1a7 in poll () from
            /lib/i686/libc.so.6
     1 Thread 16384 (LWP 18327)  0x420b0226 in nanosleep () from
            /lib/i686/libc.so.6

   (gdb) thread 2
         ^^^^^^^^ switch to thread 2 and see it's state

   [Switching to thread 2 (Thread 32769 (LWP 18330))]#0  0x420db1a7 in
            poll () from /lib/i686/libc.so.6

   (gdb) info stack
         ^^^^^^^^^^ check the call stack

   #0  0x420db1a7 in poll () from /lib/i686/libc.so.6
   #1  0x4024f9de in __pthread_manager () from /lib/i686/libpthread.so.0
                     ^^^^^^^^^^^^^^^^^ running __pthread_manager


Q. All the threads seem to be running ok.
A. Check if there's one that looks like this:

     4 Thread 32771 (LWP 27295)  0x420cd207 in sched_yield () from
            /lib/i686/libc.so.6

   Let ntop run a few more seconds (cont command), then check again.

   If it's frozen in the sched_yield, you're probably tripping a deadlock 
   situation that we've seen but don't understand - There's a lot of
   stuff on the 'net, including what seems to be the same problem with
   other resource intensive tools but no clear answers.  

   This happens most often on RedHat Linux 8.0 (9 uses a different
   threading library and the issue hasn't been reported there).  

   Regardless, if you see the deadlock or just have a hung ntop, there is a
   run time option to try: --disable-schedyield.  This option disables the
   calls at a slight penalty to ntop's interactivness.  If you're experiencing
   the deadlocks, try it.

   And, especially if you're running something other than RedHat 8, please
   let the ntop-dev list know about it.


Q. But what about SQL and mySQL
A. Removed in 2.1.50+ versions - use rrd


Q. But I really, really, need the data in an sql database.
A. If you are only interested in saving your netflow data in MySQL, use the
   script ntop/NetFlow/netFlowClient.pl (netflow v5).  With few additional lines
   you could save your data in flat files, forward the netflow data or whatever.

   What this script does it set itself up as a netflow collector and sql
   inserter. The main loop just accepts a netflow packet and inserts the flow(s)
   into sql.

   To use this with a single instance of ntop, just set ntop up as a netflow
   sender, directed at the script (e.g. 127.0.0.1 on any port you like).
   Configure the script with the port # and run it.

   The same idea would work with sFlow, you would just have to change the packet
   decoding part of the script.


Q: How does ntop use lsof? nmap?
A. ntop no longer uses these external tools.

   OS fingerprinting was replaced by ettercap.

   If you need lsof data, you should ssh to the host and run it locally (this
   allows the sysadmin to put appropriate security on lsof).


Q. Can I set the admin password from a script?
A. Yes, you can call ntop with the option:
      ntop --set-admin-password=password 	


Q. What's this warning about ownership?
A. The first time you run make install and ntop creates a new
   directory for ntop's files, there's a problem.  So you'll
   see this warning box:

      ************************************************************
      ************************************************************

        WARNING: This install created a directory for the ntop
                 files and databases:

                   some directory...

                 This directory MUST be owned by the user
                 which you are going to use to run ntop.

                 The command you must issue is something like:

                 chown -R ntop.ntop <directory>
           or    chown -R ntop:users <directory>

                 man chown to check the syntax for YOUR system

      ************************************************************
      ************************************************************

   Since root created the directory, root is the owner.  And since we don't
   know - yet - what user ntop is going to be run as, we can't issue the
   chown (CHange OWNership) command for you.

Q. And if I don't do this?
A. ntop won't run.  Typically you get a message about a bad -P file or
   endless prompts for the admin password.


Q. I can't merge interfaces (-M option)?
A. Check your plugins and see if either netFlow or sFlow is active.
   Regardless of whether you're using them, if they're active, they
   (silently) force the -M switch on.


Q. Explain -u.
A. -u root means that you are running ntop as root.  It means you don't drop
   root's superuser privledges, so pretty much you can read/write any file.
   It's not a good idea.  While we're not aware of any security problems with
   ntop, programs that run as root are targets.

   -u ntop (or -u whatever) means you run as a normal user and can be assigned
   only the privledges necessary to run ntop.  It also means you can only read/
   write files as ntop.  So less is exposed.

   For even better security, the -u userid should not have a valid shell, so
   people can't use it to login.


Q. What are the options that reduce ntop's workload?
A. These options turn off certain processing (and thus limit ntop's
   functionality), and may be appropriate for high volume installations.

   -b | --disable-decoders

     This flag disables protocol decoders. Use it for better performance
     or if you feel ntop has problems handling these protocols in your
     environment.

     This switch disables code in a number of places throughout ntop, code
     which analyzes specific protocols, but can place additional load on the
     host.  This switch could be used to run ntop on low-end CPUs or where
     ntop is acting as a collector (netFlow or sFlow) and the GUI is not
     required.

     Disabled is the analysis of:

        DNS Sniffing - where ntop captures DNS information from other hosts'
                       requests to reduce the # of DNS requests ntop must -
                       itself - make.

        NetBIOS   \
        NetWare    \
        AppleTalk   -- resource intensive protocol analysis of less
        bootp/dhcp /   common protocols.
        OSI       /

        http (80) - Request success/failure counting on port 80 and other
                    analysis, including "Virtual Host".

        ftp passive session tracking.

        "Wrong Port" monitoring for: http, ftp and smtp (used with the
              -q | --create-suspicious-packets option to dump "suspicious"
              packets to an analysis file)  With this option, ntop checks
              the payload for each new connection, looking for text usually
              present in http, ftp or smtp requests.  If these are not on the
              "normal" ports (http's 80, ntop's 3000 or squid's 3128, ftp's
              21 or smtp's 25) (or there is a non-ftp or smtp request on the
              standard ports), the packet is logged.


   -g | --track-local-hosts

     Use this flag to tell ntop that you care only about local hosts (use
     -m | -- local-subnets to specify local nets).  This flag is useful when
     ntop sees many hosts (e.g. border gateway) but only the local ones need
     to be tracked.

     This switch disables code in a number of places throughout ntop, code
     which allows ntop to track "foreign" hosts (that is ones not local
     according to the IP address(es) of ntop's interfaces or set pseudo-local
     by -m | -- local-subnets).

     Basically, ntop doesn't bother to do DNS resolution on these addresses
     and, for purposes of various counts, uses the "other" bucket instead of
     creating a unique hash table bucket for the specific host.

     This switch could be used to run ntop on low-end CPUs or where ntop is
     acting as a collector (netFlow or sFlow) and the GUI is not required.


   -o | --no-mac

     Specifies that ntop should not trust MAC addresses but just IP addresses.
     This option is useful whenever ntop is started on an interface where MAC
     (Media Access Controller - the low-level Ethernet address) addresses can
     not really be trusted (e.g. port/VLAN mirror in Switched Ethernet
     environments).

     Certain processing is performed differently:

          Hash search is via IP not MAC

     Certain capabilities are disabled:

          Analysis of bootp/dhcp requests
          localRoutersList.html report
          Wrong net mask log message and flag
          Analysis of non-tcp/udp protocols like NetWare and Spanning Tree
          Router listing on Host Detailed report.
          Traffic Matrix report

     (Note that this list is subject to change as we learn more about protocols
      that do/do not depend on the MAC address)

     See also -z | --disable-sessions


   -z | --disable-sessions

     This flag disables tcp session tracking. Use it for better performance or
     when you don't really need the tracking of sessions.

     Also, in situations where the MAC addresses cannot be trusted, ntop may
     - or may not - be able to accurately track tcp sessions.  There is no easy
     way to tell, so this switch puts control back into the users' hands.

     In versions after 2.0 up to & including 2.1.2, the -j | 
     --border-sniffer-mode flag (predecessor of -o | --no-mac) always turned
     this off.  Many users wanted to try turning session tracking back on, and
     did via code patches with mixed results.

     Suggested usage:  If you enable -o | --no-mac, try running ntop with
     sessions enabled.  If the data looks reasonable, congratulations - your
     network allows session tracking.  If the data does not look reasonable,
     then you will also need to disable session tracking with this switch.


Q. -s | --no-promiscuous doesn't work
A. It should work - it's passed to pcap_open_live.  Understand that it does mean
   ntop sees a lot less comprehensive view of the traffic.  You won't see
   anything different unless you do an ifconfig on the interface.  Note that
   while the parameter specifies if the interface is to be put into promiscuous
   mode, even if this parameter is false, the interface could well be in
   promiscuous mode for some other reason.

   If it fails, you'll see a message and ntop will refuse to startup

Q. How does -m | --local-subnets work?
A. This  flag  allows users to specify the subnets whose traffic is considered
   local (called "pseudoLocal" internally).  

   The format is
        <network address>/<# subnet mask bits>[,<network address>/<#  subnet 
         mask bits>

   For instance "131.114.21.0/24,10.0.0.0/255.0.0.0".


Q. (followup) but what does it MEAN?
A. Surprisingly, it means EXACTLY what it says.  Treat traffic on the listed
   subnet(s) as local.

   -m relates to the traffic you see on the wire at the TCP/IP level.
   -m tells ntop something it can't determine by itself.  And that is to 
   treat a range of addresses EXACTLY like it was local.

   For example, on my Cable Modem, I see broadcasts for a number of subnets
   that AT&T (now Comcast) has assigned to this area (I don't see the 
   traffic just the broadcasts).

   If you have VLANs or simply network overlays (two or more networks on the
   same wire, but with separate address spaces).  etc.

   Those are the cases where you use -m.  To tell ntop to treat that traffic
   as local.

Q. Why 
A  ntop differentiates between local traffic and remote traffic.  There are
   actually four classes (although only three are routinely reported) L->L L->R
   R->L and R->R.  Some additional processing is done and some additional 
   reporting is available for L traffic.

Q. I'm confused.  Explain, please!
A. Suppose your IP is 1.2.3.4 with a 255.255.255.0 netmask (a/k/a 1.2.3.4/24)

   Under the TCP/IP protocol, traffic with any address 1.2.3.1 -> 1.2.3.254 does
   not get routed.  It's "local".

   Your buddy is at 1.2.3.9 and the router is 1.2.3.1, so your network looks
   like this:

   the       +--------+
   world-----+ Router +--1.2.3.1--------------------------------------
   +dog      +--------+                | 1.2.3.4             | 1.2.3.9
                                  +--------+            +--------+
                                  |  You   |            |  Buddy |
                                  +--------+            +--------+

   Say you send a packet to your buddy at 1.2.3.9. You build a packet with
   SRC=1.2.3.4 DST=1.2.3.9 and your data and cast it out the wire.   (For
   purposes of this illustration, ignore the fact the your TCP stack would
   recognize the "local" nature of the packet and actually use another, lower
   level protocol, called Ethernet to deliver it.)

   The router (1.2.3.1) looks at it, does the math and ignores it - it's local
   Your buddy (1.2.3.9) looks at it, says - gee, that's me and reads it

   This is L->L traffic.

   Now you send a packet to ntop.org at 131.114.21.9.  Again, SRC=1.2.3.4 and
   Now DST=131.114.21.9.

   The router (1.2.3.1) looks at it, does the math and says - oops, I have to
   send it out to the world. Your buddy (1.2.3.9) looks at it, says - gee that's
   NOT me and ignores it

   This is L->R traffic.

   Now it's perfectly possible to have multiple (physical) networks on the same
   physical wire.  Say that your ISP chooses to put 1.2.4.1-1.2.4.254
   (1.2.4.0/24) on the same wire. (Why would they do this? - maybe it's a big
   pipe and only a few users or whatever).

   A packet from 1.2.4.4 -> 1.2.4.9 is seen by

   The router - no, that too is local, ignore it
   You (1.2.3.4): (1.2.4.9) - not me - ignore it
   Buddy (1.2.3.9) - um... 1.2.4.9 - not me - ignore it

   And that's perfectly legal.

   But what if you are the ISP and you want ntop to see ALL the traffic on that
   wire? ntop will figure out from it's own IP address that the 1.2.3.0/24
   traffic is local, but it will classify the 1.2.4.0/24 as REMOTE.

   And that is what the --local-subnets switch does.  It tells ntop to treat
   that 1.2.4.0/24 traffic as local.

   If there isn't any other traffic on the wire, then telling ntop to treat it
   as local won't change a thing.

   You can always use a packet sniffer, such as tcpdump to scan the traffic on
   the wire and see what's really there...


Q. And internally?
A. ntop is designed as a hybrid packet analyzer, not a pure Ethernet analyzer
   (layer 2) nor a pure TCP/IP analyzer (layer 3).  Most of ntop's displayed
   counts are at the TCP/IP level, and that's what confuses people.  Internally,
   ntop works both at the level of the Ethernet frame and the TCP/IP packet.

   A single MAC address can be associated with multiple TCP/IP addresses.  The
   MAC address -- unless something is horribly wrong on the network or with the
   hardware or somebody is deliberately spoofing it -- is guaranteed to be
   unique and refers to a physical host or network interface.  For many reports,
   ntop displays the information using the MAC address to separate physical
   devices.  Other data is accumulated and displayed at the TCP/IP (level 3)
   layer.

   -m relates to the traffic you see on the wire at the TCP/IP level.
   -m tells ntop something it can't determine by itself.  And that is to treat
   that range of addresses EXACTLY like it was local.

   For example, on my Cable Modem, I see broadcasts for a number of subnets
   that AT&T has assigned to this area (I don't see the traffic, but you get
   the picture) in an overlay structure (two or more networks on the
   same wire, but with separate address spaces).


Q. What about multicast traffic?
A. Multicast traffic uses the 224.0.0.0/4 address range (that is all addresses
   between 224.0.0.0 and 239.255.255.255 - see RFC 1112).  By default then,
   all multicast traffic is treated as 'Remote' by ntop.  

   Your actual network topology may be such that multicast traffic could
   all be local, all be remote or a mixture.  You can use the -m | 
   --local-subnets parameter to force some (or all) multicast groups
   to be counted as 'Local' traffic.

Q. What about VLANs?
A. VLAN traffic is also a special case.  ntop treats all traffic within your 
   machine's netmask definition as 'Local'.  For instance, you may have a /16
   network (netmask of 255.255.0.0), and you are using VLANs to distribute
   the traffic.  In this case ntop will see the a.b.c.d/16 address and classify
   all traffic within this /16 network as 'Local'.

   Don't design your network that way.

Q. But they did!
A. This MIGHT work:

   Use an un-numbered interface, so that ntop doesn't assign any implicit
   'Local' addresses.  Then use the -m | --local-subnets parameter to 
   define what is truly 'Local'.  As long as there aren't too many
   'Local' networks, this should work.  I run something sort of like this
   (but using /24s not subdividing a /16) for my development network.


Q. What are the default protocols ntop monitors?
A. (These are the ones ntop monitors if the user does not supply a -p parameter)
   Check addDefaultProtocols() in ntop.c around line 520.
   The current list (August 2003) is

     Protocol   Ports
     --------   -----

     FTP        ftp ftp-data
     HTTP       http www https 3128      /* 3128 is HTTP cache */
     DNS        name domain
     Telnet     telnet login
     NBios-IP   netbios-ns netbios-dgm netbios-ssn
     Mail       pop-2 pop-3 pop3 kpop smtp imap imap2
     DHCP/BOOTP 67-68
     SNMP       snmp snmp-trap
     NNTP       nntp
     NFS        mount pcnfs bwnfs nfsd nfsd-status
     X11        6000-6010
     SSH        22
     Gnutella   6346 6347 6348
     Morpheus   1214
     WinMX      6699 7730
     eDonkey    4661-4665
     Messenger  1863 5000 5001 5190-5193

   Note that the names come from /etc/services (or your system's equivalent).
   If you add protocols to /etc/services, you can refer to them by name on the
   -p parameter.

   The list changes over time as P2P protocols appear and disappear.  Check the
   cvs and diff ntop.c (around line 550 in void addDefaultProtocols() if you
   want the history.


Q. What about protocol XYZZY?
A. The analysis of protocols is very limited and unsophisticated.  But,
   theoretically, if it's there in plain text, we could report on it.
   The more work you can do up front in identifying the protocol (e.g. port #s,
   header structure, etc.), the easier it would be to add.




Q. How can I run ntop without being root?
A. A very simple way of doing this is:
   > su
   > chown root ntop
   > chgrp root ntop
   > chmod 6111 ntop
   > exit

   This makes ntop read-only for everyone and sets the setuid and setguid bits.

   Do not forget to use the -u flag so that ntop changes user as soon as it
   is started.

   Understand that setting the Setuid and Setguid bits allows ANY user to run
   ntop and it will run with ROOT privileges.  This is very powerful, and often
   a source of security exposure - many system hardening scripts and
   recommendations tell you to look for and remove the setuid and setguid bits.

   DO NOT suid UNLESS YOU UNDERSTAND THE RISKS!

   REPEAT: DO NOT DO THIS, IT IS A BAD IDEA!

   Also, there are unconfirmed reports of this method problems, causing a
      "socket: operation not permitted"
   message.  Probably related to something in the OS checking for real root
   not suid to open the interface in promiscuous mode.


Q. I am using a /16 (/25 or whatever) mask and I get this message:
      Truncated network size to 1024 hosts (real netmask 255.255.255.0)
A. Yes.  ntop limits each network to 1024 hosts (a /24). If you need more, alter
   the #define for MAX_SUBNET_HOSTS in globals-defines.h and recompile.  Space
   has to be reserved for this many hosts for each network, so the limit exists
   to keep memory usage from growing to absurd levels on people with "class A"
   (/8) interfaces (e.g. 10. or Cable Modems, etc.).


--------------------------------------------------------------------------------
-----Running - On-going---------------------------------------------------------
--------------------------------------------------------------------------------

Q. Explain L and R - why doesn't ntop double count?
A. Classification is all based on what ntop SEES in the packets.  ntop sees 
   packets and ONLY packets.  Packets have a FROM and a TO address.  Which 
   packets ntop sees is determined by the interfaces it is monitoring.

   Traffic (packets) is classified based on the joint classification of the
   FROM address (L or R) and the TO address (L or R).

   Only in L->L traffic will ntop see sent=rcvd and 'double count'.

                            Host: 192.168.1.x  www.yahoo.com
                                  L->R  R->L   L->R  R->L
                                  S  R  S  R   S  R  S  R
      192.168.1.x>www.yahoo.com
        HTTP GET ...             30  .  .  .   .  .  . 30

      www.yahoo.com>192.168.1.x
        HTTP 200                  .  .  .  8   8  .  .  .
     
      www.yahoo.com>192.168.1.x   .  .  .200 200  .  .  .
        <html>...</html>

      etc.


Q. Why does ntop use so much memory ?
A. ntop holds a lot of information about each host it has seen in an in-memory
   table. Periodically, it looks at all the entries in the table and flushes any
   which have been idle for a period of time.

   You can change the sizing of the table and the flushing interval via #define
   statements in globals-defines.h.

   But realistically, ntop needs enough memory to hold information about what's
   active on YOUR network.

   To reduce memory, monitor fewer protocols or use the filter (-B "bpf filter")
   option to monitor only parts of the network.

   There have been a couple of discussions on the ntop mailing list about ntop's
   memory usage - you might read them (search on gmane).


Q. So ntop's memory usage is dependent upon?
A. ntop's memory usage for host tables depends on the # of hosts it sees in
   packet traffic.  This is NOT, repeat NOT controllable by ntop in ANY way.
   If a user kicks off a port scan, 100s of hosts appear.  If somebody does a
   DOS attack against you, 1000s of hosts 'appear'.  If a user searches Kazza
   for an obscure song, it can probe 4K hosts. etc.

   Lots of those hosts appear, have a few bytes of traffic and then disappear.
   Each host has a variable amount of memory - there's a base structure, some
   optional counter structures and a large # of pointer fields, which may or
   may not be valued for any given host.  It depends on the # of active
   sessions and a lot of other things, but 8-20K is a good guess - I usually
   use 12K as an guestimating size.  Similarly, sessions may appear and
   disappear (http: opens a lot, does a small retrieval and closes them), ssh
   may last for days.  etc.   Memory is consumed tracking them too.


Q. So what does the purge do?
A. ntop purges idle hosts.  Period.

   Idle being defined as having not had packet traffic for a #define able
   period, 5 minutes by default.

   Because of the amount of linked data, these purges take time (lots of 
   free() calls on all those char* values), so we don't purge in 'real time'.
   First we build a list of idle hosts, then we purge from that list.
   Any idle host is eligible for purge (unless you tell ntop not to purge idle
   hosts, which is usable only on small networks).

   Over time, all idle hosts are purged.  Only to be perturbed by the 
   next burst of activity - say a port scan or everybody logging in back
   in after lunch.  Eventually, there's some form of steady state, but it's
   HIGHLY dependent upon network activity.  Which, remember, is external to 
   ntop and can't be predicted.


Q. Why?
A. Think about the overhead of sorting this huge structure by 'last packet
   seen time'.  1GB of ram is something like 80K+ hosts and takes a long
   time to sort (let alone free).   During which time, on that busy network,
   a couple of dozen packets are processed...  Meaning maybe your list of idle
   hosts is wrong.


Q. So how about a hard memory limit?
A. There's no way to make a hard limit without purging ACTIVE hosts (or
   non-IDLE given ntop's definition of idle).

   Think about it ... you're at that 1GB limit and you find a new host.
   Do you kick off an interim purge (with it's huge overhead?)  And wait
   2 or 3 s for an available slot??  While 1000s of other packets appear??


Q. Soft limit?
A. It might, might, be possible as a soft limit - but it's got a lot of
   issues.

   First off, tracking memory usage of the hosts tables is itself a huge
   job. There are multiple places were stuff is purged, added to the 
   structures as pointers, etc. and there's a queue of purged entries 
   for reuse to cut down on the malloc() calls.

   Secondly, the purge is resource intensive, and has been the cause of
   deadlocks before - you don't dare lock the structures for too long - 
   packets keep arriving, and FAST on the busy network that has the memory
   issues in the first place.  Since you can't lock for long, you can only
   purge a small # of entries.


Q. Why does ntop drop packets?
A. Short Answer: You need a faster processor.  Maybe.
A. Long Answer:  There are four places packets drop "in" ntop.  One in
   the NIC, one in the OS kernel, one in the libpcap library and one 
   actually in ntop.

   First off, this is largely NOT controlled by ntop.  It's inside the network
   card, the network card driver and libpcap.  All ntop does is use the stats
   provided by libpcap, pcap_stats:

      "int pcap_stats() returns 0 and fills in a pcap_stat struct. The 
       values represent packet statistics from the start of the run to
       the time of the call. If there is an error or the under lying 
       packet capture doesn't support packet statistics, -1 is returned
       and the error text can be obtained with pcap_perror() or 
       pcap_geterr()."

   to get the value and reports it in report.con the Stats | Traffic
   page, some in the configuration report (info.html) and also on
   the problem report skeleton.

   The information on the Stats | Traffic page is simply the best available.
   However, this data is not always easy to obtain nor interpret.

   It looks great, but the reality is that the stats provided by libpcap
   aren't very good!


Q. Not good?  Why...
A. The network card (NIC) reads packets from the wire into a small buffer.
   That buffer is emptied by the OS into it's own buffers which are then 
   passed to libpcap.  libpcap filters the packets (if requested) and 
   passes the packets to ntop.

   Packets may be dropped at any or all of these stages, even on a system 
   that does not appear to be exceptionally busy.

Q. How can packets be lost in the NIC?
A. A NIC has a small buffer - modern cards have 8K, 12K, even more buffer 
   space.  But if traffic comes in off the wire faster than the processes
   are pulling it out of the NIC, well that buffer gets over-written and
   packets get dropped.

   Every time a packet comes in (or on some newer cards after a few packets
   are recevied), the NIC interrupts the processor (CPU) to tell it to do
   something.  This isn't a problem - some NICs with larger buffers
   internally queuing a number of packets before interrupting the OS to
   boost performance.

   Check ifconfig for this:

      eth0      Link encap:Ethernet  HWaddr 00:D0:09:77:85:B9
                inet addr:192.168.42.6  Bcast:192.168.42.255  Mask:255.255.255.0
                UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
                RX packets:3892397 errors:30 dropped:0 overruns:0 frame:33
                                             ^^^^^^^^^
                TX packets:473009 errors:0 dropped:0 overruns:0 carrier:0
                                           ^^^^^^^^^
                collisions:1073 txqueuelen:100
                RX bytes:2606704447 (2485.9 Mb)  TX bytes:70474880 (67.2 Mb)
                Interrupt:11 Base address:0xc000

   Now, if it's an occasional burst, losing 1 or 2 packets won't kill 
   you.  TCP/IP recovers.  And ntop's statistics aren't life-critical.  
   If, however, it's continuous, on-going and the count is growing - i.e.
   the NIC/CPU combo can't keep up with the AVERAGE network flow?  

       You're toast...

   Upgrade the Processor or NIC. Maybe.

Q. Can you tell if the NIC is the bottleneck?
A. Probably not.  Different NIC cards (and NIC card drivers) record different
   information.  Different tools present pieces and parts of it differently.

   For example, a runt packet - one that is too short to really be a packet
   is supposed to be discarded according to the tcp/ip standard.  But do you
   count it as a packet received??

   On many systems, interface level (NIC) statistics are available through 
   the ifconfig (or similar) command.  But what is available and what they
   mean can be different - even though they're presented in the same format.
   Different programs can read the 'same' data and show different things.

   For example - eth1 is an Intel Ethernet Pro/100, eth2 a 3c905...

   # netstat -i --all
   Kernel Interface table
   Iface       MTU Met     RX-OK RX-ERR RX-DRP RX-OVR   TX-OK TX-ERR TX-DRP TX-OVR Flg
   eth1       1500   0  50752227      0      0      0       0      0      0      0 BMPRU
   eth2       1500   0 145053154      0      0      0       0      0      0      0 BMPRU

   No errors - great, right?  But look at ifconfig, a few seconds earlier:

   # ifconfig
   eth1      Link encap:Ethernet  HWaddr 00:03:47:B1:62:27  
             UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
             RX packets:50754251 errors:0 dropped:0 overruns:0 frame:3454146
             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:100 
             RX bytes:667103055 (636.1 Mb)  TX bytes:0 (0.0 b)
             Interrupt:9 Base address:0x1000 

   eth2      Link encap:Ethernet  HWaddr 00:60:97:04:30:33  
             UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
             RX packets:145056087 errors:0 dropped:0 overruns:0 frame:0
             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:100 
             RX bytes:1784472399 (1701.8 Mb)  TX bytes:0 (0.0 b)
             Interrupt:9 Base address:0xb800 

   If I'm reading the code right, then the frame:nnnn count is set here:

     sp->stats.rx_frame_errors += le32_to_cpu(sp->lstats->rx_align_errs);

   Meaning? No clue - the writer of the driver had to sign an NDA and so
   couldn't explain a lot of what the driver does.  Besides, that data
   isn't available to ntop anyway.


Q. How can packets can be lost in the kernel?
A. Each time that interrupt occurs, the kernel processes it by moving
   the data from the NIC to a kernel buffer, then re-enable interrupts.

   Not really.

   Actually, in Linux (and other OSes), the Kernel interrupt handler is
   broken in half - called in Linux the top and bottom.  The bottom 
   interrupt works with processor interrupts off to grab the data and 
   buffer it and does only minimal processing as fast as possible so 
   interrupts can be turned back on.  Then the top half is scheduled 
   and processes as a Kernel process (high priority), but it is less 
   time critical.

   However, if the kernel can't process the bottom half in time - because
   there isn't enough memory and/or the processor isn't fast enough to
   respond to the interrupt, you do have a problem.   The small buffer
   in the NIC will overflow and packets are dropped (this is the ONE place
   where a better NIC, with a larger buffer, MIGHT help).

   While there are LOTS of possible causes, if the kernel is *routinely*
   dropping packets, it's almost certainly an interrupt/processor
   speed/buffering issue.

   So, I guess what I'm saying is that the dropped: count from ifconfig
   might not be the NIC's fault - and that's why a faster processor is a
   better choice than an expensive server class NIC.


Q. But at least I can trust the libpcap reported counts, right?
A. Nope. You need to treat the pcap reported drops with more than a few
   grains of salt.  See this thread, for example:
      http://www.tcpdump.org/lists/workers/2001/07/msg00018.html
   and this msg:
      http://www.mcabee.org/lists/snort-users/Jan-02/msg00771.html

   There's a lot of other stuff about pcap problems, especially from
   snort (another package that stresses libpcap) - 
   STFW for "libpcap ps_drop".

   See, the call to pcap_stats reads internal counters - maybe they come
   from the NIC and maybe they're just maintained by the kernel.  That's a
   function of how the low level NIC driver is written to handle certain 
   calls from higher level drivers.  So it's a function of the specific
   NIC card, something we try to hide from everyone, ntop included.

   Ultimately, some piece of hardware or software SHOULD be counting packets
   and dropped packets.

   But, remember also, that - AT THE INSTANT OF THE pcap_stats call -
   there can be packets queued in any of these buffers (NIC, kernel, libpcap).

   Put the two together and I wouldn't trust 'em.

   Want more fear?  Read the comments through out the libpcap code.  Like 
   this gem from pcap-linux.c:

    *  Get the statistics for the given packet capture handle.
    *  Reports the number of dropped packets iff the kernel supports
    *  the PACKET_STATISTICS "getsockopt()" argument (2.4 and later
    *  kernels, and 2.2[.x] kernels with Alexey Kuznetzov's turbopacket
    *  patches); otherwise, that information isn't available, and we lie
    *  and report 0 as the count of dropped packets.

   And this from pcap-bpf.c:

    /*
     * "ps_recv" counts packets handed to the filter, not packets
     * that passed the filter.  This includes packets later dropped
     * because we ran out of buffer space.
     *
     * "ps_drop" counts packets dropped inside the BPF device
     * because we ran out of buffer space.  It doesn't count
     * packets dropped by the interface driver.  It counts
     * only packets that passed the filter.
     *
     * Both statistics include packets not yet read from the kernel
     * by libpcap, and thus not yet seen by the application.
     */

   As you see from the comments in the code, the interpretation of even
   those counts differs between operating systems and even between minor versions
   of the same OS (e.g. Linux 2.4.8 vs. 2.4.9).  For example, some systems 
   do not maintain a dropped count and will always report zero.  Differences
   may also occur on the same system for different NIC drivers.  
 
   If a BPF filter (-B option) is in place, the differences between OSes are
   more significant.

   etc.

   Trust nobody...


Q. ntop drops packets?
A. Sure. ntop runs multiple threads (we call them "NPS - Network Packet Sniffer"
   or something similar), one to handle each incomming device.  These operate
   much like the bottom-half interrupt - they accept the packet and queue it
   to another thread ("NPA - Network Packet Analyzer") for the analysis.

   Ultimately, we're interested in the counter droppedPkts, which is
   incremented in only one place, pbuf.c around 1398, which this is where 
   NPS is trying to queue a packet for NPA.

   Now, you can increase PACKET_QUEUE_LENGTH

      ntop.h:   693   #define PACKET_QUEUE_LENGTH     2048

   but if you're routinely dropping packets here it's because ntop can't 
   keep up with the flow.  Increasing the queue length will ONLY help if
   it's an occasional huge peak with times where the network is quiet 
   enough to allow you to work off the queue.  And each packet buffer 
   in use takes up memory - (about 1.5MB for a queue of size 2048).

   The instantaneous and maximum value for the queue are reported in
   the configuration report:

      # Queued Pkts to Process 0
      # Max Queued Pkts 0

   But, again, the best answer for dropped packets is probably a 
   faster processor... (Upgrading to an expensive faster NIC with a 
   larger internal buffer is rarely the best "bang for the buck" - if 
   you were running a server, then a server class NIC is a good idea,
   but for workstations or network monitors, etc. you just need something 
   that can keep up with the network flow.)


Q. So what do you show?
A. The kicker is that accurate counts of raw and dropped packets do not exist
   or are not available without using low-level, system specific logic.

   The counts that ARE available in a system-independent way are what is 
   reported in the statistics on the Stats | Traffic page.


Q. So what is reported?
A. ntop reports the packet and dropped counts as reported by libpcap and the
   received, dropped and processed counts it maintains internally.


Q. So what does ntop do with what it does receive?
A. ntop processes or queues all packets received from libpcap but may drop 
   packets if the processing queue fills up.  Losing a few packets inside 
   ntop due to a rare burst of traffic is just that - rare, and is not a 
   'bug'.

   If you consistently see packets dropped by ntop then you probably need
   to use a faster processor (increasing the buffer size beyond the default
   of 2K packets will usually not help).  

Q. How do I tell?
A. You can monitor the queue size on the 'Configuration' (info.html) page.

Q. What about the other dropped items?
A. If you consistently see packets dropped at either the interface or 
   libpcap level, there is not much we can offer in the way of help for you.

   The best suggestion is to try another NIC or a faster machine.  

Q. Why?
A. Some NICs - especially ISA ones - are just too slow at moving packets 
   off of the card. Measurements I did on an old SMC 10BaseT card in 1993
   showed that the best it could do - ISA bus, windows overhead, etc. all
   rolled together was about 1.7Mbps.

   Memory bus speeds (e.g. PC100 vs PC133, DDR333 vs DDR400, etc.) can also
   affect moving packets from the NIC to main memory.

Q. Won't a faster CPU help?
A. Maybe.  Some network card drivers are processor intensive - and would
   benefit - but others offload most of the processing to the NIC, and so a
   faster CPU wouldn't matter.

   Even a fancy (read as expensive) NIC may not work well - the drivers
   for the NIC on one OS may take advantage of features the card provides 
   and be 'faster', while the driver for the same NIC on another OS may 
   not take advantage of these features and be 'slower'.

   As we say, YMMV...


Q. So?
A. So, remember - ntop drops packets, the kernel drops packets, libpcap drops
   packets and none of 'em do it the same way.  Don't bet the farm on counting
   every packet with absolute accuracy and use ntop the way it was meant to
   be used - showing the overall usage and flows on the network.

   Still it's not all gloom and doom!

   Even if you're only processing 90% of the traffic, since the drops are
   pretty random, it's a decent bet that if 90% of what you DO see is MP3s,
   then 90% of the whole traffic flow is probably MP3s.


Q. ntop stops capturing packets, except ARP and other broadcasts.  Why?
A. Check if you have a daemon running that periodically checks for and
   resets interfaces in promiscuous mode?  If that happens, all you
   would see were broadcast packets like ARPs...

   Check back in the log and see if there is a message about the interface
   changing status.  Determine why.


Q. How much horsepower do I need to run ntop on a network of size x?
A. Nobody really knows.  ntop needs enough memory to store the active
   hosts and enough cpu to keep up with the average packet flow.  The
   buffer will handle the occasional peak, but if you see frequent
   lost packets, you're in trouble.

   Note that a few packets occasionally lost isn't a big deal for most
   users.  After all, the network itself has losses - I've seen my AT&T
   Broadband connection have spurts of 30% packet loss.  Ideally in a
   LAN environment, the packet loss should be down in the small #s...
   the Ethernet standard allows 1  error in 100,000,000(10^8), but most
   vendors beat that by a long margin (even as high as 1 in 10^12).

   Of course, those are lab measurements.  In the real world?  Not that good.
   Electrical noise can be a real bugaboo. Remember, at a certain point, if
   the NIC doesn't understand what it's seeing, it throws it away and
   declares an error.  The key is to keep up with the traffic.

   Similarly, the OS kernel does the same thing in it's interrupt handling
   (throw away packets).  Last resort, but better than hanging up the whole
   machine.

   ntop drops packets when the queue gets longer than the permitted length.
   You can see this in the configuration page as # Queued Pkts to Process
   and # Max Queued Pkts.

   One or two or a small number (you pick your tolerance) is ok, but constant
   losses isn't.  What I'm saying is that as long as ntop can keep up with the
   NIC, then the data is as good as it gets...  if ntop can't keep up, then the
   data isn't very good.

   If you have measurements - network size, traffic flow and %CPU used (with
   the hardware info, of course), shoot them over to us on ntop and someday
   maybe we'll be able to give better #s.


Q. What about my Frobozz Model xx Magic Network card? Is it good enough?
A. Probably.  Well, a lot of the cheapos just don't have the buffering 
   and cpu offload of a 3c905c or such.  If the network isn't that busy,
   anything will do.  For a busy network, buy a decent PCI NIC.


Q. Gigabit Ethernet?
A. No clue.  Remember that you can suck a lot more traffic over that
   network than an old PC can handle (i.e. the bandwidth limitations
   of 32bit PCI and PC100/133 RAM, heck even PC2700 DDR).  

Q. Do "zero copy" drivers help?
A. Yeah.  Maybe.  Once upon a time, I read about "zero copy" - look here
   http://people.freebsd.org/~ken/zero_copy/ for the FreeBSD stuff. Quoting:

      "What is "zero copy"?

      Zero copy is a misnomer, or an accurate description, depending on how
      you look at things.

      In the normal case, with network I/O, buffers are copied from the user
      process into the kernel on the send side, and from the kernel into 
      the user process on the receiving side.

      That is the copy that is being eliminated in this case. The DMA or 
      copy from the kernel into the NIC, or from the NIC into the kernel is 
      not the copy that is being eliminated. In fact you can't eliminate 
      that copy without taking packet processing out of the kernel altogether.
      (i.e. the kernel has to see the packet headers in order to determine 
      what to do with the payload)

      Memory copies from userland into the kernel are one of the 
      largest bottlenecks in network performance on a BSD system, so 
      eliminating them can greatly increase network throughput, and 
      decrease system load when CPU or memory bandwidth isn't the 
      limiting factor."

   What's that mean?  It means that BEST CASE, you have to copy the data 
   NIC->Kernel and without zero copy, it happens TWICE.  Then, once you're
   in the kernel, it has to hand the data off to libpcap (another copy) 
   and from libpcap to ntop.  So we're moving the data 3 or 4 times... 
   best case!

   Let's do some off the cuff math.  Looking here
      http://www6.tomshardware.com/mainboard/20020501/ddr400vsrambus-05.html
   shows a table of memory types and max bandwidth (picking a few):

      Label  Name  Effective Clock  Data Bus Bandwidth
      PC100  SDRAM       100 MHz     64 Bit  0,8 GB/s
      PC133  SDRAM       133 MHz     64 Bit  1,06 GBb/s
      PC2700 DDR333      166 MHz     64 Bit  2,7 GB/s

   See the problem?  A fully loaded GigE link is 1 Gb/s.  4x that is 0.5 
   GB/s - so if ALL that's going on is the capture of packets from the network,
   the CPU can keep up.  Maybe.  Those bandwidth numbers are theoretical, best
   case (nice sequential access).  Throw in some other system activity, cache 
   misses and CAS/RAS issues... and um...

   The moral is that if you're going to use ntop to monitor big fat links,
   you need screaming fast iron.


Q. Can I disable logging? Totally?
A. Sort of - if you run single threaded, without the -d or -L options.
   Multithreaded?  No.  If ntop creates child threads, they don't have
   terminal access and have to have some way of reporting things.


Q. I'm seeing weird "hosts" on my network with names like
   "Bridge Sp. Tree/OSI Route".
   What are they?
A. There is a list of "special" MAC address prefixes in vendor.c,
   specialMacInfo[].  There are blocks of MAC addresses reserved (sometimes not
   formally) for special uses, such as sharing information about Spanning Tree
   for bridges.  These do not have an IP address - they operate at a lower
   level - so nothing gets displayed in some of ntop's fields.

   A reference about protocols at the wire level is here:
      http://www.oreillynet.com/pub/a/network/2001/03/02/net_2nd_lang.html

   If you only want to see TCP/IP, then I suggest you use -B "ip" to filter
   only TCP/IP protocol on your ntop line...


Q. How do I see fully qualified names for all my hosts? Some are netbios
   names!
A. ntop doesn't SEND NetBIOS queries, it sniffs them off the traffic already on
   the network.

   There is only ONE case where ntop uses the NetBIOS names, which is if
   it can't resolve them via DNS (both it's own queries and from sniffing
   responses to other's queries off the network).

   So, if you have a properly functioning DNS, you'll see DNS names.  If
   these are (for example) internal names, unknown to the DNS server, you'll
   see NetBIOS names if they are available.  Lastly, you'll get IP addresses...

   If you do have a DNS, and the name is resolved as part of the default
   domain, you won't see a fully qualified name back from the DNS, so ntop
   won't have that information.

   So, on a real network you'll often get a mix of name resolution types:

    Host                            IP Address      MAC Address
                                                                   Other Name(s)
    netnews.attbi.com               63.240.76.16
    tigger.homeportal.2wire.net     192.168.0.xx   00:D0:09:xx:xx:xx
    homeportal.homeportal.2wire.net 192.168.0.1    00:D0:9E:xx:xx:xx
    swallowtail                     192.168.0.XX   00:A0:CC:xx:xx:xx SWTL [DMN]
    12-xxx-xxx-xxx.client.attbi.com 12.xxx.xxx.xxx 00:D0:9E:xx:xx:xx
    12-xxx-xxx-yyy.client.attbi.com 12.xxx.xxx.yyy


Q. What does this log message (and others like it) mean?
   **WARNING** releaseMutex() call with an UN-LOCKED mutex [hash.c:559] last
               unlock [pid 22176, pbuf.c:2598]
A. Those messages are part of an error check in our mutex handling routines.

   Actually they pretty much mean what they say.

   releaseMutex() call with an UN-LOCKED mutex means that the corresponding
   accessMutex() call never occured.

   releaseMutex() call with an UN-INITIALIZED mutex means that the 
   access/release occured before the createMutex().

   The file/line comments, [hash.c:559] last unlock [pid 22176, pbuf.c:2598],
   tell you where the access/release was called from (e.g. hash.c @ 559) and
   where the last recorded release was.

   There are others, look in leaks.c for the entire set.

Q. How serious are they?
A. None of these stop ntop from processing, but they're indications of
   unprotected accesses to shared data areas, which could lead to lost counts.

   Now understand that there's one set of conditions where I believe it doesn't
   matter, and that's when there are multiple NICs and multiple processors.  
   Under some circumstances - not well understood - gcc with the -O2 option 
   reorders the executable code in such a way as to cause these messages.  
   Those are spurious.

   But for most users, these warning messages shouldn't be ignored if they occur
   often.

   The problem is that a POSIX mutex doesn't allow for additional data, 
   such as we're using to track the access / release information.  So that 
   data is, itself protected by a single global mutex.  You can't hold that
   global mutex across an access call, or you'll deadlock ntop (either through
   normal mutex processing or something called 'priority inversion').

   So we use this pattern instead:

      access(stateChangeMutex)
        set state data
      release(stateChangeMutex)

      access(mutex) or release(mutex)

      access(stateChangeMutex)
        set state data
      release(stateChangeMutex)

   The problem is that these updates aren't atomic, so it's possible to have
   the state data updated by another thread:

      ---------thread A---------         ---------thread B---------
      access(stateChangeMutex)
        set state data
                                         access(stateChangeMutex)
                                         *****blocked****
      release(stateChangeMutex)
                                         (resumes)
                                           set state data
                                         release(stateChangeMutex)
      access(mutex)
      ****blocked****
                                         release(mutex)
      (resumes)
                                         access(stateChangeMutex)
                                           set state data
                                         release(stateChangeMutex)
      
      access(stateChangeMutex)
        set state data
      release(stateChangeMutex)

      Unfortunately, this is equally likely:

      ---------thread A---------         ---------thread B---------
      access(stateChangeMutex)
        set state data
                                         access(stateChangeMutex)
                                         *****blocked****
      release(stateChangeMutex)
                                         (resumes)
                                           set state data
                                         release(stateChangeMutex)
      access(mutex)
      ****blocked****
                                         release(mutex)
      (resumes)
      access(stateChangeMutex)
        set state data
      release(stateChangeMutex)
                                         access(stateChangeMutex)
                                           set state data
                                         release(stateChangeMutex)


   So at the end, you have an inaccurate picture in the state table.

Q. Threads. ugh...
A. There's a good set of articles on POSIX threads at
      http://www-106.ibm.com/developerworks/library/l-posix1/,
      http://www-106.ibm.com/developerworks/library/l-posix2/ and
      http://www-106.ibm.com/developerworks/library/l-posix3/.


Q. What does the message "URL security(1): ERROR: Found percent in 
   URL...DANGER... rejecting request" mean?
A. It means that ntop received a request with a percent sign (%) in it, often
   used as part of Unicode exploits against various web servers.  Since there is
   no situation where ntop should process this, we reject it.
   URLsecurity in http.c is the place where these tests occur.


Q. What does the message "Rejected request from address x.y.z.t (it previously
   sent ntop a bad request)" mean?
A. Once you send ntop a request that URLsecurity rejects, the sending address
   goes into a ring buffer on a 5 minute timeout where we simply drop subsequent
   requests...  rather than waste cycles ignoring an attack...


Q. What are the other URL security(#) codes?
A. 1.  Found a % in the request (Unicode problems)
   2.  Found a parameter type code (//, &&, ??)
   3.  Found a directory transversal code (..)
   4.  Found a prohibited (RFC1945) character
   5.  Found a bad extension


Q. ntop doesn't report any traffic at all.
A. Understand how ntop works:  It simply listens on the interface(s) for
   packets, then counts and interprets them.  If there aren't any packets, ntop
   doesn't count things.

   ntop does not sample.  It processes every packet it sees and counts them.
   Only if there is more traffic than ntop can handle for a long period of time
   will the packet queue hit it's limit and packets be lost.  But this is still
   not sampling.

   Make sure that there's traffic on the interface(s) you are using. You can
   use tcpdump or a similar network sniffer tool to check.

   If you are on a segmented network (i.e. switched), you may not see traffic
   that isn't destined for the ntop machine unless you configure the switch to
   set the port for the ntop host into "mirror" or "management" mode (different
   vendors call it different things, but it's a mode where ALL traffic is copied
   to a specific port, regardless of which port the destination host is on).

   If there is more than one interface in the ntop host, perhaps you aren't
   listening on the one that has traffic?  Check using ifconfig:

   eth0      Link encap:Ethernet  HWaddr 00:D0:09:77:85:B9
             inet addr:192.168.0.34  Bcast:192.168.0.255  Mask:255.255.255.0
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:1105906 errors:0 dropped:0 overruns:0 frame:0
             TX packets:601935 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:100
             RX bytes:119869887 (114.3 Mb)  TX bytes:112203781 (107.0 Mb)
             Interrupt:11 Base address:0xc000

   If the RX and TX numbers are increasing, this shows that traffic IS
   flowing...

   If you have an unnumbered interface (listening only), remember you need to
   use -m to tell ntop what is local and what isn't:

   eth1      Link encap:Ethernet  HWaddr 00:30:F1:54:55:00
             UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
             RX packets:1596612 errors:0 dropped:0 overruns:0 frame:0
             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:100
             RX bytes:566953031 (540.6 Mb)  TX bytes:0 (0.0 b)

   You can select an interface using the '-i' flag, e.g. -i eth1 or
   -i eth0,eth1.


Q. ntop doesn't understand xxxxx?
A. True. IP Packets have a source address & port and a destination address &
   port... you MUST get your head out of the application layers and revert to
   that simple concept.

   How does Apache handle virtual hosts? It analyzes the flow at the
   application level (layer 4) not the wire/packet/protocol (layers 1, 2 and
   3).  It does this by re-assembling packets into a layer 4 message (e.g. GET
   http://virtual.host.name.com/page.html)...

   Now there are some layer 4 analysis routines - virtual hosts was added in
   2.2 (and the folks who have virtual hosts have been pretty pleased), ftp, 
   http, and some others - mostly looking for traffic on non-standard
   ports, etc.

   So, since ntop works at the packet level, it doesn't understand virtual
   hosts. Unless it's SPECIFICALLY coded for.  ntop is a NETWORK analyzer, not 
   an application level one.


Q. tcpwrappers doesn't work
A. Oh yes it does... for http: connections

   1) You have to configure it this way before compiling ntop:
        ./configure --enable-tcpwrap

   2) You must have the headers and libraries installed on the build machine
      (and on the execution machine if they aren't the same).

   Remember to make the appropriate entries in hosts.allow (e.g. 
   ntop:192.168.0.) and hosts.deny (e.g. ntop:ALL)

   However, tcpwrappers and https:// is known not to work - see docs/KNOWN_BUGS


Q. My filter doesn't work!  I'm running ntop like this:
      /usr/local/bin/ntop -u nobody -L -d -E -w 3000 -S 2 \
                      -m 192.168.10.0/24,xxx.xxx.xxx.xxx/32 \
                      -M -i eth0,eth1 \
                      (src net 192.168.10.0/24 or src host xxx.xxx.xxx.xxx ) \
                       and not dst net 192.168.10.0/24
A. Yup, it doesn't work.  Use the -B option and put the filter in quotes:
      -B "(src net 192.168.10.0/24 or src host xxx.xxx.xxx.xxx ) and 
           not dst net 192.168.10.0/24"

   ntop used to assume anything it didn't recognize was a filter.  But not since 
   2.1.3.  If you try this now, you should see a log warning that says maybe you
   forgot the quotes


Q. I have experienced problems defining multiple filters: ntop reports 'syntax
   error'
A. If you believe the filter is syntactically correct then it's likely that the
   Libpcap you have used has been compiled using an old non-reentrant version of
   flex.  Please make sure you're using version 2.5.4 or above.

Q. Can you give some additional examples of filters?
A. man tcpdump -- see "expression"

   A couple of simple examples, courtesy of B. Loic:

       -B "host not 192.168.1.100 and not 192.168.1.101"

   to exclude hosts 192.168.1.100 and 192.168.1.101 from tracking (FQDN 
   such as www.yahoo.com will work too). 

   If you need to exclude a full IP range, you will want to use something like

        -B "net not 192.168.1.0/24" 


--------------------------------------------------------------------------------
-----Running - Web Server-------------------------------------------------------
--------------------------------------------------------------------------------

Q. ntop shows an older, single menu interface
A. If ntop is unable to find the file index.html it generates the page
   internally. That page refers to 'leftindex.html' which is the all-in-one menu
   you see, similar to the v1.3 menu.

   To find the html files, ntop looks in the html subdirectory in two places:

      1. In the current directory (i.e. ./html),
   and
      2. In '[prefix]/share/ntop/html'
         (where [prefix] is set by the --prefix option of your ./configure
          step).

   Common causes:

      1. Is manually installing ntop in an unusual place, having forgotten to
         update DATAFILE_DIR in config.h. Or forgetting to copy the html
         subdirectories, etc.

      2. Forgetting to run './autogen.sh -1' first and 'make install' last when
         first building ntop from source.

      3. Running ntop with an explicit path from somewhere other that the
         directory it's installed into. For example, if you install ntop 
         into /root/ntop, but run it like this:

            cd /usr/bin
            /root/ntop/ntop

         It will look 1st in /usr/bin/html and then in [prefix]/share/ntop 
         and not find the html files in /root/ntop/html!

   This often occurs when running ntop as a daemon, because the current
   working directory of the script is not what you expect it to be!


Q. Why does ntop display bits of my web site, instead of its own pages?
A. ntop is designed to search the current working directory for data files,
   such as the html subdirectory, before it searches the default directories.

   This is a feature.

   You are one of those rare souls who happen to have had an unrelated
   subdirectory 'html' with a file named 'index.html' as a subdirectory 
   of the current working directory at the time that you launched ntop. 
   cd to an acceptable directory, such as /usr/share/ntop, before 
   launching ntop.  


Q. What are High/Medium/Low risk flags
A. They are set in reportUtils.c based on fairly self-obvious functions
   Well documented in the help.html page, reachable by clicking on
   the problem descriptions or via http://127.0.0.1:3000/help.html

   Often seen if you are monitoring a backbone or common network (high)
   or if you have cloned MAC addresses for, say, a home Firewall box.


Q. What does the "Users" flag mean on a host?
A. If you go to the "Info about host xxxx" page, there will be data
   in the "Known Users" section, if it's acting as a server for certain
   protocols.

   In sessions.c, the function updateHostUsers() is used to maintain the list
   of "users" of a host.  In handleSession(), as part of the protocol level
   analysis, the "user" information for various protocols is pulled out of the
   packets.  Stuff like the "X-Kazaa-Username" header, the "MAIL FROM:" header,
   etc.

   We tag users as one or more of the following types:
        P2P_USER, SMTP_USER, FTP_USER, POP_USER, IMAP_USER

   Note that for P2P, we also record - where possible - whether this user is
   in P2P_UPLOAD_MODE and/or P2P_DOWNLOAD_MODE.


Q. Why are some of the host names in different colors?
A. Colors are used on several of the ntop pages to convey extra
   information to the user.  (in particular the ACTIVE TCP SESSIONS
   and the LOCAL HOST STATS pages). There are five colors used to
   depict how long ago the host was first seen by ntop.

   The pages which display these colors use a html style sheet called
   style.css located in the normal html subdirectory (where ntop is
   installed). This happens by setting the 'class=' parameter of
   the html 'A' (Anchor or hyper-link) tag.  The style sheet defines
   the following:

      Age of host  'class' name  Color code         Color description
      (minutes)
      -------------------------------------------------------------------
      0-5          A.age0min    { color:#FF0000 }   Red
      5-15         A.age5min    { color:#FF00FF }   Fuchsia/Magenta
      15-30        A.age15min   { color:#FF7F00 }   Coral (lt orange)
      30-60        A.age30min   { color:#007FFF }   Slate blue
      60+          A.age60min   { color:#0000FF }   Blue

   The color legend is displayed on the About | Configuration page
   (info.html).


Q. What does the P2P flag mean on a host?
A. If ntop knows enough to tag you as a P2P user, it's also looking at the
   other headers to see if it can track what files you're exchanging.  If a host
   (i.e. a workstation) downloads a file from another host ("server"), the file
   name is recorded in the list ntop maintains for both of them.

   If a host has at least one file name recorded, it's tagged with the "P2P"
   flag.


Q. What does a "Virtual Host" mean.
A. If a single instance of a web server handles many web sites, all of the
   references resolve to the same name.  The web server uses the "Host:"
   header to determine which "index.html" page to serve up.

   ntop monitors port 80 (http:) exchanges and looks for the Host: which allows
   it to build a list of virtual hosts being handled by the web server.


--------------------------------------------------------------------------------
-----Running - Web Server (https:)----------------------------------------------
--------------------------------------------------------------------------------

Q. SSL is not working! I have the following error in the log/terminal:
     10/Jun/2002 22:58:17 Started thread (6151) for network packet sniffing on
          eth0.1700:error:140EC0AF:SSL routines:SSL2_READ_INTERNAL:non sslv2
          initial packet:s2_pkt.c:187:
A. You forgot to put https:// instead of http:// in the URL you put in your
   browser!


Q. Unable to find SSL certificate 'ntop-cert.pem'
A. ntop looks such file under the current working directory, then /etc or in
   Whatever directory you configured with ./configure.

   If you want a personal certificate, you need to create it by:

      >make ntop-cert.pem

   It should be installed as part of "make install".  If you have a special
   Certificate or it's not present, do it (one-time) manually:

   For example to install it under /usr/local/etc, do:

   mkdir /usr/local/etc
   cp /usr/local/bin/ntop-cert.pem /usr/local/etc/ntop

   See docs/README.SSL


Q: What is the ssl watchdog?
A: Short answer: There are reported problems w/ the ntop web server hanging when
                 accessed via ssl (https://) from Netscape 6.2.2 (Win2K) (and
                 others).

                 The ssl "watchdog" keeps an eye on the web server - it waits
                 for 3 seconds and then if the SSL_accept call (openSSL) hasn't
                 finished, it aborts it. This leaves the user with nothing on
                 their web browser, but at least ntop's web server continues on.

                 There is no known way to send something back to the user. DON'T
                 EVEN ASK. It's not in ntop, it's the browser-server handshake
                 that's hung.  So, it looks - to the user - like a failed 
                 connection.  S'be'it...

   If you are using https:// and seem to have the problem, run ntop with the 
   --ssl-watchdog command line parameter... The item to look for on the
   configuration page (info.html) is:

       # HTTPS Request Timeouts

   Or messages in the log:

   ...: SSLWDERROR: Watchdog timer has expired. Aborting request, but ntop
        processing continues!

   You can also enable it via a ./configure parameter (./configure --help |
   less) if it's something you're going to always require.


Q. Tell me more
A. The problem is that ntop's web server is single threaded until we determine
   that the request is simply one that will be reading data.  At that point we
   fork to generate the page.  But the basic "accept a request" code is single
   threaded.  This happens all but instantaneously and hasn't been a problem
   previously.

   The code is pretty basic and pretty common:

       select() to wait for a connection, then
       ssl_accept() to fire up a "server", meaning the ssl handshake.

       Then process the http request (i.e. the GET and associated headers).

   With Netscape 6.2.2 (and others), there seems to be a bug in the Netscape
   code (ntop's is identical to other projects like sshd).

   According to something I read - but now can't find again - Netscape doesn't
   accept a legal combination of options on the handshake back from openSSL and
   hangs in a deadly embrace.  Supposedly openSSL 0.9.6c (or was it d - it's not
   in the changelog) built in a patch.  However, I didn't find the new version
   changed the behavior.

   There is stuff about a bug w/ Netscape 4.x on the openSSL website, but I'm
   not having trouble with Netscape 4.x.

   I don't understand the details and really don't care to find out.  It boils
   down to a hang in a call, SSL_accept() that doesn't have a timeout parameter.
   Argh

   Because the code is invasive, I built it (like the SIGPIPE stuff) so you can
   turn it on at ./configure time:

      --enable-sslwatchdog    Watchdog for ssl hangups (Netscape 6.2.2)
                              [default=disabled]

   or via a command line option:

      --ssl-watchdog          Use ssl watchdog (NS6 problem)

   With the "fix", ntop's web server hangs for at most 3 seconds, then continues
   on.  The user gets nada - and I don't know a way to send them anything,
   because we haven't retrieved the request yet nor done the handshake (so there
   isn't a TCP connection!)

   It only affects https:// requests and I've coded the watchdog so it doesn't
   activate unless we have openSSL and either the compile or runtime parameter
   set.  If you don't get https:// requests, it's just another idle thread.

   The fix is working for me... What I've tested (and the results with and
   without the watchdog):

    Win2k
      MS Internet Explorer 5.5 - ok
      Netscape 4.61 - ok
      Netscape 4.79 - ok
      Netscape 6.2.2 - user gets no response
                     - old: ntop webserver hung and must restart ntop!!
      Opera 6.03 - user gets a partial response
                 - old: browser says "setting up secure connection" and
                        never continues, but ntop's webserver is ok
                        (SOMETIMES you get SSL errors in log, esp.
                        if you cancel the browser)

    Linux
      Konqueror 2.2.2 - ok
      Mozilla - 1.0 - ok
      Netscape 4.78 - ok
      Galeon 1.2.5 - almost complete response, browser session is toast
                     (must restart)
                   - old: user gets nothing, but the ntop webserver is ok
      Opera 6.0B1 - user gets a partial response, but browser session is ok
                  - old: browser says "setting up secure connection" and never
                         continues, but ntop's webserver is ok.


--------------------------------------------------------------------------------
-----Running - Web Server - Security--------------------------------------------
--------------------------------------------------------------------------------

Q. I'm being prompted for a userid/password, what do I enter.
A. The default admin userid is 'Admin' (without the quotes) and the password
   is whatever you set on the 1st run of ntop (look in this FAQ for
   --set-admin-password).


Q. Why create Userids (beyond the Admin id created by --set-admin-password)
A. Multiple users allow you to control who can alter ntop's performance and/or
   view specific information. If you look on the "Admin" tab, you will see that
   you can create additional users and also control which URLs can be executed
   by whom.

   Userids could allow, for example, an ISP to allow users to access SOME
   network performance statistics, but not the proprietary stuff...

   Suppose you want to restrict who accesses the Multicast statistics page,
   multicastStats.html.

   ntop uses terminal wildcards matching the names, so multicast is treated as
   multicast* and matches multicastStats.html plus any other name beginning
   multicast...

   howto:

   1st add a new user
   2nd add "multicast" to the list of controlled screens and allow admin
       and the new user to access it (note the * wildcard is automatically
       added)

   Try and access the screen and you are prompted for a userid/password...

   Look in http.c for all the names and #defines used...

Q. So, How do I restrict access to the main http or https ntop web page?
A. To stop everyone from logging into ntop, do the following:
   Select ADMIN tab
   Select URL's then "Add Url"
   Don't fill in anything (the wildcard * is implied)
   Select only the users who you want to authorize (hold control key and
   click on user to add more users if you added users)

   and click on "Add Url"

   You will see URL '*' is added, e.g.

     'showU*'
     '*'
     'shutdown*'
     'deleteU*'
     'modifyU*'

   Then only users who know the user id and password (remember to keep the .db
   file secure!) will have access.

Q. How good is the default security ntop provides through the web server.
A. Good question...

   The default ntop configuration is not appropriate if you put ntop in a
   publicly reachable location.  We assume that ntop is either running in a
   small, trusted, LAN or that you've used other tools such as firewalls to
   protect the ntop web server.

   The userid/password scheme is good enough to prevent you from accidentally
   shutting down ntop or getting into 'dangerous' places.  But that's really
   all it does.

   Also, the security of the ntop web server is only as good as the security of
   the passwords file, ntop_pw.db - only the ntop -u userid should be able to
   read from or write to it.

   Read man crypt for information on the security of the encrypted password.


Q. The plugins aren't very secure.
A. True.

Q. How do I prevent users from turning plugins on/off?
A. The default configuration of ntop does not protect the plugin pages - no
   password is required to access showPlugins.html.

   This allows any user who can connect to the ntop web server to view reports
   FROM the plugins, but also allows them to make plugin configuration changes.

   This may not be desirable.  You may wish to add additional URLs to the
   Default list of those which require entry of a userid/password.

   You can prevent unauthorized individuals from turning plugins on/off by
   adding this URL: "showPlugins.html?" to the list via Add URL.

Q. Ok, but they can still get into the configuration pages and change things.
A. Yes.  Add the following URLs to the controlled list:

        plugins/sFlow*
        plugins/netFlow*
        plugins/rrdPlugin*

    This will keep everybody who doesn't know the userid/password out of the
    configurable plugins.  Unfortunately, it will also prevent them from seeing
    the rrd graphs, because those are created out of the rrd plugin.

A. Instead of plugins/rrdPlugin*, create these:

        plugins/rrdPlugin?d*
        plugins/rrdPlugin?h*
        plugins/rrdPlugin?i*
        plugins/rrdPlugin?r*

   It's still not perfect, the reasons why are left as an exercise for the
   user.


Q. I created a new user, adminp for administering the plugins and ntop is
   also accepting the admin userid/password.
A. Yes, the matching of userid's isn't too swift.  It's better to make sure
   that there are NO common initial substrings among ANY of the userids.


Q: How can I use Apache (with it's security, etc.) to serve up ntop pages?
A: (Toby Johnson [public@tobiasly.com], Sun 10Nov2002)

   A while back, I had written about the possibility of configuring ntop to use
   only relative URL's, in order to facilitate proxying ntop's web interface
   through Apache. I have decided it's easier to simply use Apache's ability to
   rewrite ntop's URL's when necessary. So, based on my experience, here is a
   mini-HOWTO on how to proxy ntop through Apache.

   ------

   Proxying ntop's web interface through a secure Apache virtual host is a
   convenient way to make use of any existing security measures you may already
   have. In my case, I wanted to be able to access ntop from anywhere outside
   my LAN, but opening another port on my server for ntop's dedicated web
   server wasn't an option.

   I already had a password-protected, secure web server that I use for admin
   purposes -- I'll call it https://admin.tobiasly.com. I wanted ntop's web
   interface to appear as a subdirectory under this host:
   https://admin.tobiasly.com/ntop/ .

   Here's how to configure such a setup. Change the server names and ports to
   match your own. I'm assuming that you already have a working, secure Apache
   virtual host (using HTTPS).

   First, pick a port for ntop's HTTP server. I'll use 15123. You won't need
   ntop's built-in HTTPS server, since you're proxying its content through a
   pre-existing Apache HTTPS server. Configure ntop to start with the correct
   HTTP port, and with HTTPS disabled. Something like "ntop -d -w 15123 -W 0".
   (See the ntop man page for more startup options.)

   Now, you need to tell Apache that anything under the /ntop/ URL should be
   proxied to the ntop web server. In my case, the Apache server is running on
   the same machine as ntop, so it's just a proxy to a different port on
   localhost. In your Apache secure host configuration, add a line like this:

      ProxyPass /ntop/ http://localhost:15123/

   Now, whenever Apache receives a request for something like
   "https://secure.tobiasly.com/ntop/home.html", it will proxy this request to
   the location "http://localhost:15123/home.html". Ntop will take it from
   there, generate the web content, and pass the result back to Apache. Then
   Apache passes that result back to the original client.

   It's important to note that you don't need to open port 15123 to the
   outside, since the connection actually goes through your existing Apache
   port, and then is transparently proxied by Apache on the server itself. Of
   course, you don't even have to run ntop on the same machine; as long as the
   Apache server can connect to ntop's port, it'll work.

   This is not the same as URL redirection. As far as your web browser knows,
   everything is going through https://secure.tobiasly.com/ntop/. The Apache
   server does all the proxy work behind the scenes, and simply serves up the
   results to the requesting client. And since the "outward-facing" server is
   Apache instead of ntop, you'll be using your existing Apache secure server
   certificate, instead of ntop's ntop-cert.pem.

   Everything appears to work OK at first, but we quickly run into a problem:
   some of the URL's that ntop generates are absolute. For example, to draw bar
   graphs, ntop's web pages will request the image "/gauge.jpg". This would
   translate into "https://secure.tobiasly.com/gauge.jpg". Also, host info
   pages are absolute. If I click on the host "10.1.2.3", it tries to take me
   to the page "https://secure.tobiasly.com/10.1.2.3.html".

   This is a big problem, because unless the URL is underneath the /ntop/
   directory, Apache doesn't know that it needs to proxy the request to ntop,
   and you get broken links. Luckily, Apache has the Rewrite module that lets
   us fool with requested URL's. In order to get the required URL's rewritten,
   add the following to your Apache secure virtual host configuration:

      RewriteEngine On
      RewriteCond %{HTTP_REFERER} tobiasly.com/ntop
      RewriteCond %{REQUEST_URI} !^/ntop
      RewriteRule ^/(.*)$ http://secure.tobiasly.com/ntop/$1 [L,P]

   In English, this basically says "If I get a URL request that comes from a
   page that has tobiasly.com/ntop in it, and that request doesn't begin with
   /ntop, rewrute the URL to begin with http://secure.tobiasly.com/ntop/, and
   pass this rewritten URL to the Proxy engine." At this point, the Proxy
   engine will see that it is getting a URL that begins with /ntop/, and
   correctly pass it to the ntop web server. Rewriting the request to begin
   with HTTP instead of HTTPS may seem incorrect, but since that URL will be
   handed directly to the Proxy engine, it can't be HTTPS or ntop's web server
   will not recognize it.

   Now, you should be able to simply connect to
   https://secure.tobiasly.com/ntop/ , and you're ready to go!


--------------------------------------------------------------------------------
-----Running - Web Server - i18n------------------------------------------------
--------------------------------------------------------------------------------

Q. Is ntop localized for language x? (i18n)
A. No.  ntop wasn't really written with i18n in mind.

   Most of the text is generated in-line, on the fly.  Plus ntop must
   dynamically support multiple locales simultaneously.

   However, beginning with v2.1.56 (2.2 development release), there is limited,
   optional, i18n support in ntop.

Q. So, what internationalization (i18n) support does ntop provide.
A. The key word is LIMITED

   This only applies to the pages that are pulled from .html files, NOT those
   created internally.  This includes the menus and the few static text pages,
   but none of the pages with interesting data on them.

   The localized pages must be placed in parallel directories to the existing
   html ones.

   For example, if ntop is installed in /usr/share/ntop, the html files are in
   /usr/share/ntop/html.

   To support them Canadians, then, you would need to create a
   /usr/share/ntop/html_en_CA AND that locale would need to be installed on the
   ntop host system.


   Note that there are NO i18n files distributed with ntop (yet!)


   At ./configure time, you enable support via --enable-i18n.  ntop MUST be told
   how to find the locale files.  In ./configure, a "standard" location is
   defined per OS.

   (Initially only the value for FreeBSD is populated).  All others assume the
   "default", /usr/lib/locale.  If that isn't right for your OS, then you MUST
   use the optional parameter --with-localedir= to tell ntop where to find the
   files.


   At run time, ntop scans the host for the installed locales (locale -a should
   - on most systems give you a list) and checks if a comparable html_cc_XX
   directory exists.

   This builds a list of supported languages, which (along with i18n status) is
   shown on the configuration pages, info.html and textinfo.html.


   When an http request is made, your browser sends a list of languages it is
   willing to accept in the http Accept-Language: header.

   (check View | Internet Options | Languages in IE to see what you're sending)

   For example,

       Accept-Languages: en_US, en

   Means that you prefer US English, but will accept any English dialect if US
   English isn't available.


   Be aware that the locale settings and Accept-Language settings are not well
   standardized, nor common and may not necessarily map very cleanly.  You
   should see what's defined (perhaps it's locale 'german' instead of 'gr') and
   make or link directories as necessary.  You can always create the directory
   you tell ntop to use via --with-localedir= in the /usr/share/ntop structure
   and create links from there to the real locale directories!


   Limits in the per-request and total # of languages to support are in
   globals-defines.h


   Because of directory structure limits, a lack of interest in multiple
   character sets, etc. the locale and accept-language headers are coerced into
   a common format:

   locales are                  ll[_XX][.char][@modifier]

       ll - language, usually the 2 character ISO abrev., such as us, it.
       XX - dialect (often a country), such as CA or US (en_US != en_CA)
       char - character set (we sort of assume UTF-8)
       modifier - euro

   Accept-Language: values are  ll-XX or ll or ll-*


   Once the user makes a request, each page pulled is checked:

       1. For each of the Accept-Language values.
       2. For the ntop host locale value.
       3. In the ntop default (English) set.

   These checks are performed for each of the libraries specified in the config
   value (CFG_DATAFILE_DIR).

Q. What about the country flags.
A. There are other sets available on the web, of different quality and size.
   Rather than chase down permissions and rights, we'll stick with what we have
   but let you know here of other options.

   If you find a set you like, just download them and replace the xx.gif files 
   in .../html/statsicons/flags

   Much better, but about 4x larger:
      http://users.skynet.be/hermandw/fl/smalgifs.html

   Same height, but wider (so they look better):
      http://www.kidlink.org/www/miniflags.html


Q. What pages can be customized?
A. ls /usr/share/ntop/html/*.html (or wherever the ntop pages are installed):

   frameset
   --------

       Note: There is no real benefit, except maybe the title in index.html so
             you can see that it really does work!

   index.html
   index_inner.html
   index_left.html
   index_top.html
   top.html

   Navigation
   ----------
   About.html
   Admin.html
   All.html
   FibreChannel.html
   IP.html
   Local.html
   SCSI.html
   Summary.html

   Misc data
   ---------
   Copyright.html
   dump.html
   faq.html - lots of work
   help.html - good candidate
   ntop.html - splash page
   ntophelp.html
   privacyNotice.html


   Also, remember that a file overrides ntop's internal page generation, so you
   can also use this facility to override ANY of ntop's pages and return a
   customized page (perhaps you don't want users seeing them?).


--------------------------------------------------------------------------------
-----Running - Web Server - p3p-------------------------------------------------
--------------------------------------------------------------------------------

Q. What's up with P3P?
A. P3P is a W3C recommendation - http://www.w3.org/TR/P3P/ - for specifying how
   an application(typically a web site) handles personally identifiably
   information.  What information the site collects and what it does with the
   information.

   p3p is pretty complex!  There are basically two ways to enable an application
   for p3p. One is to add another HTTP header, P3P:.  The second is to support a
   well-known file location, /w3c/p3p.xml (like robots.txt).

   Browser support is pretty spotty, as is web site adoption.

   Some 3rd party browsers have some support...  up to CrazyBrowser which claims
   "full support", whatever that means...


Q. So why put P3P into ntop?
A. It's coming.  P3P is gradually making it's way into the top web sites -
   right now (Dec2002), for example dell.com supports it and yahoo.com doesn't.


Q. Ok, but what's that got to do with ntop?
A. Since ntop collects personally identifiable data in it's access log (-a
   option) and it's various reports and makes those available to pretty much
   anyone in the default configuration, it's probably not a bad idea to OFFER
   some support.  Especially if you're running ntop at a site that has started
   to support P3P, if you don't have a mechanism for your own policies
   you'll have to adhere to corporate ones.  And that could require massive
   changes to ntop.


Q. IE6?
A. Since ntop doesn't send the P3P: header, IE6 ignores ntop wrt p3p.  Besides,
   IE6 uses p3p to block 3rd party cookies.  If you want to see the p3p stuff,
   it's view | privacy report in the menus.  If the site's policies don't match
   your settings, there will be a red "do not enter" icon in the third box on
   the bottom right of the IE6 window - double click on it to see the report.
   See http://support.microsoft.com/default.aspx?scid=KB;en-us;q293513


Q. Mozilla
A. Unknown if it's enabled by default.  Mozilla had support, ripped it out in
   Feb 2002 and put a new version back in.

Q. Other browsers
A. See their home pages or search the web.  One that I know that claims "full
   support" (whatever that means) is at http://www.crazybrowser.com/


Q. Privacy Bird?
A. A browser-add-on, AT&T's privacy bird (http://www.privacybird.com/), that I'm
   playing with is a lot more aggressive in supporting p3p.  If Privacy bird
   doesn't see the P3P: header, it then requests the "well known" file,
   /w3c/p3p.xml file and gets nailed by ntop as a hostile application, since we
   don't have support for returning .xml files (yet).


Q. So when & how does ntop support p3p?
A. A patch in the cvs on 4Dec2002 adds minimal support for p3p -- specifically:

      1) ntop will respond to queries for /w3c/p3p.xml and ntop.p3p -- returning
         the ntop.p3p file, IF ONE EXISTS.

         If the file does not exist, a 404 error is generated (vs. pre 4Dec2002
         behavior of adding the address to the myGlobals.weDontWantToTalkWithYou
         list).

      2) New parameters, --p3p-cp and --p3p-uri allow you to return the P3P:
         header with either or both of the parameters (cp="" or policyref="")
         set.

         ntop doesn't validate the text in any way other than the usual
         stringSanityCheck().

   This allows me to run the Privacy Bird and still talk to ntop.  I'll admit
   that option #2 is speculative, since I really don't have much of a way to
   test it.


Q. But there isn't a sample .p3p file provided.
A. Right.

   Please note that there is no sample file provided.  This is not an oversight.

   After careful consideration, I am not providing one.  The reason is that a
   .p3p file is intended to be a legal contract between your site and your users.
   While I could provide a default file that has the right tags - as I
   understand p3p - for the data ntop collects and stores, I don't want the
   responsibility and/or liability.

   If anyone wants this "sample p3p file", I will make it available for a fee,
   Provided your organization - through an appropriate officer, in writing:

      1) Acknowledges that Luca Deri, Burton Strauss and other developers of
         ntop have no liability for any use(s) you make of the sample p3p file
         or anything you derive from it.

      2) You will defend us - at your expense - from any lawsuit, arbitration
         proceeding, etc. filed in conjunction with your use of the sample p3p
         file.

      3) You will pay any judgments, legal expenses, etc. related to any
         lawsuit, arbitration proceeding, etc. in conjunction with your use of
         the sample p3p file.

   Since your legal department would be nuts to agree to that I doubt it will
   come up.


Q. So How do I create a .p3p file?
A. There are tools available to create p3p policy files - search the web for
   'p3p editor'. One that I've used is a zero cost albeit beta tool, p3peditor
   from IBM (http://www.alphaworks.ibm.com/tech/p3peditor).


--------------------------------------------------------------------------------
-----Networks, Network cards and Networking-------------------------------------
--------------------------------------------------------------------------------

Q. My security people won't let me run in promiscuous mode.
A. Tough...

   Or, use the -s option and accept the limitations...

   Ask them "honestly, what is the problem" - other than having an interface
   in promiscuous mode is a signature of a sniffer and security folks look for
   unauthorized sniffers?

   ntop needs promiscuous mode so that it sees the full range of traffic.  Any
   similar product will do the same thing.

   If the security people think traffic on the wire is secure, they're wrong!
   Face facts - just about every Windows user, except for 2K/XP Pro (and then
   only if TBTP have especially locked them down) can install the windows
   version of tcpdump...

   If it's a checklist item, just gen up a form to "authorize" it, have the
   boss and VP/CIO sign it and give it to them.


Q. What is Ethernet and TCP/IP and how do they differ?
A. Both are protocols - that is the definition of how
   to interpret bits on wires (or in packets) into
   meaningful conversations.

   Ethernet is the lower level, wire (or wireless) protocol,
   concerned with moving the physical bits of data.

   TCP/IP is the higher-level protocol, which explains
   how to interpret the block of bits (frame).

   TCP/IP uses a familiar 32-bit "IP" address, e.g.
   192.168.0.1.

   Ethernet uses a less familiar, 48 bit unique to the NIC
   (some times called "burned in") address, e.g.
   00:40:05:DE:AD:00.  This is called the MAC (Media
   Access Control) address.

   FYI: The official IEEE MAC address lookup is at
       http://standards.ieee.org/regauth/oui/index.shtml
   (Look up the first six digits, separated by -s, e.g. 00-40-05)


Q. OK, but how is stuff sent from my computer to, say, Yahoo!?
A. First off, your computer does a lookup - using a service
   called DNS (Domain Name Service) to convert www.yahoo.com
   to a numeric value, such as 66.218.71.80.

   Then it builds a collection of characters that says send
   this data from me, 192.168.0.1 to Yahoo at 66.218.71.80.
   This is called a packet.  That gets wrapped in an Ethernet
   frame (addressed from 00:40:05:DE:AD:00 to the MAC address
   of the local gateway router, 0:d0:9e:6:38:00 and squirts it
   out the router.

   Packets are forwarded step by step along a path from you
   to Yahoo by computers called routers.  This is done based
   on the 32 bit IP address and the router's knowledge of the
   network.

   Each router sees a Ethernet frame addressed to it (by
   MAC address), checks the TCP/IP address to figure out
   where to send it next, re-wraps the TCP/IP packet in a new
   Ethernet frame (with the from MAC as it's own and the to
   MAC as the next hop).

   This happens until the TCP/IP packet reaches the final
   segment (the last router).  Once it reaches a router that
   knows it has addresses 66.218.71.0-66.218.71.255 on one
   of it's interface, the routing stops using the TCP/IP
   address.

   The last hop is done (like each intermediate hop - at the
   lowest level) based on the MAC address!  Specifically, the
   last router does an "ARP" (Address Resolution Protocol") query,
   to find out "Who Has" address 66.218.71.80.  The NIC responds
   with it's MAC address:

      arp who-has www.yahoo.com tell router
      arp reply www.yahoo.com is-at 0:d0:9e:6:38:00

   And the packet is routed to that address.

   Alright, that's a bit simplified, but see Douglas Comer,
   "Internetworking with TCP/IP, volume I", page 25 and 73ff.


Q. Tell me more.
A. OK, gang time to teach Ethernet & TCP/IP basics one more time.
   With pictures...

   Suppose you have a network that looks like this (we'll use 
   impossible addresses 288 - just pretend it's ok):

              (ext) 288.1.1.1     (int) 288.2.2.1
                            +-----+
     World+Dog ------------ + ISA + ------- LAN ----- WS 288.2.2.2
         |                  +-----+
         |                     |(dmz) 277.1.1.1
        me 299.0.0.1           \----------- DMZ ----- MAIL 277.1.1.2
                                                  \-- WEB  277.1.1.3

   ISA can be acting solely as a router or it can be acting as a NAT device.
   That's irrelevant, so we assume it's not.

   I send you a packet.  It travels the Internet and arrives at your 288.1.1.1
   the ISA(router) with src=299.0.0.1, dst=288.2.2.2.

   Like every router along the way, ISA(router) looks at the destination
   address and realizes it has to route the packet on to 288.2.2.2.  So the
   ISA(router) sends the packet on, out the best interface to reach 288.2.2.2.

   Remember, however, the TCP/IP packet is wrapped in a lower level (Ethernet)
   packet at the wire level.  Read your TCP/IP and Ethernet standards - the
   actual delivery of packets over links is this Ethernet level and it uses the
   48 bit MAC address.

   This "Ethernet" packet is actually what travels hop to hop to hop (you can
   even see these headers if you have visibility to the traffic - it's called
   the link level header by tcpdump and you'll see the 48 bit MAC addresses if
   you use the -e parameter).

   In order to be able to handle the Ethernet level signaling, each router
   rewrites the packet so that the 48 bit source MAC address it it's own
   (from-router that is) and the destination MAC address is the one that
   from-router has in it's tables for to-router (the next hop).

   So the packet looks like this, where the srcMAC and dstMAC get rewritten
   each hop, so that the routers on both ends of the link know whom it's
   addressed to:

   Hop1 (srcMAC=00:00:00:aa:aa:aa dstMAC=00:00:00:bbbbb:bb frame=IP
   data=(src=299.1.1.1 dst=288.2.2.2 data="Hi!")
   Hop2 (srcMAC=00:00:00:bb:bb:bb dstMAC=00:00:00:cc:cc:cc frame=IP
   data=(src=299.1.1.1 dst=288.2.2.2 data="Hi!")
   Hop3 (srcMAC=00:00:00:cc:cc:cc dstMAC=00:00:00:dd:dd:dd frame=IP
   data=(src=299.1.1.1 dst=288.2.2.2 data="Hi!")

   Notice how the TCP/IP stuff isn't changed.  But the MAC address is.

   At each hop, the NIC card, operating at the Ethernet level, sees its own MAC
   address and knows to accept the packet. It passes it up the protocol stack,
   where the next layer (TCP/IP) realizes it needs to be routed further on...

   Ultimately, the packet gets delivered to some service listening on your WS.

   Here's a packet capture to show you:

   # tcpdump -Xx -c1 -i eth0 -e
   tcpdump: listening on eth0
   11:49:10.809890 0:3:47:b1:xx:xx 0:e0:18:b4:yy:yy ip 118:
   tigger.ssh > zebra.2714:
   P 1824243567:1824243631(64) ack 2328789523 win 11792 (DF) [tos 0x10]
   0x0000   4510 0068 b305 4000 4006 b1e4 c0a8 2a24        E..h..@.@.....*$
   0x0010   c0a8 2a21 0016 0a9a 6cbb bf6f 8ace 8213        ..*!....l..o....
   0x0020   5018 2e10 ee1a 0000 469a 3e34 eda7 549e        P.......F.>4..T.
   0x0030   0ec4 4847 8983 fb4f 65ea 5c3e 0bbe c325        ..HG...Oe.\>...%
   0x0040   7db8 9954 dae1 55b6 54f9 cdfd ac07 a2b5        }..T..U.T.......
   0x0050   ce4f

   So this says, the packet came from tigger (MAC address 0:3:47:b1:xx:xx) ->
   zebra (MAC address 0:e0:18:b4:yy:yy)
     Within that is the tcp/ip packet, from c0a82a24 -> c0ae2a21
   (192.168.42.36 -> 192.168.42.33)

   Here's another one, from tigger -> router.

   # tcpdump -Xx -c1 -i eth0 -e host 192.168.42.1
   tcpdump: listening on eth0
   11:52:48.712750 0:3:47:b1:aa:aa 0:d0:9e:6:bb:bb ip 72:
   tigger.32782 > homeportal.gateway.2wire.net.domain:
   41356+ A? cvs.ntop.org. (30) (DF)
   0x0000   4500 003a bf7e 4000 4011 a5be c0a8 2a24        E..:.~@.@.....*$
   0x0010   c0a8 2a01 800e 0035 0026 ce2e a18c 0100        ..*....5.&......
   0x0020   0001 0000 0000 0000 0363 7673 046e 746f        .........cvs.nto
   0x0030   7003 6f72 6700 0001 0001                       p.org.....

   (0035 = port 53, so it's a dns query)

   And one more, from cvs.ntop.org -> tigger (which has to have passed through
   the router)

   # tcpdump -Xx -c1 -i eth0 -e "not src net 192.168.42.0/24"
   tcpdump: listening on eth0
   11:53:39.688806 0:d0:9e:6:bb:bb 0:3:47:b1:aa:aa ip 69:
   195.31.151.66.cvspserver > tigger.42964:
   P 2566885448:2566885451(3) ack 2903504154 win 24616 <nop,nop,timestamp
   389837493 85533859> (DF)
   0x0000   4500 0037 5f3c 4000 2906 ad56 c31f 9742        E..7_<@.)..V...B
   0x0010   c0a8 2a24 0961 a7d4 98ff 9048 ad0f f51a        ..*$.a.....H....
   0x0020   8018 6028 279a 0000 0101 080a 173c 72b5        ..`('........<r.
   0x0030   0519 24a3 6f6b 0a                              ..$.ok.

   See the MAC address?  It's the routers, not cvs.ntop.org's

Q. Why do I care?
A. Because that's sometimes a problem you'll have with ntop. See ntop is 
   nosy, so it puts the interface into promiscuous mode, where every 
   packet - addressed to the ntop host or not - is processed.  Now the 
   next layer up, the tcp/ip layer will throw away any 'junk' (You can 
   just hear it going tisk, tisk, tisk).  But libpcap can intercept them 
   at the Ethernet level and feed them to ntop...

   For REMOTE hosts, ntop uses the IP address as it's the only valid data.
   But for LOCAL hosts, ntop prefers to use the MAC address as a way of
   resolving multi-homed or multiply addressed hosts.

   See if you have two IP addresses assigned to the same host on your local
   network, say 192.168.42.42 and 192.168.42.43, how is ntop to tell they're
   the same host?  Well, the MAC addresses are the same...  (for some
   manufacturers, e.g. Sun, ALL of the interfaces on a host use the same MAC
   address).

   read getHostInfo() in hash.c.

      /* This is a local address hence this is a potential
         multihomed host. */

      if((el->hostIpAddress.s_addr != 0x0)
         && (el->hostIpAddress.s_addr != hostIpAddress->s_addr)) {
         isMultihomed = 1;
         FD_SET(FLAG_HOST_TYPE_MULTIHOMED, &el->flags);
      }

   i.e. if the address we've stored for this host doesn't match this one, 
   it's multihomed.

      } else if((hostIpAddress != NULL)
                && (el->hostIpAddress.s_addr == hostIpAddress->s_addr)) {
         /* Spoofing or duplicated MAC address:
            two hosts with the same IP address and different MAC
            addresses
          */

         if(!hasDuplicatedMac(el)) {
            FD_SET(FLAG_HOST_DUPLICATED_MAC, &el->flags);
            ...
         }

         setSpoofingFlag = 1;
         hostFound = 1;
         break;
      }

   If the addresses DO match, we've had two MAC addresses, so this is being
   spoofed.

   etc.


Q. So why do I get bad output?
A. If, somehow, you've confused ntop - for example telling it that 
   277.1.1.0/24 in the ascii art example (above) is local, then ntop 
   is going to believe you.  And it will see a packet with the 
   277.1.1.1 IP and a MAC address. And use that.  Only it's not the MAC 
   address of the MAIL host, it's really the MAC address of ISA.

   No matter, ntop doesn't know this -- all it sees is the packets and 
   the data you gave it.  So later on, when it sees a packet with the 
   same MAC address, but a different IP, well, it will assume that it's 
   the same host... and the data will all be lumped together.


Q. Does that explain why I'm seeing xxxx as multihomed?
A. Maybe.  Remember - it only takes one packet, not even an ack, for
   ntop to create a host record.  If that's wrong, it will carry 
   forward - and you'll probably see the host tagged as 'Multihomed'
   when correct packets show up.  Say:

      Host 1: IP 192.168.1.1 MAC 00:00:00:aa:aa:aa
      Host 3: IP 192.168.1.3 MAC 00:00:00:cc:cc:cc

   If somebody has the incorrect hosts table, dns, cached, whatever 
   but has the info that Host 1 is 192.168.1.3 and is on the same 
   subnet, then it will send a packet where the Ethernet layer and
   the ip are nonsense.  But because it's on the same wire, the ip 
   is ignored:

   (Ethernet from:00:00:00:dd:dd:dd to:00:00:00:aa:aa:aa)
       (tcp s=192.168.1.4 d=192.168.1.3)

   ntop will read both out of the packet and create the association

      192.168.1.3=00:00:00:aa:aa:aa

   Since it doesn't know better.

   Then when it sees

   (Ethernet from:00:00:00:ee:ee:ee to:00:00:00:cc:cc:cc)
      (tcp s=192.168.1.5 d=192.168.1.3)

   It will create the multihomed association...


Q. So what's a hub vs. a Switch
A. A hub is a device that links a bunch of computers together
   at the wire (Ethernet) level.  Logically, Ethernet is a bus,
   that is everybody sees all the traffic, just like cars crossing
   under a highway bridge.   Physically, Ethernet is wired like
   a star - with all the wires coming back to a central "hub".
   The hub is just the device that makes the electric star look
   like a shared bus.

   Switches and Hubs operate at the Ethernet level, not TCP/IP.

A. Watch out for 'Switched hubs', which are hubs that include an
   internal switch between 2 or more segments (for example, BUT
   NOT LIMITED TO a 10BaseT and 100BaseT) segment.  These are hubs
   within a segment, but switches across segments.  ntop may not
   see the traffic you expect if you have a 'switched hubs' and
   manufacturers are pretty bad about marking them. See
   http://article.gmane.org/gmane.linux.ntop.general/5081


A. A switch is a smart hub.

   Switches improve performance by creating a virtual Ethernet
   bus for the duration of the packet that joins JUST the source
   and destination ports.

   A switch operates via an internal table of MAC addresses.
   It learns (or is programmed) that 0:d0:9e:6:38:00 is on
   port 1, while 00:40:05:DE:AD:00 is on port 3.

   A packet coming in port 1, destined for 00:40:05:DE:AD:00
   is sent out ONLY port 3.

   If the switch doesn't know (or the packet is a broadcast),
   it gets sent out all ports.

   This doesn't make for MORE bandwidth, but it does use it
   more efficiently.  That is in addition to the session between
   ports 1 and 3 at 100Mbps, a second, simultaneous 100Mbps
   session can occur between ports 2 and 4.


Q. How do I use ntop in a switched network?
A. First off, you need to be or have the support of
   your network administrator.  (Yes, you can do something
   called "ARP poisoning" to - maybe - get the switch to send
   you all the traffic, but that's beyond this FAQ... STFW)

   Many switches (although not the USD$50 cheap "workgroup" units)
   have a special port or mode, where by all the traffic for the
   entire network gets copied out that port, in addition to the
   normal switch action.

   When you invoke the monitoring mode (called span, mirror, monitor,
   analysis, etc.), you are forcing the entire switch bandwidth out one
   port.  This may exceed the bandwidth of the port.  100Mbps+100Mbps
   >> 100Mbps!

   Traffic that is being sent to the monitoring port in excess of the
   capacity of that port is usually dropped.  It should NOT slow down
   the switch on other ports.

   Some switches have some buffering capability and it *may* be able to
   keep up with an occasional burst of traffic, as long as the average
   is below the port capacity and the buffer isn't exceeded.

   See, for example, http://www.cisco.com/warp/public/473/41.html#archXL.

   One list of switch manufacturers is the document is titled "REFERENCE:
   Configuring a Switch to Monitor All Traffic" from Elron Software. (The
   URL is long, do a Google search for "site:elronsoftware.com wi6038").


--------------------------------------------------------------------------------
-----Dumping data---------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Can I use ntop from php/perl?
A. Yes you can. Please see the www directory under the ntop source tree.
A. Look at the Admin | Dump Data page.


--------------------------------------------------------------------------------
-----rrd (myrrd)----------------------------------------------------------------
--------------------------------------------------------------------------------

Q. How do I save data between runs?
A. Use rrd.


Q. What's rrd?
A. There's a 12 page writeup on what rrd is and what ntop does with it.
   A pdf version is posted at SourceForge in the "Documentation" release.
   Download it and read it.

Q. Do I gotta?
A. Yup - the pretty pictures won't work in this FAQ.


Q. What's myrrd?
A. ntop includes a frozen and slightly patched version of rrd 1.0.42 in the
   ntop source tree.  This is called myrrd.

   As of rrdtool 1.0.46 was released 04-Jan-2004.


Q. Where do I get rrd?
A. You should not need to do anything to get rrd (see myrrd, above).

   The home page for rrd is
   http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/

   rpm's are available at
   http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub


Q. What about the multi-threaded development version?
A. Stay away.  I only experimented with it a little bit, but it was not
   stable.  Use 1.0.42.


Q: I've enabled the rrd plugin and there's no data ... there are messages in the log:

        RRD call stack:
        argv[0]: rrd_update
        argv[1]: ...rrd/matrix/12.239.98.199/12.239.181.175/pkts.rrd
        argv[2]: 1037289548:1
        rrd_create(...) error: creating '...': No such file or directory
        rrd_update(...) error: opening '...': No such file or directory

A: Create the rrd directory and make sure that the -u userid has read/write
   access to it (typically /usr/share/ntop/rrd).


Q. rrdPlugin - problem with rrd/myrrd?
A. By default, ntop's Makefile binds the static libmyrrd library to create ntop's 
   rrdPlugin shared library.  THIS IS DELIBERATE so that you use the myrrd library
   and not some other version of rrd that's installed somewhere else on your system.

   Sometimes this causes problems, where there are special tricks required to tell
   (a non gnu ld) loader about static (.a) libraries, which ntop doesn't have in
   the Makefile nor the configureextra files.

   As a SHORT TERM WORK-AROUND, you can TRY this:

      $ cd myrrd

   Edit Makefile --

       all: $(LIBRRD) libmyrrd.so
                      ^^^^^^^^^^^ add this 

   Add these lines:

       libmyrrd.so: $(OBJECTS)
       <tab>ld -shared -o libmyrrd.so $(OBJECTS)

   Now do make.  You should see a libmyrrd.so file.

   The main 'make' should now complete.  Copy that libmyrrd.so file where the other
   ntop library files are, and it MIGHT work.

   However, the whole idea behind having a static libmyrrd.a is to prevent version
   conflicts and use a stable version of rrd.

   The right fix is to get the configureextra/<whatever> file changed.


--------------------------------------------------------------------------------
-----netFlow and sFlow----------------------------------------------------------
--------------------------------------------------------------------------------

Q. How do I access netFlow or sFlow data from ntop?
A. You need to configure ntop as a listener (it can also be a collector, but
   that data shows up in the receiving interface, not under netFlow/sFlow).

   First, use the appropriate plugin to set the parameters - basically the port
   you want ntop to listen on.  Then, using the Admin | Set Interface menu item,
   switch ntop to report on the sFlow/netFlow pseudo-device (NetFlow-device or
   sFlow-device).


Q. Is there any parameter to set to tell sFlow which interface/ip address
   to use when exporting the flows?
A. No. All ntop does is send a packet to the network addressed to the
   destination you request.

   Typically if the ntop host is multihomed, the routing service will pick the
   the MOST SPECIFIC route with the lowest metric # that is selected.  E.g., if
   this is the routing table:

   Kernel IP routing table
   Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
   192.168.2.0    0.0.0.0          255.255.255.0   U     0      0        0 eth0
   192.168.2.129  0.0.0.0          255.255.255.128 U     0      0        0 eth2
   192.168.2.146  0.0.0.0          255.255.255.255 U     0      0        0 eth1
   192.168.2.146  0.0.0.0          255.255.255.255 U     1      0        0 eth2
   127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
   0.0.0.0         192.168.2.1     0.0.0.0         UG    0      0        0 eth0

   A packet to 192.168.2.146 goes via eth1 (equally specific routes, so it
   chooses based on the metric 0 vs. 1)

   A packet to 192.168.2.145 goes via eth2  (192.168.2.129/25 is more specific
   than 192.168.2.0/24)

   A packet to 192.168.2.46 goes via eth0

   A packet to 10.1.1.1 goes via eth0 (the gateway is the least specific route,
   but it is the best match)

   But how it goes out (and thus the source IP address) is totally up to the OS.
   Just be aware when it gets to the sflow collector, it might have an
   unexpected ip address as the source.


--------------------------------------------------------------------------------
-----netFlow--------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Which versions of netFlow.
A. v5
   And v1/v7/v9 - in that internally a v1/v7/v9 flow is copied to a v5 buffer and then
   processed.  We default/ignore fields that are different.
   And nFlow - similar conversion.


Q. netFlow doesn't work.
A. You MUST make sure the ntop plugin is ACTIVE.  With the change to allow
   setting parameters while inactive, it's easy to miss that last step.  
   If you don't activate the plugin, you'll still have the netflow-device, but
   no data on it...


Q. What's Virtual NetFlow Interface?
A. Be sure and set it.  It's important for pseudo-local classification, which
   affects L R reporting.  You need to set it to the (network) and mask for 
   the netFlow collector.  So ntop knows 'where' the data is coming from.

Q. 'splain some more, Lucy...
A. OK.

   It's best to think of netFlow like this:

   The physical interface which is monitoring the packets is like a
   temperature probe you stick into a roast.

   Even though the display of the data can be right there at the probe, or
   the other end of a (long) wire, or somewhere entirely elsewhere via a 
   wireless connection, the probe is monitoring at the tip.  If it says 145F,
   that's the temperature of the meat - not the oven and not the kitchen.

   Similarly, the netFlow data ntop is receiving is based on the probe
   location.

   So, if you have a router and are monitoring a single interface to collect
   netFlow data, then the ip address you want to give to ntop is that of
   the router interface.

   If you are monitoring a router with more than one interface, you will 
   need to give ntop ONE of those addresses and use the -m | --local-subnets
   option to tell it that the other addresses are also local.


Q. Where is info about netflow?
A. Dale Reed pointed out a good tech doc (no flak, just the formats) for netflow
    V1/5/7:

http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/nfc/nfc_2_0/nfc_ug/nfcform.htm

   (As of Oct2003, it includes v8 and is here:)
http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/nfc/nfc_3_0/nfc_ug/nfcform.htm


Q. How Do I Enable NetFlow Data Export on a Cisco Device?
A. To enable netFlow Data Export (NDE) from a Cisco device to an ntop netFlow
   receiver on port 2055 (default) at address 10.1.1.1:

     ip flow-export destination 10.1.1.1 2055
     ip flow-export version 5

   You may want to designate the source interface, e.g.:

     ip flow-export source Ethernet0

   Enable netFlow on each interface to be monitored. netFlow normally only
   captures data from each incoming packet, so to see traffic in both directions
   netFlow must be enabled on both the incoming and outgoing interfaces. As an
   example, for an Internet access router this would mean enabling netFlow on
   both the internal (e.g. Ethernet) and the external (e.g. ISDN / Frame Relay
   etc) interfaces:

     interface Ethernet0
     ip route-cache flow

     interface Dialer1
     ip route-cache flow


   By default netFlow will only export flow statistics shortly after the flow
   Terminates or when 30 minutes have elapsed. In many environments, you want
   ntop to be a bit more up to date. To change the timeout to five minutes:

     ip flow-cache timeout active 5

   The following 'show' commands are useful for examining netFlow statistics
   directly on the Cisco box and may assist when setting up ntop:

     show ip flow export
     show ip cache flow
     show ip cache verbose flow

   Obviously, there is a lot more to it than this, for more information, see the
   Cisco web site: http://www.cisco.com/go/netflow
                                     (Created by sholmes at snapshot, 02Feb2003)


--------------------------------------------------------------------------------
-----sFlow----------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. What is sFlow
A. The core component of the sFlow toolkit is the sflowtool command line
   utility. sflowtool interfaces to utilities such as tcpdump, ntop and Snort
   for detailed packet tracing and analysis, NetFlow compatible collectors for
   IP flow accounting, and provides text based output that can be used in
   scripts to provide customized analysis and reporting and for integrating with
   other tools such as MRTG or rrdtool.

   Some info:

   http://www.inmon.com/sflowTools.htm
   http://www.faqs.org/rfcs/rfc3176.html

Q. I have activated the sFlow plugin in ntop. But it doesn't seem to
   generate any output based on the collected sflow datagrams.
A. sFlow can be a collector or a receiver or both, depending on the
   settings configured via the plugin.

   If you configure ntop as an sFlow collector, it will use sFlow data
   for generating reports, treating the remote collector(s) as another
   network interface - see Admin | Switch NIC.



--------------------------------------------------------------------------------
-----snapshot, cvs and the Wiki-------------------------------------------------
--------------------------------------------------------------------------------

Q. What was "snapshot"?
A. Snapshot was a community FAQ and documentation resource at
   http://snapshot.ntop.org. It's was also the site of "the snapshots".

   Unfortunately, snapshot seems to have 'Gone to Atlanta' (404), with
   the web server no longer responding in February 2004.

   Nicolai Petri's efforts were appreciated.  But snapshot was an effort
   of love by a gentleman who hasn't had time for the effort in quite 
   some time.


Q. Is there a community resource that replaces (enhances) snapshot.
A. Yes. There is now an ntop Wiki at
     http://www.burtonstrauss.com/pmwiki/pmwiki.php?pagename=Ntop.HomePage
   Whether it remains available depends on whether people contribute.


Q. What was "a snapshot" or "the snapshots"?
A. A snapshot was a dump of the ntop cvs structure, automatically generated
   every day at 5 minutes after midnight (Pisa time).

   Snapshots were named with their creation date, in the form of
   ntop-yy-mm-dd.tgz.

   Snapshots were neither polished nor even "releases". They contained any
   update(s) checked into the cvs during the prior day.  No more, no less.

   cvs checkins (commits) are usually tested by the developer, but perhaps only
   in one (limited) environment.  Occasionally a file is missed or a typo occurs
   and a snapshot wouldn't compile. Snapshots frequently introduced bugs that aren't
   apparent on a quick review.  Snapshots were basically a point-in-time capture
   of the moving development environment.  No more and no less.

   With release 2.2, rapid development occurred after general release and using
   the latest snapshot was your best bet. With 3.0, after a short period of stability
   (releases 2.2b and 2.2c), the development again outpaced the stable version and
   using the cvs again was almost manditory. 

   If the 3.0 release doesn't work, drop by the mailing lists and check the back
   traffic to see if this is a common problem.  If it's not, try the latest
   snapshot or ask for a recommendation of which version is the best to use.

   As with 2.2, we plan to have a cvs branch to allow for 3.0a/b/c
   releases while 3.1 is being developed (3.1.1, 3.1.2, etc.).

   Note that the ntop_2_2_patches branch broke the snapshot extract script, so
   snapshots ARE NO LONGER RECOMENDED OR SUPPORTED.


--------------------------------------------------------------------------------
-----Old answers, but goodies---------------------------------------------------
--------------------------------------------------------------------------------

Q. What is "obsolete/"
A. Obsolete code is code that is no longer being maintained nor part of ntop,
   but it's stuck off in that directory because 1) storage is cheap and 2) it
   might have usage someday and 3) somebody might be interested in resurrecting
   it...

   Code in obsolete/ IS NOT MAINTAINED, even minimally.

   Specifics?  (As of June 2002)

       Various programs and functions which supported "rules" were determined to
       be obsolete were removed from ntop in late March 2002. This included a
       substantial number of lines of code, which was simply removed. Entire
       modules were placed into the obsolete/ directory.
               ntop-rules.8
               event.c
               rules.c
               rules.h
               rules.sample

       Various plugin programs, which were no longer being supported, were
       removed from ntop in late March 2002. These entire modules were placed
       into the obsolete/ directory.
               wapPlugin.c
               rmon.h
               rmonPlugin.c

       Various lines of code (totaling a substantial number, widely scattered
       Throughout ntop), which had provided compile-time selectable support
       (#define ENABLE_NAPSTER) for analysis of the (late) Napster protocol were
       removed on 4Apr2002.  


Q. I get an error, libtool: link: CURRENT `-release' is not a nonnegative
   integer
A. This is an autoconf problem.  It should be fixed. 
   The whole set of messages is typically:

   [...]
   > libtool: link: CURRENT `-release' is not a nonnegative integer
   > libtool: link: `-release' is not valid version information
   > make[2]: *** [libntop.la] Error 1
   > make[2]: Leaving directory `/usr/local/src/ntop'
   > make[1]: *** [all-recursive] Error 1
   > make[1]: Leaving directory `/usr/local/src/ntop'
   > make: *** [all-recursive-am] Error 2

   From Luca:

       "AFAIK this is a bug of autoconf that's not able to expand some macros,
        namely those that contain the version number. The workaround is to
        downgrade to the previous autoconf version."

       Workarounds:
       - downgrade to a stable autoconf version
       - edit all the ntop Makefile(s), add "0:0:0" behind any
         occurrence of "version-info" and "2.0" behind
         "-release".


   Other Solutions:

   1) Burton posted a personal patch to do the above for Makefile.am on
      Thu Dec 20 2001 - "Compile problems with -release". Check the ntop-dev
      archive.

   2) Older versions (e.g. Slackware 7.xx) of Linux installations have older
      versions of automake, which don't exhibit the bug.

   3) (Sean O'Neill) "I cheated a bit and hacked libtool by changing the
      following"

        current="$2"
          to
        current=0

    4) The new ./configure scripts - call it versions after Nov2002 - fix this.


Q, How to Build the (obsolete) RMON plugin
A. Without any guarantees...

    0) Please do NOT use a precompiled UCD package unless you know what you're
       doing.

    1) Fetch UCD-SNMP [See http://net-snmp.sourceforge.net/developer.html]

    2) Compile the stuff as specified as follows :
        [see also http://net-snmp.sourceforge.net/tutorial/toolkit/demon/]
    > ./configure --with-mib-modules="agentx"
    > make
    > make install

    3) Once you this stuff ready you can build the ntop/RMON plugins.

    4) Now start ntop.

    5) The ntop/RMON plugin listens on port 161 (the default SNMP port).
        If you want to test the agent you can use the 'tkMib' tool that comes
        with UCD-SNMP [see http://net-snmp.sourceforge.net/tkmib.jpg].

   NOTE: If you use tkMib do not forget to do "setenv MIBS ALL" (csh shells),
         otherwise you won't see the RMON MIB.

   How to Build the RMON plugin (bis) created 04/03 2002 by pierlo

   The following works with :
     - ntop v.2.0.1
     - UCD-SNMP v.4.2.3
     - openSSL v.0.9.6c

     0. Requirements
     * Make sure that ntop works properly

     * Make sure that openSSL & openSSL libraries are correctly installed
       (no WARNING when ntop is started !). If not, download and install
       the latest versions)

     * fetch & install UCD-SNMP :
     tar -xvzf XXXXX.tar.gz
     ./configure --enable-shared
     make
     umask 022
     make install

     1. uncomment lines around 474 and 1035 in configure.in file :

     dnl>OPTIONAL UCD-SNMP
     dnl>AC_HAVE_HEADERS(ucd-snmp/ucd-snmp-agent-includes.h)
     (...)
     dnl> check for `UCD-SNMP' library by University of California
         [http://ucd-snmp.ucdavis.edu]
     dnl>AC_CHECK_LIB(ucdagent, register_mib, [AC_DEFINE(HAVE_SNMP) 
           SNMPLIBS="-L/usr/local/lib -lsnmp -lucdagent -lucdmibs"],
           , -lsnmp -lucdagent -lucdmibs $LIBS $MORELIBS)

     (remove the dnl> from the AC_ lines)

     * in /ntop/plugins/rmonPlugin.c file, replace :

     line 702 approx. : droppedPackets with droppedPkts
     (there is only one occurence, so it may not be difficult to find)

     line 676 approx. :
     if (header_simple_table (vp,name,length,exact,var_len,
          write_method,numDevices)

     with :

     if (header_simple_table(vp,name,length,exact,var_len,
          write_method,myGlobals.numDevices)

     and :

     long_ret = (long)(device[ifNum]

     with

     long_ret = (long)(myGlobals.device[ifNum]

     (lines 690 to 782 approx., about 13 occurences to change)

     2. launch /ntop/autogen.sh -1

     3. launch /ntop/configure

     Be sure that :

     * there is no warning about openSSL libraries !

     * the following checks are successful :
     " Step 4. Looking for both required and optional system headers....
     (...)
     checking for ucd-snmp/ucd-snmp-agent-includes.h... yes"
     (...)

     "Step 7. Looking for optional GPLed libraries....
     (...)
     checking for register_mib in -lucdagent... yes"

     -> NO warning about openSSL libraries here ! If any trouble,
     check config.log file, which may help to get round the problem...
     Looking at ls.so.conf file and checking libraries pathes may also be
     helpful.

     3. edit /ntop/plugins/Makefile : replace every ocurence of "icmpPlugin"
        with "rmonPlugin".

     4. Launch /ntop/plugins/make

     5. Theoretically, RMON plugin is built and is ready to work ! If you have
        a error message like "undefined symbol..." when ntop started...
        I don't know how to fix it.

     Hope I didn't forget anything... Have fun !

     P-L.


Q. I don't understand -j | --border-sniffer-mode
A. Welcome to the club <grin />

   Quoting from Luca's comment:

   "-j is used when you are starting ntop on a mirrored interface where you
   cannot trust MAC addresses."

   ntop uses MAC addresses for many things to differentiate among machines.
   IP addresses and names can mean many things, but hardware addresses are
   supposed to be unique.  This is usually true, but gets hairy when you
   introduce a switch into the network which is also copying all of the
   packets it sees to a monitoring port.

   Understand how a switch works: In short, a switch monitors the network it
   sees and knows which mac address(es) are on which ports.  When it received a
   packet, it forwards it to only that port. Broadcasts are forwarded to all
   ports.

   So an ntop instance, sniffer, whatever only sees a fraction of the traffic.

   In many managed switches, there is an option for a "repeater" or "monitoring"
   or "spanned" port, which receives all traffic, so that network monitoring can
   be performed there.

   However, When the switch sends out packets on the monitoring port, it must
   rewrite them to be valid Ethernet packets with a valid (i.e. the switch's)
   MAC address and ntop gets confused.

   Note that:

   1. -j usually requires you to specify the local network (-m) as a mirrored
      interface might have a wrong/ip-less/privare IP address.

   2. -j disables some features as TCP session tracking etc.

   In future versions -j will disappear and it will be replaced with more flags
   for better controlling all these options.

   With multiple switches in a hierarchy, you have to place the ntop instance
   or instances carefully, depending upon what you want to monitor.

   For example, most lans have a switch in each area with its uplink connected
   to a backbone switch.  Servers and gateways are then placed off one or more
   backbone ports.  This keeps departmental traffic isolated from each other,
   while making enterprise wide and inter-department traffic feasible.  ntop
   would have to be monitoring the backbone switch, but you would need to be
   aware of what ntop is NOT seeing and place additional monitors.

   For example, you could place additional ntop instances in the departments,
   using the netflow or sFlow plugins to receive flow information from them
   which wouldn't be visible to the backbone instance. (I'll note that I haven't
   actually tried this, myself. -----Burton)


Q. When I run with -j | --border-sniffer-mode, there are different menus.
A. Yes.

   The menus are just html files.  There is a set for regular mode and a
   j_xxx.html set for border sniffer mode.  Why?  Because there are things
   that simply don't work in border-sniffer-mode, so why let the user
   request those pages.


Q. OK, but it changed after 2.1.2 (i.e. in 2.1.50) what are the NEW parameters?
A. There were four new parameters introduced when -j | --border-sniffer-mode
   was removed.  Whether the old parameter really did all the same things as
   the new ones is irrelevant and left to code-archeologists.  What matters
   is what the new ones are and what they do!


Q. What was the -S option?
A. The -S option was the --store-mode option, or the "Persistent storage mode"
   Ntop's internal structures are basically an array of devices (network
   interfaces), which contains an array of hosts (specific machines seen on the
   device.

   So device[0] is the 1st network interface, and device[2] the third.
   device[0].host[0] would be, say, the local file server and device[0].host[1]
   would be a simple host. device[1].host[1] is a completely different set of
   counts from device[0].host[1].

   The -S options tells ntop to store information about a specific host in a
   database from run to run (-S 0 none, -S 1 all and -S 2 only local hosts).

   This is only the count information about the host and does not store the
   information about a device (a network interface). Further, items of
   dynamically allocated storage (the devices name) are not stored.

   Data is retrieved on a subsequent run ONLY when traffic is seen from that
   host after the restart.  (I suppose you could script a ping to each host you
   care about and force the reload that way, but it hasn't been tested...)

   So if you go into the host details (e.g. the 192.168.1.1.html page) you
   should see prior-run information.

   But if you're looking for device throughput to be preserved... nope...

   Also, ntop stores the information during 1) reset and 2) shutdown. So if ntop
   crashes, the persistent data will be lost.

   This option was removed from ntop in the 2.1.52 development version.


Q. What was the -A option, a/k/a --accuracy-level?
A. This option was used to set ntop into various modes which performed less
   processing, in order to handle higher traffic volumes.

   It was added on 14Dec2001 (just before v2.0) and was removed on 11Mar2002,
   although traces survived in usage() until April 2002.

   Note that the CODE remains in initialize.c for use by EXPERTS if necessary.


Q. What was the --no-admin-password-hint option?
A. This option was used to remove the hint text from the administrative password
   entry message box.

   It was added 4Feb2002 and removed 8Mar2002 (the last traces, in usage(), were
   removed on 4Apr2002).

--------------------------------------------------------------------------------
-----Solaris--------------------------------------------------------------------
                                        During the ntop 2.2 development cycle,
                                        we did development/testing under:
                                           Solaris 2.5, 7 and 8 (i86)

                                        During the ntop 3.0 development cycle,
                                        we did development/testing under:
                                           Solaris 8 (i86) and 9
--------------------------------------------------------------------------------

Q. How do I install the ntop package on Solaris?
A. For instance do 'pkgadd -d ntop-2.2-solaris.i386'


Q. What c compiler do I need?
A. Sun's cc or gnu's gcc.

Q. What about /usr/ucb/cc is that the one?
A. No.  cc is software you pay for.  /usr/ucb/cc is a stub compiler and good for,
   recompiling the kernel and absolutely nothing else.

   The cc I mean is Sun's Commercial Compiler.


--------------------------------------------------------------------------------
-----xxxxBSD--------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. When I type 'make' it complains about a makefile error.
A. Always remember to use gnu make.

   On many *BSD systems which have other 'make's, gnu make is called gmake.
   Try make --version -- if it shows a Gnu version stamp you're ok, otherwise
   try gmake.


--------------------------------------------------------------------------------
-----BSD - FreeBSD--------------------------------------------------------------
                                        During the ntop 2.2 development cycle,
                                        we did development/testing under:
                                           4.6.3
                                           5.0

                                        Users reported success with:
                                           4.7
                                           4.8

                                        During the ntop 3.0 development cycle,
                                        we did development/testing under:
                                           4.8 and 4.9
                                           5.1 and 5.2

                                        Users reported success with:
                                           4.6.2
                                           4.5.1
--------------------------------------------------------------------------------


Q. I can't compile ntop under FreeBSD 5.0 or 5.1.
A. In some situations, we produce gcc link lines which expose a conflict 'tween
   FreeBSD 5.x and libtool.

   You will see the following error messages:

       expr: illegal option -- l
       usage: expr [-e] expression
       etc.

   See the release notes for FreeBSD 5.0 and the expr(1) man page:

       http://www.freebsd.org/releases/5.0R/DP2/relnotes-i386.html
       http://www.freebsd.org/cgi/man.cgi?query=expr&sektion=1&manpath=FreeBSD+5.0-current

   Solve this by setting the compatibility flag before running make.

       $ export EXPR_COMPAT=Y

   configure.in and configure have been updated to do warn you to do this.


Q. I get "ntop: /dev/bpf0: Device not configured", what's wrong?
A. This is because bfpX has not been configured inside the generic bsd-kernel
   config file.

   If you use generic kernel config file put "pseudo-device bpfilter 16" in
   kernel config file and rebuild the kernel.


Q. I remember rumors about something not being right under FreeBSD with
   threads?
A. Yes.  See FreeBSD bug bin/17437 at 
   http://www.FreeBSD.org/cgi/query-pr.cgi?pr=17437

   Basically, due to limits in FreeBSD, there is no pthread_atfork() function.
   So, when ntop does it's fork() call to create http pages, it can't fixup
   the Mutexes.  It wrong and could conceivably cause problems.

   However - ntop ran for years without the pthread_atfork() code, so we're
   no worse off in 3.0 under FreeBSD than in 2.2 or 2.1...

   There was a flurry of problems late in the 3.0 development cycle having to
   do with a seeming deadlock of the ntop web server (it's actually not dead,
   just walking at about 0.001KPH).

   Thanks to Yeoman efforts by Stanley Hopcroft, Michal Meloun and, well, me,
   we have a work-around.  

   If you're running under FreeBSD, use the flag, --set-pcap-nonblocking.

   For more on this, read the threads at gmane - look for "FreeBSD and pthreads" - 
   that's probably the best summary.  But there's stuff on this back at least to 
   October 2003 - look for Stanley's problems with CPU usage.


Q. Um... something about bad fds?
A. Oh yes, see bug bin/51535: uthreads bug: new opened files may get stale fcntl flags

   They fixed the code, sort of:

         OS      Version
         ------- --------------
         FreeBSD 4.9    FIXED
         FreeBSD 5.1    BUGGY
         FreeBSD 5.2    FIXED
         FreeBSD 4.8    BUGGY
         FreeBSD 4.6.2  BUGGY

   There's a work-around in ntop, called the HAVE_FILEDESCRIPTORBUG fix.
   If you even might possibly need it, it should be set in configureextra/ but 
   go ahead and set it anyway, it can't hurt.


--------------------------------------------------------------------------------
-----BSD - OpenBSD--------------------------------------------------------------
                                        During the ntop 3.0 development cycle,
                                        we tried development/testing under:
                                           3.1 and 3.4
--------------------------------------------------------------------------------

Q. So?
A. 3.1 - I could never make it work.  Problem seems to be that the gnu tools such
   as ld don't support a.out systems very well.

A. 3.4 - Should work.  Julien Touche <julien.touche@lycos.com> sweated buckets
   to make this work and reported that it did in January 2004.


--------------------------------------------------------------------------------
-----BSD - NetBSD---------------------------------------------------------------
                                        During the ntop 2.3 development cycle,
                                        we tried development/testing under:
                                           1.5.3
                                           1.6
                                           1.6.1
--------------------------------------------------------------------------------

Q. So?
A. Some testing was done using - ntop works only in SINGLE THREADED mode and so
   is fragile.

Q. And Multithreaded?
A. There isn't (yet) a standard POSIX thread package for NetBSD.  Both "proven"
   and "unproven" threads have issues.  One user was bound and determined to
   make GNU Pth work, but pth isn't quite POSIX threads and he's didn't get
   far...

Q. But there IS a POSIX threads...
A. Yes.  And when NetBSD -current finally incorporates the POSIX threads (its
   been committed but not yet part of a release) this should be revisited.


--------------------------------------------------------------------------------
-----Linux - all----------------------------------------------------------------
                                        During the ntop 2.2 development cycle,
                                        we did development and/or testing under:
                                           Debian
                                           LinuxFromScratch 3.0pre3
                                           RedHat AS2.1, 7.2, 7.3, 8.0

                                        During the ntop 3.0 development cycle,
                                        we did development and/or testing under:
                                           Debian 3.0 (Woody)
                                           Gentoo 1.4 (rc5 and released)
                                           LinuxFromScratch 3.0pre3
                                           Mandrake 9.1
                                           RedHat AS2.1, 7.3, 8.0 and 9
                                           Slackware 9.0
                                           SuSE 8.2

                                        And loads more - see the list above.

                                        Others should certainly work and there
                                        are many user reports of success.
--------------------------------------------------------------------------------



--------------------------------------------------------------------------------
-----Linux - RedHat-------------------------------------------------------------
--------------------------------------------------------------------------------

Q. NPTL?
A. When RH9 came out with NPTL, we had lots of problems.  NPTL is much closer
   to POSIX threads, but very different from the old linuxthreads.  At the same
   time, we had problems with the glibc 2.3.x (vs. 2.2.x) update.

   By the end of the development cycle, a couple of memory management fixes
   seem to have made the NPTL/glibc problems go away.  But if they're still out
   there, they are quite nasty.  Random unexplained BOMBs and such.

   If you hit these problems, you can try disabling NPTL via this flag.
   Before both ./configure AND make, set the following:

        export LD_ASSUME_KERNEL=2.4.1"

   This forces the use of the old linuxthreads library.

   This flag may also do something to glibc, but it's undocumented and we
   never could really figure it out.  For that run time behavior change
   to happen, you would also have to have the LD_ASSUME_KERNAL flag set
   at run time too.


Q. The cause?
A. Unknown.

   A lot of them seemed to go away when we fixed a free() that occurs in
   a fork()ed child of memory malloc()ed in the parent.  Under Linux, the
   fork() gets a copy of memory, but a technique called copy-on-write is used
   to reduce the time the fork() takes.

   We suspected that the free() was corrupting the parent's memory structures,
   but this isn't supposed to be possible and we couldn't nail it down enough
   to report it to the glibc people.

Q. What can I do to help?
A. There is code in leaks.c tagged "DIAGNOSTIC".  Turn it on, collect a set of
   potential problems and report them.  However, this code isn't that bright.
   It will report all the free() calls in fork()ed children, even those that
   are legit (i.e. the malloc() was also in for fork()ed child).


Q. "application bug: ntop(...) has SIGCHLD set to SIG_IGN but calls wait().
   (See the NOTES section of 'man 2 wait'). Workaround activated."

   This message and the NOTES section of the man page lead me to believe
   that the problem is handled but the kernel feels the need to report it
   from time to time.
A. Read the NOTES section.  The problem has been handled, but the kernel
   feels the need to report it from time to time.  See the article at
   http://article.gmane.org/gmane.linux.ntop.general/5304


Q. libpng version conflicts?
A. See libpng, below.

--------------------------------------------------------------------------------
-----Win32 - Common-------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Where can I find GDBM for Windows?
A. GDBM for windows can be found at http://www.roth.net/libs/gdbm/


Q. ntop -i1 ... doesn't work
A. ntop has special parameters under Win32

         Under win32 there are TWO COMPLETELY SEPARATE TYPES OF PARAMETERS.

     There are the parameters to the win32 stub AND there are parameters to ntop
     itself.

     AFTER THE win32 parameters are the ntop parameters in the standard (Unix)
     -xxx format.

         ntop /c <normal parms>  runs ntop INTERACTIVELY with the specified ntop
         parameters

         ntop /i <parameters> installs ntop as a service to run with the specified parameters

         ntop /d deletes the ntop service

   Remember, ntop /i and ntop /d don't actually run the service - you need to
   start it.


Q. How do I figure out what my network interface numbers are for the -I
   parameter?
A. (Thanks to jac engel [jacengel@home.nl] for the example)

   If you only have ONE network interface, it doesn't matter as the default is
   fine.  However, that's the RARE case.  Most people have multiple network
   interfaces (NICs), with virtual ones for VPNs, Dialup Networking, etc.

   The Windows tools ipconfig, winipcfg and the Device Manager (depending on
   which version of Windows you have) will probably show you them.  However,
   it's easier and better to use ntop to show you how ntop sees the network
   interfaces.

   If you start  ntop /c (interactive mode, with only the default parameters) it
   Will display all your network interfaces (NICs), like this:

    Running ntop for Win32.
    Wait please: ntop is coming up...
    23/Aug/2002 20:43:55 Initializing IP services...
    23/Aug/2002 20:43:55 Initializing GDBM...
    23/Aug/2002 20:43:55 Initializing network devices...
    23/Aug/2002 20:43:55 Found interface [index=0] '\Device\Packet_{14...1C}'
    23/Aug/2002 20:43:55 Found interface [index=1] '\Device\Packet_{86...B4}'
    23/Aug/2002 20:43:55 Found interface [index=2] '\Device\Packet_NdisWanIp'
    23/Aug/2002 20:43:57 ntop v.2.1 MT [WinNT/2K/XP] (11/07/2002 build)
    23/Aug/2002 20:43:57 Listening on [3F1C}\Device\Packet_{14...1C}

  By default, ntop will use the lowest numbered interface.  Because #s are
  assigned based on the sequence cards are discovered, and this is altered if
  cards are removed and added, this is often not what you want.

  After you figure out which NIC you want, start ntop /c -i1 or -i2 or
  whatever...


Q. OK, but how to I translate \Device\Packet_xxxxx to my Froboz ModelT network
   card and not the Fubar27 that's on the motherboard.
A. ntop should report both the index and the human readable information.
A. A Google search on script "CurrentVersion\NetworkCards" finds a couple of
   scripts/utilities that might work in various environments.

A. Otherwise...

   You're going to need to view the registry.  All the usual warnings - back up
   your pc, etc.

   If you damage the registry, you may not be able to reboot the computer.

   You're not going to CHANGE anything, but an inadvertent keystroke could be
   disaster ... BE CAREFUL!

   Under WinNT/2K, to find the interface name of your NIC look in the registry
   at the keys in 
     HKLM\Software\Microsoft\Windows NT\Currentversion\NetworkCards\
   The two subkeys, Servicename and the Description tells you which id maps to
   which NIC.


Q. Where does ntop look for html (and gif) files under Win32?
A. ntop looks in two places. The first is the current directory and the second
   is configurable through a constant in ntop_win32.h, #define DATAFILE_DIR "."

   Note that the current directory, or ".", may not be what you expect.

   When running ntop as a Win32 service, "." is %SystemRoot%\system32, meaning
   that ntop looks in %SystemRoot%\system32\html for the .html and .gif files.

   When running ntop from the command line,

       ntop /c parameters...

   "." is whatever directory is current.  This means that if you run ntop with a
   full, explicit path (c:\ntopnew\ntop /c ...) there may be an unexpected
   difference between what ntop finds for "." and what you THINK "." is!  This
   will lead to missing .html and .gif files.

   If you wish to have ntop look in a specific place for the files, the best
   choices are:

     1) Create a .bat file to run ntop which does a cd to the expected directory
        first.
     2) Edit ntop_win32.c and then recompile.

   Note that the settings for DATAFILE_DIR (and other constants) are reported on
   the text version of the configuration page, textinfo.html.



--------------------------------------------------------------------------------
-----Win32 (MinGW) (Windows)----------------------------------------------------
--------------------------------------------------------------------------------

Q. What's the scoop with ntop on Windows?
A. Semi-officially,

   ntop for Windows 95/98/ME/NT/2K is also provided as a binary application with
   limited capture capability (1000 packets). This is intended to allow 
   demonstration of ntop for people without access to a Unix system.

   We call this version Win32 after the old official name of the Windows library.

   If you want to use the full version with unlimited packet capture you can
   either:

       * Recompile ntop from the source by yourself (Luca says just open the
         files in MS Visual C++ 6.0 and press compile)
       * Register your ntop for Windows 95/98/NT/2K copy by paying a 
         convenience fee to receive the prebuilt executable.

   If you decide to register your copy, Luca will send you an URL from which
   you  can download the full version periodically.


Q. So where did MinGW come in?
A. All of the necessary open software tools have been ported to run under
   windows (sort of), so it is theoretically possible to build and run ntop
   under Windows.  

   However, Windows and *nix are very, very different internally.  So we have
   to use special versions of the packet capture library (winpcap instead of
   libpcap).  And we needed our standard tools (gcc et al), plus some 'glue'
   so we could make *nix calls and have Windows things happen.

   For the tools and glue, there are three choices: native, Cygwin and MinGW.

   Native means you put lots of #ifdefs in to make Windows calls where you
   had *nix calls.

   Cygwin is a shim - it's a Windows dll (Dynamic Link Library) that pretends
   to support the *nix calls and then does the right Windows things for you.

   MinGW is a project to create native Windows tools and executables from
   *nix code.

   Each of these have limits, pluses and minuses.  For example, Cygwin's dll
   has caused all sorts of problems (dlls and their *nix equivalent, shared
   libraries, usually do).  MinGW has some limits.  As MinGW grew, ntop for
   Win32(MinGW) got closer to ntop under *nix.  Native means supporting two
   'separate' code bases.

   Luca actually picked a hybrid - he uses Microsoft Visual C++ 6.0 - which
   has it's own (albeit incomplete) shim layer.  As long as you stay within
   the limits of the shim, the same code works across platforms.  In other
   places you have to make thinks like Windows wants them.  How close are
   we?  grep for WIN32.

   The advantage of the hybrid was that is was also pretty close to working
   under MinGW - creating native executables from the base code using free
   tools.  So people sent in patches and it pretty much worked.  Up until
   2.1.3, that is.  But it was never officially more than an 'it also runs'.

   After 2.1.3, Luca embarked on a process of bringing the WIN32 and *nix
   code closer together.  Surprisingly, this actually broke MinGW!


Q. Where do I get MinGW?
A. The MinGW home page is http://www.mingw.org.  Quoting:

      "MinGW is a collection of header files and import libraries that 
      allow one to use GCC and produce native Windows32 programs that do
      not rely on any 3rd-party DLLs. The current set of tools include 
      GNU Compiler Collection (GCC), GNU Binary Utilities (Binutils), 
      GNU debugger (Gdb) , GNU make, and a assorted other utilities. We
      are currently working on creating a complete set of Mingw-hosted
      GNU toolchain, and looking for volunteers."


Q. Does ntop work under Cygwin?
A. The .exe distributed through ntop.org is built with Visual C++ 6.0.
   It proved just barely possible to use the same code under MinGW.
   Forget about cygwin.


Q. Does ntop (v3.0) work under MinGW?
A. No - There are problems with versions of ntop after 2.1.3 under MinGW -
   it won't build.


Q. When I type 'make' it complains about a makefile error.
A: Remember to use -f Makefile.MinGW or whatever is appropriate - see
   BUILD-MinGW.txt.


Q. Mingw make of Ntop fails when using the single-file distribution
   MinGW-1.1.tar.gz with make errors about version.c like :
    zsh: no matches found: *version
    make: *** [version.c] Error 1
A. Be sure that your PATH setting in the DOS command box of the Mingw bin
   directory ends with a backslash.
    This is OK:
        set path=C:\Mingw\bin\;%path%
    This is wrong:
        set path=C:\Mingw\bin;%path%
                             ^


--------------------------------------------------------------------------------
-----Win32 (MS Visual C++)------------------------------------------------------
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
-----Other platforms------------------------------------------------------------
--------------------------------------------------------------------------------


HP-UX
=-=-=
During the 2.2 development cycle, some work was done to make ntop work under
HP-UX 11 without breaking the HP-UX 10.20 support (limited as it was). During
the 3.0 development cycle, HP-UX was not considered.

It will almost certainly fail.


IRIX (v1.3 information)
=-=-=-=-=-=-=-=-=-=-=-=-
During the ntop 2.2 and 3.0 development cycles, IRIX was not considered.

It will almost certainly fail.


Digital UNIX (v1.3)
===================
During the ntop 2.2 and 3.0 development cycles, Digital UNIX was not considered.

It will almost certainly fail.


AIX (v1.3)
==========
During the ntop 2.2 and 3.0 development cycles, AIX was not considered.

It will almost certainly fail.


--------------------------------------------------------------------------------
-----Libpng---------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Bad things - I see the following messages:
        libpng warning: Application was compiled with png.h from libpng-1.0.x
        libpng warning: Application is running with png.c from libpng-1.2.x 
        gd-png: fatal libpng error: Incompatible libpng version in application
              and library
A. You have a version problem with libpng.

   First off, following the instructions in BUILD-NTOP.txt should work just
   fine. These problems come about when you have libpng installed (i.e. using
   shared libraries).

   1. If you are compiling from source, you may have png.h left over from the
      earlier version of libpng. Remove it.

   2. (Most common under RedHat). RedHat 7.2 installs a libgd.so.1.8.4 library,
       which was compiled against 1.0.x series of libpng (which is fine, because
       RedHat 7.2 includes libpng-1.0.12).

   Updating RedHat to newer (RawHide) packages for libpng,
   http://www.rpmfind.net//linux/RPM/rawhide/1.0/i386/RedHat/RPMS/libpng-1.2.2-5.i386.html,
   should work. However, there are reports of version conflicts and required
   updates to multiple packages. Proceed with caution (especially if you decide
   to uninstall 1.2.2-5).
   Also, do not use --nodeps or --force, as this can leave you with two
   partially installed versions (see item #1, above).

   3. (Slackware) Users have reported this error from an older header file in
      /usr/include.
   Make sure to run "make install" in the libpng directory so that the latest
   files are in the common library locations. You can do this with buildAll.sh,
   just navigate back down to the libpng-1.2.x directory first.

   4. If you are building ntop on one machine and running on another, they may
      have different libpng.so versions.  Even if you think you are using the
      static linked version (buildAll.sh), be careful - see the entry (above) on
      "make install" for libpng.

Q. When I run ./configure, it finds png.h but not libpng:
      *******************************************************************
      *
      * ERROR: libpng header or library routines are missing
      *           (yes means it was found, no means it was not found)
      *
      *              png.h...yes
      *              png_read_info() in -lpng...
      *
      *>>> No way to proceed.
      *
      *???        Install libpng (and/or libpng-devel), check www.libpng.org
      *???    and Rerun ./configure
      *
      *******************************************************************
A. You're missing libpng.so.  Look for it (locate libpng) and tell ntop
   where it is via the --with-png-lib= parameter.


Q. I seem to be missing libpng.so - when I do locate libpng, it finds:
      /usr/lib/libpng.so.3
      /usr/lib/libpng.so.3.1.2.2
      /usr/lib/libpng12.so.0
      /usr/lib/libpng12.so.0.1.2.2
      /usr/lib/libpng.so.2.1.0.13
   but no libpng.so...

A. Yeah.  Send RedHat a nasty gram.

Q. I don't understand...
A. In a normal libpng install, say from the source, you would have - in 
   addition to the .so.n.n.n.n files - a symlink named libpng.so, like this:

      lrwxrwxrwx 1 root  root   19 Apr 23 15:46 libpng.so -> libpng12.so.0.1.2.2

   But, that link seems to be missing.  Without it, -lpng doesn't properly
   resolve and you get the ./configure error.

Q. Why?
A. RedHat ships Linux with both versions of libpng, a 1.0.x and a 1.2.x version.

   Do this:

      $ rpm -qa | grep libpng

   And you'll see the libpng 1.0.x run time:

      libpng-1.2.2-8
      libpng-devel-1.2.2-8
      libpng10-1.0.13-6

   Dig into the files installed by them and you'll not find libpng.so.

   Since they're incompatible RedHat doesn't create the libpng.so link.
   Instead they patch the makefiles to point the various packages at one or the
   other .so file they install.  This allows them to ship packages that require
   one or the other.

   It works fine unless you try and install something other than via an rpm.
   Then you're missing the libpng.so file that normal packages look for...

   Best bet is to create a symbolic link from the libpng.so.xxxx installed 
   by the package which matches the -devel (because that's where png.h is 
   found), e.g.:

   ln -s /usr/lib/libpng.so.3.1.2.2 /usr/lib/libpng.so

   And remember this in case you update the libpng rpm's in the future.


--------------------------------------------------------------------------------
-----Silly Season---------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Who is Pixel
A. My cat.


--------------------------------------------------------------------------------
-----HowTo Ask For Help (ntop mailing lists)------------------------------------
--------------------------------------------------------------------------------

HOWTO ask for help on the ntop or ntop-dev mailing lists:

WHERE TO POST
=============

ntop is for user questions - "How to I install", "data isn't being recorded",
etc.

ntop-dev is for code and development questions.  The ntop-dev list goes to fewer
people, those who have self-selected themselves to be interested in ntop at the
code level.

Discussions about the current cvs version belong in ntop-dev.

If a discussion gets too technical, you may be asked to "move it to ntop-dev".
Please honor that request (even if you have to subscribe for a while - ntop-dev
is fairly low traffic).

There used to be mailing lists and trackers at SourceForge, which were rarely
looked at and have been discontinued. Use the ntop and ntop-dev lists (go to
http://www.ntop.org to signup for them).


OFFICIAL vs UNOFFICIAL
======================

A response from Luca Deri should be considered official.  He is the author of
ntop and controls the project and it's destiny.

***Please understand that the mailing lists are a community support effort***

Besides Luca Deri, a number of people answer questions to the best of our
ability. None of the rest of the people who may respond to your question on ntop
or ntop-dev are able to respond "officially".

Likewise, this HOWTO is unofficial.

Everyone is welcome to help with the evolution of ntop - that is to find
problems, create and test patches and send them in to patches@ntop.org for
inclusion.  There are a small number of people with write access to the cvs,
but anything we commit is subject to being ripped out by Luca for any reason...
or no reason at all...


QUESTION FORMATS
================

ONE QUESTION per message, and you MUST use meaningful message subjects - one's
that would have helped YOU find the prior discussion of this or a similar
problem in the archives.  Titles such as "urgent" or "ntop problem" will often
not get a response - it may be urgent to you but it's probably not an issue
for others.

WE STRONGLY SUGGEST YOU USE THE BUILT IN AUTOMATICALLY GENERATED PROBLEM
REPORT (About | "bug icon").  This includes most of the internal configuration
data we ask for (and more) and has blank spots for you to fill it.

Generate one, cut & paste into your mail client, edit the data and sent it.

Beyond that, don't worry -- it's about information, not format.


RESPONSES
=========

Despite any individual's frequent postings, nobody is "responsible" for
answering your question. It's all on a "best efforts" basis. This is equally
true of the entries in ntop's Wiki. Our responses may be incomplete, inaccurate,
even dead wrong. Caveat Emptor! The only "guarantee" is that free support will 
be worth what you've paid for it.  It may be worth MORE, it won't be worth
LESS.

Just because you post a question does NOT mean that you are OWED an answer.

If nobody answers, then maybe it's because:

   * Nobody knows.
   * People are busy.
   * You've asked the same question multiple times and it's already been
     answered.
   * You have been asked for additional information and are unable/unwilling
     to supply it.

or, well, any one of a dozen other reasons.

Asking the same question multiple times - or asking it again because you don't
like the answer you received - is a slap in the face of the person who took the
time to answer you in the first place and will more than likely not get a
different response.  If you're not sure that your message posted, check the
archives to see if your message is there -- please don't just keep reposting it.

You can always use gmane (http://www.gmane.org) to see the last 600 or so
postings to the lists.

Please direct all original postings and subsequent replies to the list, not to
someone privately.  Most of us will reply solely to the mailing list, unless
you specifically request otherwise.  If you do request otherwise, the individual
you sent it to may choose not to respond.  Our posting here is NOT a public
invitation to invade our e-mail boxes for your free private support.


THE BACK AND FORTH PROCESS
==========================

"Why don't you just fix my problem instead of asking for more information?"

Understand that we can't see your machine (and wouldn't want the
responsibility of sshing into somebody else's box as root).  The only
information we have is what you post and the responses to our questions.  Few
failures in ntop are related to the core processing routines - so if you're
having a problem, it's most likely because of some combination of your network
and your ntop configuration.  It may be unique to you -- and only with YOUR help
can it be resolved.


WHAT IS SUPPORTED?
==================

Releases are hosted at SourceForge.

At the time of this writing the stable version is 3.0. What support is available
is for the development version ("the cvs").  All support is in the form of
fixing things in the cvs.

However we also attempt to support the current "stable" release (3.0).

Older versions are not supported -- especially 1.3 and the 2.0.99 series of
2.1 release candidates.  If you have a problem with them, please obtain the
current cvs version and see if it's still a problem.  Unlike certain much larger
projects, we don't fix things in older versions - there simply aren't enough
resources available.

intop is not supported.  It's gone and no longer in the cvs.  For a look at
Rocco's new, work-in-progress, download ntcsh, the ntop enabled tcsh shell.

Please understand that the only way to fix your problem may be a source patch,
which you will have to apply, compile, install and test against the cvs version
prior to it's inclusion in the cvs.

If you aren't capable of or willing to do these steps -- for whatever reason --
then you should not be compiling from the cvs.


CVS
===

The cvs is at http://cvs.ntop.org, userid is anonymous, password ntop.

The cvs is a DEVELOPMENT version.  The code in the cvs is subject to rapid
change.  At any point in time, it may not compile.  It may not compile with
certain options or on some platforms. s'be'it -- it's a DEVELOPMENT version.


2.1.3
=====

The actual flow of ntop development was 
    2.1 -> 2.1.1 -> 2.1.2 -> 2.1.50 -> 2.1.51...

Version 2.1.3 was provided by Dennis Schoen [ds@teuto.net] as part of the Debian
project.  Dennis (manually) maintains a bitkeeper tree, based on 2.1.2 with
various patches which - in HIS opinion - were important enough to be back
ported.  Releases in this tree are identified as 2.1.2-n.  Dennis reports you
can obtain his current version via:

bk clone bk://ntop.teuto.net:ntop-debian ntop-stable

Version 2.1.3 is an export from Dennis' tree with the version number changed and
is equivalent to 2.1.2-1.  Dennis' graciously provided the extract and we
accepted with thanks!

2.2
===

The actual flow of ntop development was 2.1 -> 2.1.1 -> 2.1.2 -> 2.1.50
                                     -> 2.1.51..59 -> 2.1.90..92 -> 2.2

   after 2.2, came 2.2a/2.2b (Win32 only), then 2.2c.  These were branches,
   containing unique 2.2 fixes and back-ports of some critical items.

3.0
===

The actual flow of ntop development was 2.2.1..4 ->
                                    -> 2.2.90..99 -> 3.0pre1 -> 3.0


FEE-BASED SUPPORT
=================

If you want better than "best-efforts" support, contact the individual you
Desire support from off-list to make financial arrangements.  Please understand
that people are doing development in areas that are of personal interest to
them, to improve ntop.
If you want to discuss payment for support or a specific change that is of
Interest to you, feel free to email the individual off-list - some of us are
Computer consultants and can be bought, with the understanding that the work
product is offered back to the community in the spirit of the open source
movement and the strictures of the GPL.


SO WHAT INFORMATION SHOULD I POST?
==================================

BEFORE POSTING:

1. Please review the output from ./configure.

    We all have the bad habit of skipping over this, but there are often
    warnings which explain why things don't work.  ntop tries to build itself by
    turning off features where the required libraries and/or headers aren't
    available.
    The minimum required set is just that - minimal.  This is often the source
    of "feature x or switch y doesn't work" reports.

2. Please review the docs/FAQ file.

3. Please review back message traffic from the mailing lists.

     Yes, we know that there isn't a search function at ntop.org.  Did you know
     that the lists are spidered every couple of months or so and can be
     searched through Google??  For example, "site:lists.ntop.org rpm" will find
     mail list messages with the word "rpm" in them.

     Do you know about gmane (http://www.gmane.org) has archives (searchable) of
     the ntop lists going back into late 2001.  The lists are called

          gmane.linux.ntop.devel
          gmane.linux.ntop.general

     You can read these online (the last 600 messages or so) or through the nntp
     server.


POSTING:

Do not worry about posting TOO much information - we're pretty good at filtering
out the noise.

WE STRONGLY SUGGEST YOU USE THE BUILT IN AUTOMATICALLY GENERATED PROBLEM
REPORT (About | "bug icon").  This includes most of the internal configuration
data we ask for (and more) and has blank spots for you to fill it.

Generate one, cut & paste into your mail client, edit the data and sent it.

If you can't or won't use the automated problem report (say, for example you
can't get ntop up to generate it) - don't worry -- it's about information, not
format.  Send us what you can, organized this way:

1. A brief summary of the problem.

2. Operations
     The EXACT command line you use to invoke ntop.
          If it's in a script, cut & paste it and
          resolve all the variables!

     Error Messages:  Cut & paste the exact text.
          If it's in the log, give us 15 or 20 lines before.

     The exact URL you used from the browser.

3.  Software
     ntop version, source and any applied patches

            If you've compiled from the source, say so!

            If you're using a package (such as an .rpm), where
                did you get it from and what is the EXACT name,
                version information and date? (for example, post
                the output from rpm -q ntop -i)

     OS vendor & version

     gcc version (e.g. gcc --version)
       (For ./configure problems, the versions for autoconf, automake and
        libtool too)

     glibc version

     Any major upgrades (kernel, networking, etc.)

     What else is running

4. Hardware
     Type & # of processors

     Amount of memory

     # network interfaces and types (vendor, bus, etc.)

5.  Network
     Roughly where are the interface(s) you're monitoring
          (Public Internet, Private LAN, what?)

     What's the bandwidth (e.g. 10 Mbps University internet,
           1.5 Mbps T1, Cable Modem capped at 1.5Mbps, 56K dialup)

     How many machines (traffic sources/destinations) and users

(If you're uncomfortable giving specifics, then leave it generic, but the
information is necessary to allow efficient use of the community's time helping
YOU with YOUR problem)


AFTER POSTING:

Please let us know if our help fixed the problem, didn't solve it or enabled you
to solve it yourself and what the result was.  The historical record of the ntop
and ntop-dev archives is the complete chain from problem to resolution.

(Originally posted on 07Jan2002 to ntop and ntop-dev, updated.  This is version 12April2003)

--------------------------------------------------------------------------------
-----GDB ultraMini-tutorial - Running ntop under gdb (debugger)-----------------
--------------------------------------------------------------------------------

The very best way to debug a segmentation fault in ntop is to use gdb. The
Standard ntop compile already has the flags necessary to do this set.

(Note - if you don't have gdb, or aren't compiling yourself, this won't work)

> gdb /usr/bin/ntop (or wherever ntop is installed)
...
(gdb) set args (your usual arg string) -K

[That is, add the -K argument. While you are at it, don't give it the -d
argument and add -u root (replace any existing -u value) - yes, it's insecure
running as root, but you're not planning on doing this in production nor as a
routine situation!]

it will run... when it bombs...

(gdb) list [this shows where in the code it died]

(gdb) info stack [this shows the call stack]

if there are any variables involved, you can print them:

(gdb) print deviceId

[gdb can handle pretty complex arguments in the print command, so you can say
"print myGlobals.device[0].hash_hostTraffic[myGlobals.broadcastEntryIdx]"
if that's what it bombed on.]

"bt full" does a decent job of printing the stack and the back trace and the
local variables at each level. Just make sure you are in the thread you are
interested in:

(gdb) bt full
#0 0x40592557 in __libc_pause () from /lib/i686/libc.so.6
No locals.
#1 0x4046b5a3 in pause () at wrapsyscall.c:123
result = -1073743680
oldtype = 0
#2 0x0804ac1b in main (argc=22, argv=0xbffffa44) at main.c:928
argc = -1073743680
argv = (char **) 0x0
i = 0
userSpecified = 1
ifStr = "eth0,eth1", '\000' 
lastTime = 1025633918
#3 0x404f3647 in __libc_start_main (main=0x804a74c , argc=22, ubp_av=0xbffffa44,
init=0x8049600 <_init>, fini=0x804d000 <_fini>, rtld_fini=0x4000dcd4 <_dl_fini>,
stack_end=0xbffffa3c) at ../sysdeps/generic/libc-start.c:129
ubp_av = (char **) 0xbffffa44
fini = (void (*)()) 0x40016b4c <_dl_debug_mask>
rtld_fini = (void (*)()) 0xbffff87c
ubp_ev = (char **) 0xbffffaa0
(gdb)


gdb has all kinds of included help, and frankly that's all I know...
Here are some extracts and examples:

(gdb) help info thread
IDs of currently known threads.

(gdb) info thread
  13 Thread 11276 (LWP 8967)  0x405f651e in __select () from
/lib/i686/libc.so.6
  12 Thread 10251 (LWP 8966)  0x405f651e in __select () from
/lib/i686/libc.so.6
  11 Thread 9226 (LWP 8965)  0x405f651e in __select () from
/lib/i686/libc.so.6
  10 Thread 8201 (LWP 8964)  0x405f651e in __select () from
/lib/i686/libc.so.6
  9 Thread 7176 (LWP 8963)  0x405f651e in __select () from
/lib/i686/libc.so.6
  8 Thread 6151 (LWP 8962)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  7 Thread 5126 (LWP 8961)  0x4053db85 in __sigsuspend (set=0x4300998c)
    at ../sysdeps/unix/sysv/linux/sigsuspend.c:45
  6 Thread 4101 (LWP 8960)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  5 Thread 3076 (LWP 8959)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  4 Thread 2051 (LWP 8958)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  3 Thread 1026 (LWP 8957)  0x4053db85 in __sigsuspend (set=0x4100967c)
    at ../sysdeps/unix/sysv/linux/sigsuspend.c:45
  2 Thread 2049 (LWP 8956)  0x405f4e17 in __poll (fds=0x823dbec, nfds=1,
timeout=2000)
    at ../sysdeps/unix/sysv/linux/poll.c:63
* 1 Thread 1024 (LWP 8952)  0x405ca027 in __libc_pause () from
/lib/i686/libc.so.6

The * indicates the current thread.  Switch with this:

(gdb) thread 7

Once you have a thread selected, you can look at the call stack:

(gdb) help stack
Examining the stack.
The stack is made up of stack frames.  Gdb assigns numbers to stack frames
counting from zero for the innermost (currently executing) frame.

At any time gdb identifies one frame as the "selected" frame.
Variable lookups are done with respect to the selected frame.
When the program being debugged stops, gdb selects the innermost frame.
The commands below can be used to select other frames by number or address.

List of commands:

backtrace -- Print backtrace of all stack frames
bt -- Print backtrace of all stack frames
down -- Select and print stack frame called by this one
frame -- Select and print a stack frame
return -- Make selected stack frame return to its caller
select-frame -- Select a stack frame without printing anything
up -- Select and print stack frame that called this one

Type "help" followed by command name for full documentation.
Command name abbreviations are allowed if unambiguous.
(gdb) info stack
#0  0x4053db85 in __sigsuspend (set=0x4300998c) at
../sysdeps/unix/sysv/linux/sigsuspend.c:45
#1  0x404db1c9 in __pthread_wait_for_restart_signal (self=0x43009be0) at
pthread.c:969
#2  0x404dc1ec in __new_sem_wait (sem=0x804d5c8) at restart.h:34
#3  0x402960e0 in waitSem (semId=0x804d5c8) at util.c:1126
#4  0x4027ae0f in dequeueAddress (notUsed=0x0) at address.c:425
#5  0x404d8c6f in pthread_start_thread (arg=0x43009be0) at manager.c:284

And select a specific frame via:

(gdb) select-frame 4

(gdb) help list
List specified function or line.
With no argument, lists ten more lines after or around previous listing.
"list -" lists the ten lines before a previous ten-line listing.
One argument specifies a line, and ten lines are listed around that line.
Two arguments with comma between specify starting and ending lines to list.
Lines can be specified in these ways:
  LINENUM, to list around that line in current file,
  FILE:LINENUM, to list around that line in that file,
  FUNCTION, to list around beginning of that function,
  FILE:FUNCTION, to distinguish among like-named static functions.
  *ADDRESS, to list around the line containing that address.
With two args if one is empty it stands for ten lines away from the other
arg.
(gdb) list address.c:425
420
421         while((myGlobals.addressQueueLen == 0)
422               && (myGlobals.capturePackets) /* Courtesy of Wies-Software
<wies@wiessoft.de> */
423               ) {
424     #ifdef USE_SEMAPHORES
425           waitSem(&myGlobals.queueAddressSem);
426     #else
427           waitCondvar(&myGlobals.queueAddressCondvar);
428     #endif
429           key_data.dptr = data_data.dptr = NULL;


(gdb) help print
Print value of expression EXP.
Variables accessible are those of the lexical environment of the selected
stack frame, plus all those whose scope is global or an entire file.

$NUM gets previous value number NUM.  $ and $$ are the last two values.
$$NUM refers to NUM'th value back from the last one.
Names starting with $ refer to registers (with the values they would have
if the program were to return to the stack frame now selected, restoring
all registers saved by frames farther in) or else to debugger
"convenience" variables (any such name not a known register).
Use assignment expressions to give values to convenience variables.

{TYPE}ADREXP refers to a datum of data type TYPE, located at address ADREXP.
@ is a binary operator for treating consecutive data objects
anywhere in memory as an array.  FOO@NUM gives an array whose first
element is FOO, whose second element is stored in the space following
where FOO is stored, etc.  FOO must be an expression whose value
resides in memory.

EXP may be preceded with /FMT, where FMT is a format letter
but no count or size letter (see "x" command).

(gdb) print key_data
$3 = {dptr = 0x0, dsize = 1076452514}
(gdb) print data_data
$4 = {dptr = 0x0, dsize = 4}


==========================================================================
==========================================================================
Original version Luca Deri, 1999-2001
Updated Burton M. Strauss III 2002, 2003, 2004

