Vispan

Current version 3.0.0
8 April 2007

Donate

If you find this software useful and would like to make a donation please follow one of the links below.

Download

Latest Stable Release
   Vispan V3.0.0
 

Beta Release
    None
Please note that this release is a BETA release and may not work correctly. It is intended for people who want the latest features and are prepared to provide the author with feedback.
Please Note: This version requires the use of the GD graph library.

Installation

perl Makefile.PL make install 

This will install the necessary files into your system.

If necessary it will amend your crontab to add an entry to call the script.

Please note that the config file will be installed into /etc. If it already exists then it will be upgraded and a backup copy made of the original file. You should check that the upgraded file contains the correct settings.

UPGRADING - the latest version should read the stats in from your existing file, however, this code has not been extensively tested and no guarantees are made about the accuracy of the resulting statistics.

Dependencies

• perl module Number::Format.
• perl module HTTP::Date.
• Perl module Net::DNS
• Perl module Net::CIDR (should already be there - used by MailScanner)
• Perl module GD::Graph (this requires the GD library)
• Mailscanner.
• Spamassassin.
• GEOIP Database and perl module (optional).
• perl module Mail::Sendmail

Example

Support

Description

Vispan is a PERL script which analyses the mail log file to produce useful statistics. It requires MailScanner to provide the necessary log file entries. The virus list is dependent on the virus scanner you have installed.

In order to detect the spam correctly it is strongly recommended to use SpamAssassin with MailScanner

The script can also use heuristics in the senders of the spam emails and can then automatically add them to the sendmail access file which will cause further mails to be rejected. After a definable period of time they will be removed from the access file and once again allowed to send mail to you.

The heuristics provide an escalation mechanism so that when an IP address is removed from the block then a grace period is started. If that IP sends any spam during that period it will escalate and block the IP for twice the previous block time. For example using the default settings:
Level 1 = 5 days
Level 2 = 10 days
Level 3 = 20 days
Level 4 = 40 days
etc

I believe that this is superior to the RBLs that are maintained since you have control over them and it is specific to the behaviour of your particular mail server

Usage

Vispan is usually called via the cron mechanism - an entry is created at install time to do this. Note that the time interval set by the cron job should not be changed since Vispan uses this to perform some calculations. Changing the run interval in the cron job will cause some very weird results.

There are command line options as follows:

-h

Print out a help message and quit

-f <file>

Use the alternate config file specified

-r xxx.xxx.xxx.xxx

Remove the IP address specified from the access database and the Vispan data file.

Config file

The Vispan.conf file contains configuration information which you should change to suite your requirements as follows:

UseAccess

Set to 1 to use the access file in sendmail set to 0 to avoid it

UseIPTables

Set to 1 to use the iptables facility to block the sender at the firewall level. Note setting this to 1 will effectively set UseAccess to 0 as the packet will never get as far as the MTA so there is no point blocking at the MTA level

UseHeuristics

Set to 1 to use the heuristics system. Set to 0 to just produce a list of IP addresses that have sent spam - this might get pretty large!

BlockTime

The amount of time that the sender should stay in the access block list. Specified in minutes

IPTables

The location of the iptables command

IPTablesConf

The location of the iptables config file. This MUST be set to the system location of your config file so that the IP blocks get reloaded on a system restart

AccessList

Path to the access file - only used if UseAccess is set to 1

MakeMap

The location of the makemap command - needed to rebuild the access file for sendmail

WhereIs

Location of the whereis command - used to determine the version numbers. Usually /usr/bin/whereis for Linux systems and /usr/ucb/whereis for Solaris.

WhereisPath

The paths that the whereis command will search to find the virus scanners and mail scanner

LogFile

The location of your mail log file

Queue_Dirs

A space separated list of the full paths of any extra mail queue dirs.

WhiteList

An array containing the IP addresses of servers that should not be added to the access file.i Can be specified as individual addresses, a range or in CIDR format.

tmpfile

The location of the temporary file that is created

WorkDir

The directory that you want to use to keep the working files in.

HTMLDir

The directory to hold the output HTML code

Header

The location of a header file that can be used to provide HTML to be added before the generated HTML. The contents are added immediately after the body tag and before any HTML generated by Vispan.

Footer

Similar to the header but added after the generated HTML and immediately before the horizontal line at the bottom of the page. Setting this to an empty file will prevent the W3C logo appearing.

Logo

The name of a file to include as a logo.

StyleSheet

If you use a style sheet you can specify it here - it should be relative. A sample style sheet is provided in the distribution.

SMLogString

The string that is logged to the log file for sendmail. Most installs use sendmail but FreeBSD appears to use sm-mta

UseGeoIP

Set to 1 to use the geographic functions to determine the location of the spam/virus based on the IP address. This requires the use of the GeoIP library and should only be enabled if you have already loaded it.

HighScore

The value of the high spam score as defined in the MailScanner config file

PageTitle

The title to appear on the page

Scanner

The virus scanning software you are using. Currently can be one of sophos, sophossavi, mcafee, command, bitdefender, etrust, inoculan, f-secure, f-prot, rav, antivir, clamav, clamavmodule, trend or mailscanner.

Spam_Reject_Text

The text placed after the IP address in the access file. This is sent to the sending MTA when mail is rejected. This is used when an IP address is added due to excessive Spam mail being received.

Virus_Reject_Text

The text placed after the IP address in the access file. This is sent to the sending MTA when mail is rejected. This is used when an IP address is added due to excessive viruses being received.

DisplayTop

A number to indicate that the program should only display the top n in the country list and the SpamAssassin trap report.

Mins, Hours, Days, Months, Years

These specify the number of sets of data to keep for each of the intervals. These are used in the detailed statistics output.

FromLeft

Set to 1 to make the graphs grow from the left instead of the right. That is, new values will appear on the left.

ServersToLookAt

A list of servers to look for in the log file. Leave blank for all servers.

AccessNotify

When set to 1 will send an email when an IP address is added to the access file

NotifyToAddress

The address to send the email to.

NotifyFromAddress

The address that the email comes from

NotifySubject

The subject line of the email.

SMTPServer

The SMTP server to use to send the email.

NumSpams

The maximum number of spams allowed from a particular source in a 24 hour period before it gets added to the access file.

NumViruses

The maximum number of viruses from a given IP address in a 24 hour period before it gets added to the access file.

Changelog

Vispan-3-0-0

	* Corrected the code when there are no maillog entries for the
	  interval. This caused a blank value to be stored for the current
	  file position.

	* Added RBL support

	* Added RBL support

	* Provide the detailed statistics including the load.

	* Moved the setting of the time field to earlier so that graphs are
	  generated correctly on heavily loaded systems.

	* [no log message]

	* Only add entry in whitelist if name lookup succeeds.

	* Correct missing Vispan:: entries

Vispan-2-0-4

	* Added packaging information to ensure that the modules don't get
	  confused with other packages.

	* Added packaging information to ensure that the modules don't get
	  confused with other packages.

	* [no log message]

	* [no log message]

	* Added facility to allow whitelist to be specified by domain name
	  as well as IP address.

	* Added fixes from Aaron Moore for virus scanner lines and numcols.

	* Modified whitelist code ton allow domain names to be used as
	  well.

	* Added percentages to the cache hit reports.

	* Added the analysis and reporting of cache hits which was added in
	  MailScanner V4.50

	* Changed the order of the details on the spam page so that blocked
	  IPs are at the top.

	* Added functionality to allow whitelist to contain FQDN as well as
	  IP addresses.

	* Changed code so that if virus scanning is not enabled the count
	  of messages and size still works.

	* Changed comment about sophosSAVI to make the SAVI uppercase

	* Corrected typo in variable definition.

	* Added a command line option of -v to get the version numbers of
	  the virus scanners.

	* Added check to make sure that the virus scanner specified in the
	  config file is valid.

Vispan-2-0-2

	* Change the blocktime from hours to minutes to allow shorter block
	  periods.

	* Added support for the version number of McAfee virus scanner.

	* Amended the whereis path so that the executable MailScanner is
	  detected correctly.

	* Changed locking mechanism to detect stale lock files and remove
	  them.

	* Correct the reporting of IP addresses which would have been
	  blocked except that they are in the whitelist. This resulted in
	  the spam page showing an undefined level.

Vispan-2-0-1

	* Corrected typo - added missing comma.

	* Changed version number

	* Add check to make sure that the size of a message is greater than
	  0.

	* Added config option for the path for whereis command to search.

	* Changed version number.

	* Added support for the version number in McAfee.

	* Changed the regex for Clam since on Linux systems, the log line
	  ends in a space but on Solaris it doesn't!

	* Added support for version number in Antivir. Also tidied up the
	  search string.

	* Format the y-axis values to be more readable for large values.

	* IPTables block should block the destination port 25 not the
	  source port 25.

Vispan-2-0

	* Tidied up the graph x axis labels

	* Changed version number

	* Added config option to specify the location of the whereis
	  command.

	* Added config option to specify where the iptables command is.

	* Only send email when the IP address has actually been blocked -
	  not when it has been detected as a candidate to be blocked but is
	  whitelisted.

	*  Added code to support blocking using iptables

Vispan-1-5-5

	* Changed Version number

	* Bug: 49 - Corrected the code which handles the -r option. It
	  wasn't removing the IP address since the data wasn't loaded
	  before the call to remove the IP address.

Vispan-1-5-4

	*  Bug: 47 - added facility to specify header and/or footer HTML
	  files.
	  Specifying a footer file will prevent the W3C logo from appearing
	  since it is not possible to guarantee that the included files
	  contain valid XHTML.

	* Bug: 45 -  added config option for sendmail logging text.

	* Remove the printing of unknown log entries - this was really a
	  debug feature

	* Changed the detection string for F-Secure to cope with the change
	  in output.

	* Corrected the code for the optional GeoIP changes.

	* Made the GeoIP functions dependent on a configuration setting. So
	  if people don't have the GeoIP library loaded they can still use
	  Vispan.

	*  Update version number.

	* Added command line option to remove IPs from the blocked list.

	* Corrected the storing of theday values so that the day average
	  graphs are generated correctly.

	* Corrected the typo in the Print routine from > to <

	* Added a dependency on the version of MakeMaker to the makefile.

	* Bug: 46 - Fixed the problem of the invalid index when starting
	  with a clean install.

	* Added code to detect what graphics file format is supported and
	  produce that type of graph.

Vispan-1-5-3

	* [no log message]

	* Change to reflect the version number reporting change in ClamAV

	* Corrected the initialisation of stats on various boundaries

	* Make day average graph x axis labels start correctly.

	* Bug 44: Prevented IPs which are whitelisted in MailScanner from
	  being added to the blocked list.

	* Correct a divide by 0 error if there were no mails in an
	  interval.

	* Corrected the HTML to access the right graphs

	* Sorted out the reporting of the average message delay.

	* Changes to add the graphs page.

	* Corrected code for rolling up the average spam score.

	* Added code to report the average spam score for spam and highspam
	  in the interval reports.

	* Tidied up the documentation.

	* Automatically generate the ChangeLog file from the CVS log when
	  the distribution is built.

	* No need for ChangeLog to be in the CVS

	* Change to reduce the font size of the summary tables on the index
	  page.

	* Corrected the path to the default css file. Removed the leading
	  slash.

Vispan-1-5-2

	* Added commenting to the subroutines

	* Correct the reporting of blocked IPs

	* Corrected the code for the number of blocked IPs

	* Tidied up the code to make sure everything is reported correctly.

	* Remove spurious > from spam listing.

	* Correct the CSS for visited links.

	* Tidied up the use of CSS values.

	* Added summary graphs for load, mail, delay and queue. Also tidied
	  up the HTML to make it XHTML 1.0 conformant.

	* removed temporary print statement.

	* Moved the storage of the IP addresses into the dat file.
	  Reformatted the spam output to include colours to show more
	  detail.  Changed the link to dnsstuff as the openrbl one no
	  longer worked.

	* Corrected the code to use the access list.

	* Added routines to detect if an IP address already exists in the
	  access file. If so don't add it. Also change the display of
	  blocked IPs to show those that have reached the block limit but
	  have not been blocked.

	* Added vispan.css to the manifest

	* Added the css file. Changed the way the version number is picked
	  up.

	* Added Licence file.

	* Tidied up HTML to produce valid code. Removed unnecessary MRTG
	  config options.  Removed absolute path names.

	* Remove hardcoded path.

	* No longer use MRTG so remove the file and amend the others
	  accordingly.

	* Added support for Postfix

	* Changes to support the multiple history pages.

	* Added support for Trend and various changes for historical stats.

	* Minor changes for the historical data.

	* Made major changes to layout. Also incorported changes from Wayne
	  Fox

	* Made sure that the thumbs dir was created if it didn't exist.

	* Change version number for build.

	* Added the dependency on Net::CIDR

	* Updated the version number.

	* Added the facility to list IPs in the whitelist by Ip address,
	  range or CIDR format.

	* Made Vispan more Unix generic for system load. Also limited virus
	  lists to the value of DisplayTop

	* Added check for non existent temp file and also changed the
	  tempfile location to /tmp

	  Virus lists are now only listed to the value of DisplayTop.
	  Whitelisted entries are counted.

	* Updated install

	* Modified the installation to create a cron file in /etc/crond.d

	* Tidied up some of the output.

	* Updated version number and corrected minor problems.

	* Fixed the bug in the virus counting which meant that the total
	  virus count only worked if the Use Heuristics options was set to
	  1.

	* Correct the installation of the cron entry to include 0 minutes.

Vispan-1-2

	* Added comment to indicate how to specify multiple scanners.

	* Change version number; correct install location;

	* Correct the virus count reporting and also make the installation
	  easier

	* Updated the MANIFEST to include the Makefile.PL and MANIFEST
	  files.

	* Added the perl makemaker files for ease of installation and
	  distribution

Vispan-1-1

	* Added functionality to block persistent virus sources.

	* Updated comment in conf file relating to scanners.

	* Added support to log the update time for multiple scanners.

	* Added support to report on more than one virus scanner.

	* Added feature to log multiple virus scanners.

	* Added mailing facility Added support for ClamAVmodule Added high
	  scoring spam stats Fixed bug which meant that if you turned on
	  Log Non Spam it would block legitimate senders.  Added facility
	  to configure the number of emails before a host is added to the
	  access list.

	* Corrected bug which meant that IPs were reported as blocked when
	  they hadn't been added to the access db.

	* Changed version number

Vispan-1-0

	* Added instructions on how to setup crontab.  I still need to
	  figure how to do it automatically even if there is something
	  currently in the users crontab.  --Joe

	* Added support for Antivir.

	* Added support for eTrust

	* Corrected minor typos etc.

start

	* Imported files

	* Initial revision

Contact the This e-mail address is being protected from spam bots, you need JavaScript enabled to view it .