Getting Started with Funnel Web Command line

The following article is intended to give users a quick guide to getting started using the Funnel Web Analyzer command line options on Unix,Linux and Mac OSX Commandline platforms. For a comprehensive listing of all Funnelweb Analyzer commandline options, check the Article here

Once you have installed the application successfully using the instructions in the Applications release notes, it is a good idea to have a look at all the available commandline options. To get this information, enter the following command if you are using the Enterprise version:

fwa5 -h

Or if you are using the Freeware version

fwa5-free -h

This will give you a listing of all the commands available to you.

Processing Log Files

To process some log files initially, the simplest way is to enter the following command:

fwa5 -l /logfilelocation/logfile.log

This will process the logfile in the log file location specified and using default settings, will create a html report in a location called /Report/index.html This location will be relative to where you run the application from if you have not specified a report location. Most people want to process more than one log file. There are various ways of doing this. In the following example, two different logfiles are specified to be processed and the report location is also specified:

fwa5 -l /logfilelocation/logfile.log -l /logfilelocation/logfile2.log -o /reportlocation/index.html

This will process the two log files into the report location specified. You can also use wildcards to specify all logfiles in a specific location as in the following example:

fwa5 -l "/logfilelocation/*.log" -o /reportlocation/index.html

This will process all logs in the folder location with the file extension of .log Notice that the log location above is surrounded in " quotes. This prevents the unix shell from seeing * as a command which it tries to decode and allows Funnelweb to identify the logfiles to process correctly. Many web servers include the date of the logfile in the logfile name eg. ex042004.log If you want to use the same command to process the logs all the time, you can use date tokens to process log files as in the following example:

fwa5 -l "/logfilelocation/ex{MM}{DD}{YY}.log" -o /reportlocation/index.html

The command above will process a log file with the Current Month{MM},Day{DD} and Year{YY} in it's log file name. If you want to process a log file from the Previous day, this is also possible by entering the following command:

fwa5 -l "/logfilelocation/ex{MM}{DD-1}{YY}.log" -o /reportlocation/index.html

Note the {DD-1} value in the command above. This causes Funnelweb to identify a log from the previous day. You can also use a combination of wildcards and Path tokens to process say all logs from the previous month as in the following example:

fwa5 -l "/logfilelocation/ex{MM-1}*.log" -o /reportlocation/index.html

Thie example above will process all logs from the month with different days and years in the log name. You are also able to process logs via ftp: or http: access by entering the full address of the log file in the following format:

fwa5 -l ftp://username:password@site/file.log -o /reportlocation/index.html
fwa5 -l http://myserver.com/logs/access_log -o /reportlocation/index.html

Using Settings

Funnelweb uses a settings file to process logs which saves all of your personalized preferences. For instance, the following example saves your preferences for later use in a file called settings.fwp in the location specified:

fwa5 -l /logfilelocation/logfile.log -reportformat rtf -o /reportlocation/index.html -saveprefs /settingslocation/settings.fwp

The command above will produce a report in RTF format in the location specified and will save these settings in the settings.fwp file. To process this log with the same settings in future, you only need to enter the log file and the settings file details as in the following example:

fwa5 -l /logfilelocation/logfile.log -r settingslocation/settings.fwp

This will create an RTF report in the location specified in the settings file. With this method you can use settings files created with any version or platform of FunnelWeb analyzer from version 4.0 onwards. All commandline options and settings are saved within the settings file if you use the -saveprefs command. Settings files once created can be edited with a text editor.

Using a Funnelweb database

Processing large log files can take a long time, and storage of log files can also take up a lot of space. For these reasons, Funnelweb has a database feature which stores all log file information and can be reprocessed later on much faster. Funnelweb has both a compact and extended database in the Enterprise version. In the compact database, all the report information is stored, but not all the log file information. This makes the compact database much smaller and quicker to process. Using the compact database, you need to process log files in date order into the database, as in the following example:

fwa5 -l "/logfilelocation/ex200404.log" -o /reportlocation/index.html -database_active -database_file /databaselocation/database.fdb

fwa5 -l "/logfilelocation/ex210404.log" -database_active -database_file /databaselocation/database.fdb

This will produce a database called database.fdb in the specified location. You can use wildcards to procee all logs in a location, but not if any logs in that location are from a date prior to a date contained in the database. You can reprocess the database as follows to get the statistics recorded in it from the two log files:

fwa5 -l /databaselocation/database.fdb -o /reportlocation/index.html

Notice that you process the database as if it were just another logfile.

The extended database is more flexible when you re-process or add log files into it. The extended database contains all information from the log files in a compressed format. You do not need to process the log files in date order when processing into an extended database, it will simple add the information from all the logs you specify. Be careful not to add the same log file twice ! Here is an example of how you would create an extended database

fwa5 -l "/logfilelocation/ex200404.log" -o /reportlocation/index.html -database_active -database_extended-database_file /databaselocation/{MM}database.fxdb

Note that in the above example, a Month {MM} token is used to create a monthly database. This is a recommended practice, as databases can get very large if you have many logs to process. You can also re-process a Funnelweb extended database as if it were a log file. Another advantage of the extended database is that you can process more than one extended database at a time, which is not possible with the compact database format. This allows you to reprocess databases from two months as in the following example:

fwa5 -l /databaselocation/03database.fxdb -l /databaselocation/04database.fxdb-o /reportlocation/index.html

Network Options

In order to get more information about Visitors to your site, Funnelweb has the abiltity to look up IP address and resolve them to their DNS names. Here is an example of how to use this feature:

fwa5 -l /logfilelocation/logfile.log -o /reportlocation/index.html -dnson

This will attempt to resolve ip addresses for visitors from the web and requires an internet connection. Another feature that requires an internet connection is the Retrieve Page titles feature. This feature goes to the web and gets the title of pages contained in the <title></title> tag and includes this as the Pages name in the Pages report instead of the filename. Here is an example of how to use this feature

fwa5 -l /logfilelocation/logfile.log -o /reportlocation/index.html -remotetitle -siteurl http://www.mysite.com -indexfile index.html

This feature generally works better with a direct internet connection. Note that the site url and index page also has to be specified for this feature to work correctly, as it gives the location for the site where Funnelweb has to look (since this information is not recorded in the log file)

E-mail Notification

Funnelweb has the ability to notify via email when logs have finished processing and attach a zipped report (Enterprise feature only). Here is an example of the command to do this:

fwa5 -l /logfilelocation/logfile.log -o /reportlocation/index.html -post_emailon -post_email address@home.com -post_emailfrom myaddress@home.com -post_emailsub "email subject" -post_emailsmtp mailserverdetails -post_emailattachreport -post_emailmsg "Logs are processed" -saveprefs email.fwp

The above command will generate an email from myaddress@home.com to address@home.com with a subject of "email subject". Your own mail server details must be entered. A zipped report will be attached and the message in the body of the email will say "logs are processed". Since this is a long command, it is a good ide to save these settings via the saveprefs command to avoid re-entering them later.

Summary

This article has given a brief explanation of Funnelweb commandline operation for new users to the commandline. There are a wide variety of features and commands outside the scope of this article, which basically covers some of the more commonly used options by users. Some of the more advanced features include customizing the look and feel of your report, settings for customer billing and advanced settings for processing streaming log file formats. For a full list of these features, check the article here I hope this article has proved useful and gets you on your way to understanding Funnelweb Analyzer commandline operation.

Greg Stehle
Funnel Web Analyzer

This site looks best using Safari or FireFox.

Get Firefox!