 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Site Log Analysis Case
Study
Tips for Customizing
Server Log Report Tools
Published Spring 2001
As mentioned before, the client's hosting company provides monthly server log reports created by a server-based WebTrends solution. Although we found out that the hosting company would change the default settings if asked, no one had bothered to do so. What a waste! A lot of very useful data was lost forever.
Here are some tips you can follow for customizing your own server log analysis tool. Whether you use WebTrends, Analog, or any of the other useful tools out there, these tips are general enough to help you out, as well.
Apply
Filters Wisely
Any reporting tool worth its exhorbitant licensing fee will allow you to exclude "noise"-causing data from the reports. I recommend you set your exclusion lists to filter out at least the following:
1.
Extraneous files
You only want the reporting tool to focus on files whose pageviews you
actually care about. For example, if your site uses frames for some reason,
set the reporting tool to ignore all of the frameset files.
2.
Inapplicable referrers
A "referrer" is the page the user was on before the current
page. Most servers, when asked to give a new page, will in return ask
the browser what URL it was at when the user clicked on the link. This
is fascinating data that can help you understand who else in the world
is linking to you. Then, if you can figure out why, you may be able to
more effectively drive future traffic.
Unfortunately, reporting tools aren't terribly smart. If the user is on one page in your site, then clicks on a link to any other page in your site, the reporting tool will count the first page as a referrer. So set the reporting tool to ignore any referrer URLs beginning with "www.yoursite.com". Be sure to add "yoursite.com" minus the "www" as, again, reporting tools are dumb.
3.
Irrelevant users
The web server captures the IP address of each visitor's computer when
it requests pages from your web site. To avoid skewing the usage stats,
set the filter to ignore IP addresses including your company, your client's
company, and at your discretion, users from your competitor. However,
do not filter out IP addresses for your client's competitors, as that
can prove very interesting to track.
Reporting
Numbers
In addition to filtering out the bad stuff, you want to make sure you have enough of the good stuff. WebTrends, for example, only tracks the 10 most popular pages in your site by default. Well, what if your site is moderately sized with over 500 pages? A list of 10 popular pages would be way to short to do your forensics investigation much good. Depending on the size of your size, you might benefit from having your report list the top 100 or even 500 pages!
Here is a list of the types of stats your reporting tool is probably capable of creating. For the modestly-sized consumer product site I studied, I would have customized the report for the next reporting period to include 50-100 items each.
- Most requested pages
- Least requested pages
- Top entry pages
- Top exit pages
- Single access pages
- Top paths through the site
- Visitors by number of visits
- New vs. returning visitors
- Top visitors
- Most active organizations (specifically, who their ISP is)
- Activity by length of visit
- Number of views per visitor session
- Visitor session stats
- Top referring sites
- Top referring URLs
- Top search engines
- Top search engines with phrases detail
- Top search engines with keywords detail
- Top search phrases with engines detail
- Top search keywords
- Top search keywords with engines
Other Links in this Case Study
- Case Study Introduction
- Lessons Learned
- Tips for Customizing Server Log Report Tools (this page)