SQL Server Data Mining—Extracting Information from IIS Logs

Microsoft .NET Framework, ASP.NET, Visual C# (CSharp, C Sharp, C-Sharp) Developer Training, Visual Studio

Extracting Information from IIS Logs

Although in our example solution, we will be taking advantage of the log parsing facilities that are built in to the e-commerce application, many companies have built a custom Web site where the only available tracking information is the raw IIS logs.

The first step in extracting this information is to parse the log files and extract the information into a staging database. You could create an Integration Services package to perform this extraction, possibly with an additional tool to make the data easier to work with. Microsoft has a free Log Parser utility (www.logparser.com), and third-party parsers are also available.

However, after you have extracted the raw information, the real fun begins, and is not for the faint of heart. Finding discrete "sessions" involves looking for an identifier in the logs such as username or cookie and then identifying a time period that could identify a "visit" fact.

If you also want to look at what pages the users visited, you need to parse URLs to deal with pages that are parameterized with identifiers such as product IDs (for example, product.aspx?ID=322442). On the whole, it is generally much easier to take advantage of an e-commerce application's existing parsing facilities if they exist, or otherwise find an existing tool that meets your needs.