UPDATE: Due to a system crash and my poor backups, I no longer have the code for this post :( I'll try to reproduce it if I have time, but that's harder and harder to come by
OK, so I have the squid shaga engine code logging most everything to a MySQL table.
In order to successfully compile squid with this patch you will need liburi. You can download the pristine source for liburi here:
http://savannah.nongnu.org/cvs/?group_id=301
I grabbed the HEAD branch out of CVS which is marked as version 2.14.
I've also provided Fedora 10 binary RPMS and source RPMs for liburi for Fedora and CentOS/RHEL5. The source RPMs can be used to build an RPM for any RPM-based Linux distro.
Grab all the code you need here:
http://www.itnervecenter.com/downloads/squid/
This patch applies directly to the squid 3.0STABLE13 source RPM currently available in Fedora 10. I've also checked and it applies and works with the direct source download from the Squid website.
I'll be building a CentOS/RHEL5 RPM via my Koji build system soon and I'll post that up here as well as the source RPMS.
[WARNING] - this code is not well tested and probably has memory leaks galore so only use it in a test environment.
Enjoy.
----------------------------------
I've been working on trying to get Squid (http://www.squid-cache.org) to replace our proprietary web proxy at work and I couldn't believe that even the most recent versions do not have a mechanism to log directly to a database. There is a branch in the squid source code control system (called shaga) that does have this functionality, but the patch doesn't apply cleanly to recent versions of Squid.
I spent yesterday modifying the patchset (original version available at http://www.squid-cache.org/~hno/changesets/squid3/patches/6729.patch). I've got a patch working with the current source RPM provided by the Fedora project.
The ShagaEngine only logs 4 pieces of data from the Squid access log so it has limited information available in the MySQL tables, but it's a start.
I'm working on modifying the ShagaEngine to log more robustly so that it logs into a table that looks more similar to the ones that need to be created for the insertdb2.php script written by Adam Delves. I hope he does mind me using his DB schema :)
You can download my updated Squid ShagaEngine patch, the necessary DB sql script to create the MySQL DB, and a source RPM below:
Once you've installed the new squid or rpm you'll need to add the following to your squid.conf and modify it to suit your environment:
# TAG: enable_shaga_engine on|off
# When this set to on,squid will write some entries from access.log file into database
#
#Default:
enable_shaga_engine on
# TAG: shaga_db_host mysql server ip address
# TAG: shaga_db_port listening port of mysql server
# TAG: shaga_db_name shaga database name
# TAG: shaga_db_user shaga database user
# TAG: shaga_db_pwd shaga database password
shaga_db_host localhost
shaga_db_port 3306
shaga_db_name test_shaga
shaga_db_user shaga
shaga_db_pwd shaga
My plans for the updated ShageEngine code include logging more information to the SQL table, adding a config file option for the name access log table in the database (currently hardcoded to tmp_traff in shaga.h), and possibly being able to specify your table schema as well in squid.conf
Check back soon for more updates.
Enjoy.
