So your company has a requirement to maintain log files for a year?
You don’t know how to go about it and you need to implement it now?
I have a solution for you and best of all it’s free. This solution however is not supported by me, there will be no bug fixes by me, and any damages you cause to your own servers is your own fault. That is my one sentence disclaimer to tell you that you truly are on your own.
For this solution or temporary fix (depending on your organization) you are going to need the following helper programs:
Info-zip – we’ll use this to compress down files and save space specifically we need the zip.exe file
MD5SUMS – this allows us to generate MD5 checksums to verify if any file tampering has taken place after the fact. Specifically we need the md5sums.exe file.
Dump Event Log (Dumpel.exe) – this is a tool offered by Microsoft to dump your event logs to a text file. Though this link is part of the Windows 2000 Resource Kit I have tested and it does work with the Windows XP and Windows Server 2003 log files.
We take these 3 programs and wrap them to work together via a batch file. All 3 of these programs MUST be in the same directory as the batch file for it to work as designed. Here is the batch file:
@echo off
REM Sets date variables for file name
for /F “tokens=1,2″ %%d in (‘date /T’) do set day=%%d & set date=%%e
set yyyy=%DATE:~6,4%
set dd=%DATE:~3,2%
set mm=%DATE:~0,2%
set startDate=%yyyy%-%mm%-%dd%
REM Adds Computer to prefix the date
set outputname=%computername%-%startdate%
REM Cleans out previous zip files from a bad run
del %outputname%.zip
REM Dumps each of the log files going back for 2 days
REM allowing for overlaps we may miss due to time changes
dumpel -f %outputname%.sec -l security -d 2
dumpel -f %outputname%.app -l application -d 2
dumpel -f %outputname%.sys -l system -d 2
REM creates an MD5 hash for verification checking
md5sums %outputname%.sec >%outputname%.md5
md5sums %outputname%.app >>%outputname%.md5
md5sums %outputname%.sys >>%outputname%.md5
REM Compresses the 4 files
zip %outputname%.zip %outputname%.*
REM Cleans up the unneeded files to save
del %outputname%.sec
del %outputname%.app
del %outputname%.sys
del %outputname%.md5“
I’ve included my comments in the batch file – but let’s go through it a section at a time so you can fully understand it.
@echo off
“
If you don’t know @echo off supresses everything from your screen in a batch, I wouldn’t suggest modifying my script since this is batch file programming 101.
for /F “tokens=1,2″ %%d in (‘date /T’) do set day=%%d & set date=%%e
set yyyy=%DATE:~6,4%
set dd=%DATE:~3,2%
set mm=%DATE:~0,2%
set startDate=%yyyy%-%mm%-%dd%
REM Adds Computer to prefix the date
set outputname=%computername%-%startdate%“
This section adds the prefix to the files we are going to be using on all of your files – these allows us to work with files that include the computer’s name you are running this on and the date on which it was run.
del %outputname%.zip
“
This verify actually cleans up the zip file if this script has already been run during the current day. Mine is modified to delete the zip file completely since at the end of my script I move my files to a remote location and don’t need archived logs filling up my hard drive quickly.
dumpel -f %outputname%.sec -l security -d 2
dumpel -f %outputname%.app -l application -d 2
dumpel -f %outputname%.sys -l system -d 2“
This area does the physical dumping of the logfile. Dumpel is the command. The -f switch allows us to specify a file name. If you notice I used the %outputname% as the first part of the file with the file type of which log file it is as the suffix. The -l switch let’s us specify which logfile we are dumping from the event log (security, application, or system). the -d switch allows us to specify how many days we wish to save. I chose 2 days to allow some overlap on the log files which is good for security reasons since we shouldn’t miss any events if you change the time of the day the script is run. It also give us two more logfiles to verify the authenticity of the log data we are looking at.
When this section is done running you should have three files. If your computer’s name was BOB-SERVER and the date you ran this was one January 3, 2007 the file names would read like this; BOB-SERVER-2007-01-03.sec for your security log, BOB-SERVER-2007-01-03.app for your application log, and BOB-SERVER-2007-01-03.sys for your system log.
md5sums %outputname%.sec >%outputname%.md5
md5sums %outputname%.app >>%outputname%.md5
md5sums %outputname%.sys >>%outputname%.md5“
This section generates an MD5 Hash of the logfile data allows you to see if the data was tampered from when it was originally generated. It is next to impossible to edit a file and maintain the same hash data. This allows you some security that your log files are authentic. For those wondering “well can’t I just rerun the md5 program and generate a new hash and save that after modification?” – I have your answer. I didn’t include how to store these files after they are generated and we will touch upon that question under “What do you do now?” at the bottom. The command outputs your three BOB-SERVER-2007-01-03 files and output it to a single BOB-SERVER-2007-01-03.md5 file that includes a section with each of the above files. I decided personally that I didn’t need an md5 file for each of them – feel free to modify this if your needs differ.
zip %outputname%.zip %outputname%.*
“
This compresses the dumped events logs down to a manageable size. I managed to get a 60 MB log file that I generated during varying testing phases down to just over 6 MB. I also manged to get 480 KB of log files down to 14kb. At this point you should have a BOB-SERVER-2007-01-03.zip file which includes your three event logs and you md5 file.
del %outputname%.sec
del %outputname%.app
del %outputname%.sys
del %outputname%.md5“
This section cleans up the files outside of the zip. I manage t0 get these files down 90% in size I don’t need these to eat up extra space.
What do you do now?
From here I would add in a line at the end to move the logs to another server where you can store them for length your organization deems necessary. To help combat the MD5 re-engineering I mentioned above I would copy the compressed archived to two locations on your network. This will help make having an MD5 meaningful. Another option is adding a script that e-mails you the MD5 hash so you have it saved for reference. Having the MD5 and collecting 2 days of information from the logs and would mean an attacker may have to edit 2-4 archives and regenerate md5′s for them – double that if you store a second set of archives in another location.
While this may fulfill your needs for log file capturing and an easy way to store them, it does not address the fact of easy log file auditing and tracking down events. There are all in one solutions out there for you to use and I don’t say in any terms this a solution to those. You need to address your own needs and decide what works for you. This is to give you sometime until your decide what you are going to do.
Source: http://creeva.com/2008/01/18/cheap-and-dirty-windows-log-file-archiving/
ITSolutionDesign (Martin's technology notepad) 