#75938 - 2003-07-17 09:11 PM
Log Management / Rotation / Archiving toolkit
|
Glenn Barnas
KiX Supporter
Registered: 2003-01-28
Posts: 4396
Loc: New Jersey
|
Do you have 15 applications writing to log files? Do you have 15 different scripts to manage those log files? If not, are your log files taking over your system?
NOT ANY MORE!
Here's a KiXtart script package that will do all the work for you just by maintaining a single configuration file.
Now, to make Lonk happy, the script comes first, and the documentation follows! Caveat: You may want to change the "\usr\local" and "\Temp" references before using this code. These are standards in our environment. I prefer NOT to use %TEMP% because it is different for each user on 2K and higher (and I've had higher priorities than to change it on 300+ servers).
; Utility to archive and remove log files based on age ; Glenn Barnas / FRIT-EROC ; 3-4-2002 ; ; VERSION 1.3 ; ; Obtains settings from \usr\local\logmaint.ini, based on the SECTION argument ; ; Must be called as "kix32 \path\to\this\script\logcleanup $SECTION=section_name" ; See the LogMaint.BAT file for more information. ; ; see readme.txt or "Common Log Cleanup utility.doc" for more info ; 06/11/02 - Corrected error in debug code, added DEBUG option to the ini file, ; Added test code - MD command executes only if target doesn't exist ; Enclosed path parameters in quotes to support paths with spaces ; when used by shell commands (ugh!) ; 06/13/02 Added code to write errors to log file and log errors to LMSTATUS.ini ; to facilitate central log status collection.
BREAK on ; allow terminating the script without logging off ; this basically turns off "login script mode"
; NOTE - throughout the script, the $RTN variable is used to collect the return code ; of various functions. It is tested only when necessary. Its primary function ; is to prevent the return code from being displayed on the console (as a ; series of zeros or other numeric values)
;================================================================================= ; If the section argument isn't defined, carp and die If $SECTION = "" ? "Required parameter to identify logs to archive was not specified." + @CRLF + "Can't continue!" Exit EndIf
; format today's date for use in file names $DATE = Substr(@DATE,1,4) + Substr(@DATE,6,2) +Substr(@DATE,9,2)
; Read the parameters from the LOGMAINT.INI file ; Wrapping the READ statement in the EXPANDENVIRONMENTVARS function allows ; environment variables to be defined in the INI file for the path values ; (ie SRCDIR=%SYSTEMROOT%\logs) $INIFILE = "%SystemDrive%\usr\local\logmaint.ini" $SRCDIR = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "SRCDIR")) $DSTDIR = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "DSTDIR")) $DEBUG = ReadProfileString($INIFILE, "COMMON", "DEBUG") $SFLIST = ReadProfileString($INIFILE, $SECTION, "FILES") $BACKUP = ReadProfileString($INIFILE, $SECTION, "BACKUP") $DELETE = ReadProfileString($INIFILE, $SECTION, "DELETE") $MAXAGE = ReadProfileString($INIFILE, $SECTION, "AGE") $SERVICE = ReadProfileString($INIFILE, $SECTION, "SERVICE") $PREARCH = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "PREARCH")) $POSTARCH = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "POSTARCH"))
; define the archive log name and location $AFILE = "_" + $DATE + "_archive.wri" $ARCHIVELOG = $DSTDIR + "\" + $AFILE $FIRSTYEAR = 2001 ; anything older than this year is not calculated, just deleted
;================================================================================= ; Make sure that the DSTDIR exists before we start $P = $DSTDIR ; use a working VAR $MP = "" ; Make this Path
If InStr($P, ":") = 2 ; If the second char is a ":" $P = SubStr($P, 3, 252) ; drop the leading drive definition $MP = Left($DSTDIR, 2) EndIf
if Left($P, 1) = "\" $P = Substr($P,2,99) ; remove the leading dir delimiter for now EndIf
$DIRS = split($P,"\",-1) ; put the individual dirs of the path into an array for each $D in $DIRS $MP = $MP + "\" + $D ; add the next dir to the root path If Not Exist($MP) ; make sure each dir in the path exists md $MP ; create it if necessary EndIf
If $DEBUG = "T" ? "processing $D" ? "$MP" ? "@SERROR" EndIf
Next
; Open the archiving log file $RTN = Open(1,$ARCHIVELOG,5) If @ERROR ? "Error opening archive log file - exiting @CRLF FILE: $ARCHIVELOG" exit EndIf
; Append today's date and a message to the log file $RTN = WriteLine(1,"Log cleanup executed on @DATE using [" + $SECTION + "] parameters." + @CRLF)
; Update the LMStatus file $TMP = ReadProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "LastRunD") If $TMP = "" $TMP = "Never" ; LastRunD doesn't exist, set PriorRun to "Never" EndIf $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "PriorRun", $TMP) $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "LastRunD", @DATE) $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "LastRunT", @TIME)
;================================================================================= ; If BACKUP <> 0, then MOVE the file(s) from the SRCDIR to the DSTDIR, adding the ; current date to the front of the file name (test.log becomes 20020301_test.log) ; Using MOVE in the same directory results in a simple RENAME operation! If $BACKUP <> 0
$RTN = WriteLine(1,"Archive process starting." + @CRLF) $CNT = 0 $ECNT = 0
If $SERVICE <> "" ; Stop any service defined prior to archiving If $DEBUG = "T" ; to insure the files can be moved/renamed ? "xnet stop " + Chr(34) + $SERVICE + Chr(34) Else Shell "xnet stop " + Chr(34) + $SERVICE + Chr(34) EndIf EndIf
; PREARCH/POSTARCH - commands to run prior to archiving. An alternative method ; for stopping but not starting services that require manual password entry, or ; procedures that require multiple steps to prepare the files for archiving. ; It is assumed that the creator of the config file will include any quotes or ; other escape sequences to handle proper argument passing!!! If $PREARCH <> "" ; If a PREARCH command is defined If $DEBUG = "T" ? "running $PREARCH" Else Shell "$PREARCH" ; run it! EndIf EndIf
; Loop through all files in the SRCDIR to see what should be MOVED $FILES = Split($SFLIST, ",", -1) ; split the list into an array For Each $FILE in $FILES ; process each arg in the array $SPATH = $SRCDIR + "\" + $FILE $FNAME = Dir($SPATH) ; find the matching files in the SRC directory
While $FNAME <> "" And @ERROR = 0 ; process all non-null names ; skip parent paths ("." & "..") and previously archived files If Left($FNAME,3) <> "A-!" And $FNAME <> "." AND $FNAME <> ".." $MSG = " " + $SRCDIR + "\" + $FNAME + " -> " + $DSTDIR + "\A-!" + $DATE + "_" + $FNAME + @CRLF $RTN = WriteLine(1,$MSG) $CMD = "cmd.exe /c move " + CHR(34) + $SRCDIR + "\" + $FNAME + CHR(34) + " " + CHR(34) + $DSTDIR + "\A-!" + $DATE + "_" + $FNAME + CHR(34)
If $DEBUG = "T" ? "$CMD" Else shell $CMD ; move/rename the file If @ERROR <> 0 ; if an error occured, ? ? "@SERROR" ; display the error and write it to the log $RTN = WriteLine(1," @SERROR") $ECNT = $ECNT + 1 ; increase the error count EndIf EndIf
$CNT = $CNT + 1 EndIf $FNAME = Dir() ; get the next file Loop Next
$RTN = WriteLine(1,"Archive process complete, " + $CNT + " files archived." + @CRLF + @CRLF) ; write to the common log showing # of files archived and number of errors encountered $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "Archive", $CNT) $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "ArchErr", $ECNT)
If $POSTARCH <> "" ; run a post-acrhiving command if defined If $DEBUG = "T" ? "$POSTARCH" Else Shell "$POSTARCH" EndIf EndIf
If $SERVICE <> "" ; start a service if defined If $DEBUG = "T" ? "xnet start " + Chr(34) + $SERVICE + Chr(34) Else Shell "xnet start " + Chr(34) + $SERVICE + Chr(34) EndIf EndIf
Else ; rather than write nothing (and leave the viewer guessing), say that we did nothing! $RTN = WriteLine(1,"Archive process not performed, 0 files archived." + @CRLF + @CRLF) $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "Archive", 0)
EndIf ; BACKUP PROCESS
;================================================================================= ; If DELETE <> 0 then DELETE all files in DSTDIR older than MAXAGE days that have ; the auto-archive filename prefix of "A-!" ; Include "*_archive.wri" (our own log file) in the list of files that we check If $DELETE <> 0
$RTN = WriteLine(1,"Cleanup process starting." + @CRLF) $CNT = 0
; Create an array of monthly day values (cumulative days, 13 values including the ZERO arra y cell) ; TO keep the logic simple, the array starts with a ZERO postion, but we use only  ;positions 1-12 ; which correspond to the month integer values. Position ZERO holds "0" as a placeholder on ly. $MV = split("0,0,31,59,90,120,151,181,212,243,273,304,334", ",", -1)
; calculate today's date in days from 1/1/2001 (first day of $FIRSTYEAR) $CDATE = split(@DATE, "/", -1) $X = Val($CDATE[1]) $CDV = ((Val($CDATE[0]) - $FIRSTYEAR) * 365) + $MV[$X] + $CDATE[2]
; archived files begin with A-! or are our own log files ; if we don't do archiving (BACKUP=0) then archived files are defined ; in the SFLIST variable. The SFLIST value is included when we DO archiving ; ONLY when the SRCDIR and DSTDIR values are different. If $BACKUP = 0 $ALLFILES = $SFLIST + ",_????????_archive.wri" Else $ALLFILES = "A-!*,_????????_archive.wri" If $SRCDIR <> $DSTDIR ; Add SFLIST if SRC and DST dirs are different $ALLFILES = $SFLIST + "," + $ALLFILES EndIf EndIf
; Loop through all files in the DSTDIR to see what should be DELETED $FILES = Split($ALLFILES, ",", -1) ; split file list into array For Each $FILE in $FILES ; process the files in the array $DPATH = $DSTDIR + "\" + $FILE $FNAME = Dir($DPATH) ; read the directory While $FNAME <> "" And @ERROR = 0 If $FNAME <> $AFILE ; get timestamp $FD = Left(GetFileTime($DSTDIR + "\" + $FNAME),10) $FDATE = split($FD, "/", -1) If Val($FDATE[0]) < $FIRSTYEAR ; prior to FIRSTYEAR? ANCIENT FILE!! If $DEBUG = "T" ? "Del " + $DSTDIR + "\" + $FNAME Else Del $DSTDIR + "\" + $FNAME EndIf $CNT = $CNT + 1 ; Append the deleted file to the log file $RTN = WriteLine(1," " + $DSTDIR + "\" + $FNAME + " - " + $FD + " - ANCIENT!" + @CRLF) Else ; convert to Days since 1/1/2001 if year is 2001 or later $X = Val($FDATE[1]) $FDV = ((Val($FDATE[0]) - $FIRSTYEAR) * 365) + $MV[$X] + $FDATE[2]
; find number of days between file date and today ; delete $FNAME if difference is greater than $MAXAGE If ($CDV - $FDV) > $MAXAGE If $DEBUG = "T" ? "Del " + $DSTDIR + "\" + $FNAME Else Del $DSTDIR + "\" + $FNAME EndIf $CNT = $CNT + 1 ; Append the deleted file to the log file $RTN = WriteLine(1," " + $DSTDIR + "\" + $FNAME + " - " + $FD + @CRLF) EndIf EndIf EndIf $FNAME = Dir() Loop Next
$RTN = WriteLine(1,"Cleanup process complete, " + $CNT + " files deleted." + @CRLF + @CRLF) $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "Cleanup", $CNT)
Else
$RTN = WriteLine(1,"Cleanup process not performed, 0 files deleted." + @CRLF + @CRLF) $RTN = WriteProfileString("%SystemDrive%\LMStatus.ini", $SECTION, "Cleanup", 0)
EndIf ; DELETE
; Close the log file $RTN = Close(1)
Here's a batch file the I use to invoke the script. It makes it easier to create a scheduled task with "logmaint LogID" instead of "kix32 logcleanup.kix $SECTION=LogID"
code:
@echo off REM Front-end batch file used to pass arguments to the LogCleanup script REM Glenn Barnas / Frit - 6/2002 REM This is the script called from the scheduler - it simply invokes the Kix REM script with the provided parameter. REM One parameter is required - if it's blank, go complain If "%1"=="" goto USAGE REM run the kix script, passing the SECTION value kix32 c:\usr\local\bin\logcleanup.kix $SECTION=%1 goto DONE :USAGE echo. echo Must specify SECTION name from \usr\local\logmaint.ini file echo. :DONE
And here's a sample INI file used to manage the event logs (daily) and the iPlanet LDAP logs (weekly)
code:
[COMMON] DEBUG= [EVENTLOG] # Export / Clear the event logs, move them to the Archive folder, # delete all archived event logs older than 30 days SRCDIR=C:\TEMP DSTDIR=D:\Archive\Event Logs FILES=APP.evt,SEC.evt,SYS.evt BACKUP=1 DELETE=1 AGE=30 # dump the event logs to local files prior to archiving them PREARCH=kix32 c:\usr\local\bin\eldump.kix [LDAP] # backup the logs each week, delete logs older than 30 days SRCDIR=D:\iPlanet\Servers\slapd-%COMPUTERNAME%\logs DSTDIR=D:\archive\iPlanet\slapd-%COMPUTERNAME% FILES=access,errors BACKUP=1 DELETE=1 AGE=30 PREARCH=net stop slapd-%COMPUTERNAME%
And, as an added bonus if you order in the next 30 minutes, the ELDUMP.KIX script referenced in the sample above.
; ELdump.kix - dump all event logs prior to archiving ; Glenn Barnas / Network Management component ; FRIT - 6/10/2002
; Remove any prior .EVT files from the temp location ; Dump (backup) the requested event log ; If no errors occur while dumping, Clear the requested event log ; log maintenance moves the newly dumped .EVT files to an archive location ; using a name that identifies the date and data source (A-!20020913_APP.EVT)
Break on
; make sure the TEMP folder exists! md "%SystemDrive%\temp"
; make sure that any previous .EVT output files don't exist... del "%SystemDrive%\temp\*.EVT"
$RTN = BackupEventLog ("Application", "%SystemDrive%\Temp\APP.EVT") If @ERROR = 0 $RTN = ClearEventLog ("Application") EndIf
$RTN = BackupEventLog ("Security", "%SystemDrive%\Temp\SEC.EVT") If @ERROR = 0 $RTN = ClearEventLog ("Security") EndIf
$RTN = BackupEventLog ("System", "%SystemDrive%\Temp\SYS.EVT") If @ERROR = 0 $RTN = ClearEventLog ("System") EndIf
OK - now some documentation:
Common Log Cleanup utility The goal of the Common Log Cleanup utility is to provide a single tool to perform log management. Using a single tool consolidates development effort and minimizes post-deployment troubleshooting. The basic operational parameters of this script are maintained in a configuration INI file. Specifying the section name of the INI file causes the parameters from that section to be read and processed. This allows one script and one configuration file to manage dozens of application log files. To properly invoke the script, the following syntax must be used to pass the SECTION argument:
Kix32 c:\usr\local\bin\logcleanup.kix $SECTION=name
A batch file – logmaint.bat – is included to simplify calling this script. It takes one argument, the name of the section from the INI file.
The configuration file is a standard INI file format. It can support a large number of log archive definitions. This is a plain ASCII text file that uses the following entries:
[NAME] SRCDIR=c:\logs DSTDIR=c:\logs\backup FILES=*.log,events.txt BACKUP=1 DELETE=1 AGE=30 SERVICE=svcname PREARCH=prerun.bat POSTARCH=postrun.bat
In the above example, any file in the C:\logs directory that ends with “.LOG”, and the specific file “events.txt” will be moved to the C:\Logs\Backup directory as specified by the DSTDIR value. The files that are moved will be renamed with the current date, using the format “A-!YYYYMMDD_originalname”. The use of the “A-!” filename prefix indicates that it was automatically archived and aids in the later cleanup process. The file archive process is performed because the BACKUP value is not zero. If it were zero, then no archiving would be performed. This behavior may be desired when the application itself creates a new log file on a regular basis, but does not remove the old files. (This is the case in many web server applications such as IIS.)
Once the archive process is completed, if the DELETE value is non-zero, the script will examine the files in the DSTDIR folder. Any files that have an “A-!” filename prefix will be examined and deleted if the file is more than AGE days old. The age comparison is based on the number of days since the file was last modified and the current date. The day that the file was archived (the date part of the filename) is not evaluated. This should be taken into account when the AGE value is defined! Note that if BACKUP=0, then the DSTDIR is scanned for the files named in the FILES key, since it is unlikely that any would have the auto-archive prefix.
The script maintains its own archive log, listing the date that the archive and cleanup process was performed, and listing each file that is archived or removed. When DELETE actions are performed, these archive script logs older than AGE days are also deleted.
In addition to the per-application log, a common log is maintained in an INI format in the root of the C: drive. In the application_name section, keys are maintained for the most recent run date and time, along with values for the prior run date, number of files archived, and number of files deleted. Since this file only has information about the last run, it can be quickly scanned (by person or program) to determine if the log maintenance was successful.
The script has the capability of stopping a service before archiving, and starting it again after archiving is complete. Note that some services will be in a start-wait state if a certificate password is required! Operations staff should be instructed to log in and provide the passwords when these situations occur. A better method might be to use an external batch file to stop the service, call the archive script, and not restart the service, requiring a manual login and service start instead. A facility, described below, can provide this functionality.
The PREARCH and POSTARCH values allow external programs or batch files to be called. The PREARCH command, if defined, is called after any service is stopped, and before the archive is begun. Conversely, the POSTARCH command is called after archive is complete, and before any defined service is started.
Setup Scripts The installation is performed via the standard SWDIST tool install. The installation script accepts two optional parameters. One is used to specify a remote system name, while the other indicates that a cleanup task should be added.
The first step is to install the tools and configure the basic tasks. By not specifying the “-a” parameter, the system will perform an initial install. This consists of installing the script and configuration files, and scheduling basic O/S tasks. For all windows servers, a task is scheduled to clean up the event log files dumped by the elvis program. Any file in D:\Archive\Event Logs older than 30 days is deleted. For DMZ servers, where elvis does not operate, the task first dumps and clears the event logs, then clears the old files.
For NT-4 systems, a PerfMon log rotation task is scheduled for Sunday nights. This task stops the service, removes the prior week’s log, renames the current log, and restarts the service. This task is unnecessary on W2K systems, which use a circular logging format.
To schedule an additional log cleanup task, specify a “-a” parameter on the command line. When this parameter is encountered, the script reads the configuration file and obtains the list of defined events. You may enter one of the event names displayed, or define a new one. You will be prompted for all of the settings required to schedule the event.
With either of the above tasks, you can specify the process to be directed to a remote system by specifying “-r:server_name”. The script will confirm that the target system has the prerequisite scripting tools installed before proceeding.
To aid in configuring special events, a confedit tool is present in the \logcleanup\support directory. This command can be used to edit the config file on the local system, or – by using the “-r:server_name” parameter – remote system. This is a simple, text based editing tool that displays the current parameters of a task, and allows them to be saved under existing or new task names.
INI File Format SRCDIR and DSTDIR are standard path statements, including drive letters, but without a trailing slash. (Ie: d:\archive\logs). Both values may be the same if archiving within the same directory is desired. SRCDIR defines the location of the original log files, while DSTDIR defines the folder that should hold the archived copies.
Environment variables may be included in these specifications, as may be necessary when maintaining Netscape web or directory services. Thus, a path specification of D:\Netscape\Suitespot\https-%COMPUTERNAME%\logs is valid. This permits a more generic INI file to be distributed and minimizes the per-server customization.
FILES can be a comma-separated list of specific file names or wildcard specifications (Ie: "*.evt,this_file.txt,*.log"). The only limit is that the string should be less than 250 characters in length.
The BACKUP and DELETE parameters are 0 (ZERO) when disabled, any non-zero value when enabled. Enabling BACKUP causes files in the SRCDIR folder to be moved to the DSTDIR folder using the naming convention described above. Enabling DELETE will cause files in the DSTDIR folder having an “A-!” name prefix to be deleted if they are more than AGE days old. In addition to these files, any archive logs (_date_archive.wri) older than AGE days will also be deleted. If BACKUP is disabled, (for situations where the application rotates log names) the delete process changes slightly. It looks for the files specified in the FILES parameter in the DSTDIR folder and deletes them if they are older than AGE days, along with any log files.
AGE is the maximum age of a file, in days, before a file in the DSTDIR folder will be deleted if DELETE is set and the file is more than AGE days old.
The following parameter/value pairs are optional
SERVICE is the name of the service that should be stopped before archiving and then restarted when archiving is complete. Services are NOT stopped when only DELETE actions are performed. It is only necessary to stop services when active log files are archived – DELETE only purges old (inactive) log files.
PREARCH specifies the name of an executable or batch file to run prior to the BACKUP process. It is called after any defined service has been stopped.
POSTARCH specifies the name of an executable or batch file to run after the BACKUP process. It is called after any defined service has been started.
WARNINGS! If services are holding log files open, they must be stopped by external means prior to running this script, and restarted again after this script completes, or by specifying the SERVICE parameter.
When SERVICES are restarted, a password may be required. MANUAL INTERVENTION is necessary to enter the password after logging in with admin rights. Use caution when using this option – it is best to schedule the archive the night before a scheduled reboot
If the PREARCH or POSTARCH references a batch file or a DOS command, you should invoke the command-interpreter to run the command. For example:
PREARCH=%COMSPEC% /c yourcommand.bat
A support script – ELdump.KIX – is available to dump and clear the server’s event logs. This script should be called as part of the EventLog process via the PREARCH function. The script dumps the logs to C:\Temp, clears the logs, and exits. The main script then moves these files to the archive directory, usually D:\ARCHIVE\Event Logs. The ELdump script should be called as
PREARCH=kix32.exe c:\usr\local\bin\ELdump.kix The task should be scheduled for 23:59 each night so that all events of the current day are in the resulting log files. While it is possible for up to 50 seconds of events to be logged into the next day’s file, this is significantly less troublesome than having the resulting log files dated a day later than the data they actually contain.
Keywords: Log cleanup, Log Archiving, Log Rotation, Log Management
_________________________
Actually I am a Rocket Scientist!
|
Top
|
|
|
|
#75940 - 2003-07-18 01:32 PM
Re: Log Management / Rotation / Archiving toolkit
|
Glenn Barnas
KiX Supporter
Registered: 2003-01-28
Posts: 4396
Loc: New Jersey
|
Actually, we're legally bound by our audit department to maintain copies of our computer logs for a minimum of 2 years, and 7 years in certain cases. We keep them on tape (via the mainframe), but it was a nightmare to maintain individual scripts to stop a service, copy the log to an archive location, give it a unique name, and restart the service. We had - in one case - 4 copies of the same script in 4 locations to back up logs for 4 instances of a web server on the same machine. Each script was just slightly different in structure, the result of different people troubleshooting various problems. This script was the solution. 1 script in 1 location, managing any number of logs. One of our servers has 11 logs managed with this script - Event Log, PerfMon Log, 3 web instance logs, 2 LDAP logs, 2 Cold Fusion logs, and 2 SilverStream logs!
Oh - we don't ignore the logs, either. We have scripts (perl, sorry!) that parse the logs, strip off the info messages, then summarize the warnings and errors in an email each morning to the admin staff. This lets us get a quick view of the kind & number of warnings on each of the 200+ servers. With 300 more being deployed this year, we rely heavily on automation.
Glenn
_________________________
Actually I am a Rocket Scientist!
|
Top
|
|
|
|
#75941 - 2004-03-09 12:33 PM
Re: Log Management / Rotation / Archiving toolkit
|
The_Apprentice
Fresh Scripter
Registered: 2003-05-30
Posts: 16
Loc: Guildford, England
|
Nice utility.
We want to archive off the log files at a preset date (eg end of month) and then delete the files in the source directory that have been archived (to cleanup).
At the mo yours removes all files from the source.
We will then delete the archive about a week later using a seperate script.
This will allow time to ensure we have had a good backup.
Do you have a variation of your utility to do this, or know of one that can?
Failing that, do you mind if I hack your code apart, unless you fancy doing it
EDIT: Thinking aloud............
We could use another variable in the .ini file for a "start date".
Edited by The_Apprentice (2004-03-09 01:12 PM)
|
Top
|
|
|
|
#75943 - 2004-03-09 02:27 PM
Re: Log Management / Rotation / Archiving toolkit
|
The_Apprentice
Fresh Scripter
Registered: 2003-05-30
Posts: 16
Loc: Guildford, England
|
Thanks for a speedy reply, which was cunning, but not for what we need.
Our plan is to archive all log files, for servers that are not part of a backup regime, to a single repository (server). I was gonna use PREARCH to call a job to map the drive. That server will then be backed up. The Archive directory will then be deleted about 7 days later, allowing for a working backup to have been taken. (this can be done by a simple scheduled deltree).
For servers that are backed up, it is pretty much the same except the will bhe archived to a local drive.
Thing is, the end of month is not always the end of month, if you know what I mean, hence the requirement to specify the date to archive from.
I'm just trying to work out how to add a date caculation into your code for the file move based on a date supplied in the .ini file.
When I come up with some code I will post it, though I am a bit snowed under at the mo.
|
Top
|
|
|
|
#75946 - 2004-03-09 04:23 PM
Re: Log Management / Rotation / Archiving toolkit
|
The_Apprentice
Fresh Scripter
Registered: 2003-05-30
Posts: 16
Loc: Guildford, England
|
Quote:
The end of the month is always the beginnign of the next month minus one day.
In an ideal world - Yes.
Our end of month, especially in relation to "end of month backups" seems to fall anytime within 4-5 days after the previous month has actually ended.
But sods law end of month one month will be the end of the month, in which case we would like to specify the day before the end of month.
If you see what I mean
EDIT:
I guess what I am trying to say, is it would be nice to have a date calculation before the Move.
At the mo ALL files are moved from the source directory regardless of age.
Edited by The_Apprentice (2004-03-09 04:31 PM)
|
Top
|
|
|
|
#75948 - 2004-03-10 02:17 PM
Re: Log Management / Rotation / Archiving toolkit
|
The_Apprentice
Fresh Scripter
Registered: 2003-05-30
Posts: 16
Loc: Guildford, England
|
Ahhhhh, it all makes sense now
It would still be nice to use a defined date, and that then calculates the "age" from the date it actually runs.
Anyways, thank you for all your help and a great utility.
|
Top
|
|
|
|
Moderator: Glenn Barnas, NTDOC, Arend_, Jochen, Radimus, Allen, ShaneEP, Ruud van Velsen, Mart
|
0 registered
and 557 anonymous users online.
|
|
|