cyri
(Getting the hang of it)
2006-01-03 12:19 AM
ftp exist udf?

I have seen some of the FTP posts, but none seem to be quite what I'm looking for. I want to be able to query the McAfee FTP/HTTP DAT sites to see if there is a newer DAT available. Only query, nothing else.

The FTPGet() UDF actually retrieves the file, plus you need to know the name of the file to get. I would only know that the file will be something like "dat-XXXX.zip". Where XXXX is the DAT version. To determine whether it is newer I plan on comparing that to the DAT I currently have to see if it's newer.

I was looking at the OS built in FTP, but it seems a bit cumbersome and I don't know if that FTP can return the exist or not exist to Kix.

This could also be done using McAfee's HTTP site, but I'm not sure how that would be done either. I've toyed with the wget program, but again that will retrieve the file, which I don't want to do.

Would it be easier if I just retrieved the file to a temp location and then do the compare? Or if anyone knows of a way to query without the retrieve I would greatly appreciate it.


ShawnAdministrator
(KiX Supporter)
2006-01-03 12:38 AM
Re: ftp exist udf?

Strictly in terms of getting "better" information, would it be wiser to maybe read McAfee's delta.ini file here:

http://speedownload.nai.com/Products/CommonUpdater/delta.ini

that give this information:

[Contents]
LatestIncremental=621
Created=Mon Jan 2 04:40:00 2006
CurrentVersion=4665

amongst other things. Than maybe use Kixtart's ReadProfileString func to parse out the latest and greatest info.


cyri
(Getting the hang of it)
2006-01-03 12:42 AM
Re: ftp exist udf?

Thanks Shawn, didn't even know that location existed. I'll have to check my HTTP site again, but I think it's pointing somewhere else.

So would you recommend pulling that down with wget or something similar and then making the comparison?


NTDOCAdministrator
(KiX Master)
2006-01-03 12:44 AM
Re: ftp exist udf?

Well FTP does not support an If Exist contruct directly.
You might be able to do some listing to a temporary file and then parse the temporary file though.

Here is something that works well on the Symantec site which I'm sure could be modified to work on McAfee's site if wanted.


;REQUIREMENTS: 
;1: Must be run with an account that has Admin rights on the AV Servers
;2: KiXtart v4.22 or newer.
;Download: http://www.kixtart.org/binary/distrib/KiX2010_451.zip
;3: Only requires the WKIX32.EXE file from within the KiXtart zip file,
;all other files are not required
;4: IE Proxy settings need to allow access to the Symantec site to download files.

;*** NOTE - WARNING ***
;Must only be run for Symantec Antivirus 8.x and 9.x and 10.x Servers.
;Do not include any 7.x Servers in the list.

Break On
Dim $SO
$SO=SetOption('Explicit','On')
$SO=SetOption('NoVarsInStrings','On')
$SO=SetOption('NoMacrosInStrings','On')

Dim $URL,$Link,$RC,$Page,$Server,$Servers,$File,$Download,$Target,$FileSize
$URL='http://www.sarc.com/avcenter/download/pages/US-SAVCE.html'
$Link='http://definitions.symantec.com/defs/xdb/'
$RC=readHTML($URL)
$File=InStr($RC,$Link)
$File=SubStr($RC,$File+41,12)
$Download = $Link + $File
; Location where the file will be downloaded to
$Target='C:\DOWNLOAD\'
; Array list of 2000/2003 Servers
$Servers='AV01','AV02'

'Downloading ' + $File + ' Please wait...' ?
If FTPget($Download,$Target+$File)
; Unless link is completely down, probably will never get this error
'Error occured: ' + @ERROR + ' ' + @SERROR ?
Else
'Download complete... File saved in ' + $Target + ' as ' + $File ?
; Check the size of the file to confirm you didn't download just blank
; or small html file
$FileSize = GetFileSize($Target+$File)
If $FileSize >9000000
'Ready to update AntiVirus Defs ' ?
'File size downloaded was: ' + $FileSize ?
;Copy the XDB file to the 2000/2003 Servers
For Each $Server In $Servers
Copy $Target+$File '\\'+$Server+'\c$\PROGRA~1\SAV'
'Copying to ' + $Server +'... ' + @ERROR + ' ' + @SERROR ?
Next
; This line archives the definition files to AV02 for reference
MOVE $Target+$File '\\AV02\xdb\'
'Moving xdb file to AV02 archive... ' +@ERROR + ' ' + @SERROR ?
'Definition updates completed...' ?
'Please press a key to continue...' ?
Else
'The file size appears to be too small. '+
'Update aborted, please try later...' ?
'File size was: ' + $FileSize ?
Del $Target+$File
'Press a key to continue...' ??
GET $Pause
Quit 1
EndIf
'Press a key to continue...' ??
GET $Pause
EndIf

Function readHTML($page)
Dim $obj
$obj=createobject("microsoft.xmlhttp")
$obj.open("get",$page,not 1)
$obj.send
$readHTML=$obj.responsebody
EndFunction

Function FTPget($sURL, $sTargetFile, optional $sUser, optional $sPass)
Dim $oFTP, $oStream
$sUser=""+$sUser
$oFTP = CreateObject("Microsoft.XMLHTTP")
if @error $ftpget=1 exit 1 endif
$oStream = CreateObject("ADODB.Stream")
if @error $ftpget=2 exit 2 endif
if $sUser
$oFTP.Open("GET", $sURL, not 1, $sUser, $sPass)
else
$oFTP.Open("GET", $sURL, not 1)
endif
if @error $ftpget=3 exit 3 endif
$oFTP.Send
$oStream.Type = 1
$oStream.Mode = 3
$oStream.open
$oStream.Write($oFTP.responseBody)
if @error $ftpget=4 exit 4 endif
$oStream.SaveToFile($sTargetFile, 2)
if @error $ftpget=5 exit 5 endif
$oStream.Close
EndFunction



ShawnAdministrator
(KiX Supporter)
2006-01-03 12:48 AM
Re: ftp exist udf?

ja, ftpget seems to do the trick:

Code:

break on

FTPget("http://speedownload.nai.com/Products/CommonUpdater/delta.ini","c:\delta.ini")

exit 0

Function FTPget($sURL, $sTargetFile, optional $sUser, optional $sPass)
Dim $oFTP, $oStream
$sUser=""+$sUser
$oFTP = CreateObject("Microsoft.XMLHTTP")
if @error $ftpget=1 exit 1 endif
$oStream = CreateObject("ADODB.Stream")
if @error $ftpget=2 exit 2 endif
if $sUser
$oFTP.Open("GET", $sURL, not 1, $sUser, $sPass)
else
$oFTP.Open("GET", $sURL, not 1)
endif
if @error $ftpget=3 exit 3 endif
$oFTP.Send
$oStream.Type = 1
$oStream.Mode = 3
$oStream.open
$oStream.Write($oFTP.responseBody)
if @error $ftpget=4 exit 4 endif
$oStream.SaveToFile($sTargetFile, 2)
if @error $ftpget=5 exit 5 endif
$oStream.Close
EndFunction



cyri
(Getting the hang of it)
2006-01-03 12:56 AM
Re: ftp exist udf?

Thanks for the posts. I think what I'm going to end up doing is using the FTPGet() UDF and picking up the delta.ini file that Shawn recommended. Then I'll do my compare. This process is only going to be used for notification and not the actual download.

Thanks for the replies. And I see Shawn already replied with the FTPGet code that I will need.

PS - I read a post about the FTPGet() showing up as VBS/Psyme (Trojan) and the auto-email from the boards I got at work was picked up by McAfee. Sure enough it was tossed into the quarantine. Heh, oh well.


NTDOCAdministrator
(KiX Master)
2006-01-03 12:56 AM
Re: ftp exist udf?

So where's the rest of the code solution

ShawnAdministrator
(KiX Supporter)
2006-01-03 12:59 AM
Re: ftp exist udf?

cyri sounds like he knows what he's talking about. dont think he needs any hand-holding.

NTDOCAdministrator
(KiX Master)
2006-01-03 01:00 AM
Re: ftp exist udf?

ROFLMAO - Okay, be that way

NTDOCAdministrator
(KiX Master)
2006-01-03 01:01 AM
Re: ftp exist udf?

No size or date check needed for Symantec as they update every day - if wanted there is even one that is updated hourly but I think that's a bit overkill.

NTDOCAdministrator
(KiX Master)
2006-01-03 01:19 AM
Re: ftp exist udf?

Oh... don't forget an EXIT code - Shawn is bagging on me on MSN

Howard Bullock
(KiX Supporter)
2006-01-03 01:26 AM
Re: ftp exist udf?

I am curious how he wants to use this new DAT info and why his approach is to use Kix to check for new DATs.

NTDOCAdministrator
(KiX Master)
2006-01-03 01:28 AM
Re: ftp exist udf?

Not sure. Doesn't McAfee update daily too?

Howard Bullock
(KiX Supporter)
2006-01-03 01:36 AM
Re: ftp exist udf?

Well I see in one of the post above he mentions notification. But why? Does he not have ePO running on a schedule to update the DATs? Avert sends out new DAT notifications...

NTDOCAdministrator
(KiX Master)
2006-01-03 01:40 AM
Re: ftp exist udf?

What is the alert for? To tell you it did it, or that it failed? Not sure I'd like a page / e-mail every day for that.

cyri
(Getting the hang of it)
2006-01-09 09:27 PM
Re: ftp exist udf?

The notification I was talking about was an email that gets sent to my team. The script runs at 7am everyday to let us know that ePO has the latest DAT. It's a quick and easy way for those in my team that aren't as familiar with ePO to know if there is an issue with the DATs or not.

Putting the why question aside. I do have a problem with the script. Here is my FTPGet command.
Code:
FTPget("ftp://ftp.mcafee.com/CommonUpdater/delta.ini","C:\delta.ini")



When I check the C:\delta.ini file, it is not the current file. I browse to that location with IE and grab the file and it is the latest version. Here is the example of the contents of the delta.ini file. I'll give what FTPGet gives me and what browsing manually gives.

FTPGet Version
[Contents]
LatestIncremental=621
Created=Mon Jan 2 04:40:00 2006
CurrentVersion=4665

Manually browsing via IE Version
[Contents]
LatestIncremental=626
Created=Mon Jan 9 04:40:00 2006
CurrentVersion=4670

Am I doing something wrong with the FTPGet command?


NTDOCAdministrator
(KiX Master)
2006-01-10 12:21 AM
Re: ftp exist udf?

Maybe check/set your IE cache setting.

$IEKey='HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings'
$SetIECache=WriteValue($IEKey,'SyncMode5',3,REG_DWORD)


cyri
(Getting the hang of it)
2006-01-10 12:27 AM
Re: ftp exist udf?

This script is running with no logged on user as a scheduled task on a server. It has built in credentials in the task. Would this still apply?

NTDOCAdministrator
(KiX Master)
2006-01-10 12:55 AM
Re: ftp exist udf?

Well then perhaps it's using the Default User profile.

Try modifying that one.

HKEY_USERS\.DEFAULT\Software\Microsoft\Windows\CurrentVersion\Internet Settings


ShawnAdministrator
(KiX Supporter)
2006-01-10 12:58 AM
Re: ftp exist udf?

I think (personally) that you are running into IE cache issues, here is a snippet of knowledge base:

Microsoft Internet Explorer Cache issues

Internet Explorer implements caching for GET requests. Authors who are not familiar with HTTP caching expect GET requests not to be cached, or for the cache to be avoided as with the refresh button. In some situations, failing to circumvent caching is a bug. One solution to this is using POST request method, which is never cached, but is intended for non-idempotent operations. A solution using the GET request method is to include a unique querystring with each call, as shown in the example below.

req.open("GET", "xmlprovider.php?hash=" + Math.random());

or set the Expires header to an old date in your script that generates the XML content. For PHP that would be

header( "Expires: Mon, 26 Jul 1997 05:00:00 GMT" ); // disable IE caching
header( "Last-Modified: " . gmdate( "D, d M Y H:i:s" ) . "GMT" );
header( "Cache-Control: no-cache, must-revalidate" );
header( "Pragma: no-cache" );

Alternatively, force the XMLHTTPRequest object to retrieve the content anyway by including this in the request:

req.open("GET", "xmlprovider.php");
req.setRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT");
req.send(null);

The source link for above is here. Thinking maybe you could try this "GET request" work-around I highlighted in the first paragraph. Studying this more closely myself.


LonkeroAdministrator
(KiX Master Guru)
2006-01-10 01:19 AM
Re: ftp exist udf?

to make a completely unique string per request, one should use time tags.
example:
$req.open("GET", "xmlprovider.php?hash=" + @date + @time)


ShawnAdministrator
(KiX Supporter)
2006-01-10 01:24 AM
Re: ftp exist udf?

ja, cant test it much but it does seem to at least see past the hash-stuff, and download the file ... hopefully the most recent version ;0)

FTPget("ftp://ftp.mcafee.com/CommonUpdater/delta.ini?hash=+@date + @time","e:\delta.ini")


ShawnAdministrator
(KiX Supporter)
2006-01-10 02:05 AM
Re: ftp exist udf?

btw - anyone catch that fancy word in that quote above ... non-idempotent ... don't know wtf that means, sounds like some kind of medical condition I don't want to catch.

cyri
(Getting the hang of it)
2006-01-10 04:02 PM
Re: ftp exist udf?

I tried this code.
Code:
FTPget("ftp://ftp.mcafee.com/CommonUpdater/delta.ini?hash=+@date + @time","e:\delta.ini")



But it's not downloading the file now. Am I missing something?


ShawnAdministrator
(KiX Supporter)
2006-01-10 04:10 PM
Re: ftp exist udf?

nah, it was just a shot in the dark.

JochenAdministrator
(KiX Supporter)
2006-01-10 04:24 PM
Re: ftp exist udf?

Shawn : Wiki entry for idempotent

ShawnAdministrator
(KiX Supporter)
2006-01-10 04:54 PM
Re: ftp exist udf?

Thanks cappy - I'm having a hard time even pro-nouncing that word, without like - spitting all over my monitor.

Anyone want to comment / take a stab at writing a POST request version of FTPGet ?


cyri
(Getting the hang of it)
2006-01-10 05:10 PM
Re: ftp exist udf?

I'm still playing with this hash used in the first method (below).
req.open("GET", "xmlprovider.php?hash=" + Math.random());

The third method (shown below) I tested and it still grabbed the old copy of the file.
req.open("GET", "xmlprovider.php");
req.setRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT");
req.send(null);

As for attempting to write a modified UDF. I think thats a bit out of my skill range. I'm not familar enough with XMLHTTP.


NTDOCAdministrator
(KiX Master)
2006-01-10 07:23 PM
Re: ftp exist udf?

And modifying the .DefaultUser profile didn't work?

ShawnAdministrator
(KiX Supporter)
2006-01-10 07:30 PM
Re: ftp exist udf?

Quote:


It has built in credentials in the task. Would this still apply?





You mean to say the task is running under a user account ? If so, might also try (as a test) a scripted DEL of temporary internet files - just as a test I mean. At least you would know for sure whether caching was the issue.


cyri
(Getting the hang of it)
2006-01-10 07:32 PM
Re: ftp exist udf?

I feel a little sheepish. I got a bit involved in some of the other posts that I didn't even try it...until now. Your reg hack worked NTDOC. Thank you for that.

Though I'm sure there is probably a way to do this within XMLHTTP as well without modifying IE settings.


LonkeroAdministrator
(KiX Master Guru)
2006-01-10 09:37 PM
Re: ftp exist udf?

the post version of ftpGet is no biggie.
just change the send() to have some sh*t in it:
send("hash=sh*t")


ShawnAdministrator
(KiX Supporter)
2006-01-10 10:43 PM
Re: ftp exist udf?

ok so try this POST version of FTPGet then ...

Code:

Function FTPget($sURL, $sTargetFile, optional $sUser, optional $sPass)
Dim $oFTP, $oStream
$sUser=""+$sUser
$oFTP = CreateObject("Microsoft.XMLHTTP")
if @error $ftpget=1 exit 1 endif
$oStream = CreateObject("ADODB.Stream")
if @error $ftpget=2 exit 2 endif
if $sUser
$oFTP.Open("POST", $sURL, not 1, $sUser, $sPass)
else
$oFTP.Open("POST", $sURL, not 1)
endif
if @error $ftpget=3 exit 3 endif
$oFTP.Send
$oStream.Type = 1
$oStream.Mode = 3
$oStream.open
$oStream.Write($oFTP.responseBody)
if @error $ftpget=4 exit 4 endif
$oStream.SaveToFile($sTargetFile, 2)
if @error $ftpget=5 exit 5 endif
$oStream.Close
EndFunction



-Jaffa


LonkeroAdministrator
(KiX Master Guru)
2006-01-10 11:44 PM
Re: ftp exist udf?

no...
you didn't even touch the send method.

hmm...
on the other hand, we don't have anything really to send.
did you didn't try this out yourself?


ShawnAdministrator
(KiX Supporter)
2006-01-10 11:47 PM
Re: ftp exist udf?

Make it go.

LonkeroAdministrator
(KiX Master Guru)
2006-01-10 11:53 PM
Re: ftp exist udf?

k, checked out the syntax and this should work for you:
Code:

Function FTPgetPOST($sServer, $sFile, $sTargetFile, optional $sUser, optional $sPass)
Dim $oFTP, $oStream
$sUser=""+$sUser
$oFTP = CreateObject("Microsoft.XMLHTTP")
if @error $ftpget=1 exit 1 endif
$oStream = CreateObject("ADODB.Stream")
if @error $ftpget=2 exit 2 endif
if $sUser
$oFTP.Open("POST", $sServer, not 1, $sUser, $sPass)
else
$oFTP.Open("POST", $sServer, not 1)
endif
if @error $ftpget=3 exit 3 endif
$oFTP.setRequestHeader = "Content-Type", "application/x-www-form-urlencoded"
$oFTP.Send($sFile)
$oStream.Type = 1
$oStream.Mode = 3
$oStream.open
$oStream.Write($oFTP.responseBody)
if @error $ftpget=4 exit 4 endif
$oStream.SaveToFile($sTargetFile, 2)
if @error $ftpget=5 exit 5 endif
$oStream.Close
EndFunction



ShawnAdministrator
(KiX Supporter)
2006-01-10 11:56 PM
Re: ftp exist udf?

Cool. By the way, I did try out the other one and it does download the file, but I cant really tell about the cache stuff. Plus your code looks like you put more thought into it ...

LonkeroAdministrator
(KiX Master Guru)
2006-01-11 12:00 AM
Re: ftp exist udf?

but I didn't
just ripped some source from bbChecker


NTDOCAdministrator
(KiX Master)
2006-01-11 12:45 AM
Re: ftp exist udf?

Well glad you got it going Cyri, but looks like all of us got a little something more and learned a bit more from this thread so that's cool too.

Thanks Shawn and Jooel for the UDF code.