Sealeopard
(KiX Master)
2003-03-02 12:26 AM
Problem with semisyncronous WMI query

I've been testing the UDF below with KiXtart and I've found that it apparently does not support semisynchronous queries with WMI. Semisynchronous queries are faster and require less memory overhead on the queried computer. These queries are generated through a ExecQuery($query,'WQL',48) call.

When uncommenting the line that activates semisynchronous queries, the UDF still works without raising an error message. However, all array fields are empty even though the correct number of rows/columns are returned.

code:
; some test code first
$events = ReadEventlog('Security',528,@WKSTA,'2003/03/01 00:00:00')
? ubound($events,1)
? ubound($events,2)
for $a=0 to ubound($events,1)
for $b=0 to ubound($events,2)
? 'Events['+$a+','+$b+'] = '+$events[$a,$b]
next
next
$rc='SELECT TimeGenerated, User FROM Win32_NTLogEvent WHERE Logfile="System" AND TimeGenerated>="20030225000000.000000-300"'
$events = ReadEventlog($rc)
? ubound($events,1)
? ubound($events,2)
for $a=0 to ubound($events,1)
for $b=0 to ubound($events,2)
? 'Events['+$a+','+$b+'] = '+$events[$a,$b]
next
next

; here's the UDF

;FUNCTION ReadEventlog()
;
;ACTION Retrieves events from the eventlog
;
;AUTHOR Jens Meyer
;
;VERSION 1.4
;
;KIXTART VER 4.20
;
;SYNTAX RETCODE = READEVENTLOG(EVENTLOG, EVENTID, OPTIONAL COMPUTER, OPTIONAL DATETIME,
; OPTIONAL USERNAME, OPTIONAL PASSWORD)
;
;PARAMETERS EVENTLOG
; Name of the eventlog, e.g. 'Security', 'System','Application'
; Alternatively, a custom WQL query can be provided. Date fields in
; a WQL query MUST be properly formatted as YYYY/MM/DD HH:MM:SS:000
;
; EVENTID
; Optional Event ID number to be retrieved
;
; COMPUTER
; optional name of a remote computer which eventlog is to be queried. If no
; username/password is provided then the current users credentials will be
; used to connect to the remote event log.
;
; DATETIME
; optional date/time string denoting the start date of the events in
; the form of YYYY/MM/DD HH:MM:SS, YYY/MM/DD, or HH:MM:SS
;
; USERNAME
; optional username which will be used to connect to a remote computer
;
; PASSWORD
; optional password which will be used to connect to the remote computer
;
;RETURN array of events or empty string
;
;REMARKS returns a 2-dimensional array with the following columns. If custom WQL is
; used, then the SELECT part of the custom WQL determines the field assignments.
;
; Column 0 = Category
; Column 1 = CategoryString
; Column 2 = ComputerName
; Column 3 = Data
; Column 4 = EventCode
; Column 5 = EventIdentifier
; Column 6 = EventType
; Column 7 = InsertionStrings
; Column 8 = Logfile
; Column 9 = Message
; Column 10 = RecordNumber
; Column 11 = Source Name
; Column 12 = TimeGenerated
; Column 13 = TimeWritten
; Column 14 = Type
; Column 15 = User
;
;DEPENDENCIES WMI
;
;EXAMPLE $events = ReadEventlog('Security',528)
; $events = ReadEventlog('Security',528,,'COMPUTER')
; $events = ReadEventlog('Security',528,'2002/09/01 00:00:00','COMPUTER','Administrator','password')
; $events = ReadEventlog('SELECT TimeGenerated, User FROM Win32_NTLogEvent
; WHERE Logfile="Security" AND EventCode=528 AND
; TimeGenerated>="2002/09/01 00:00:00:000"'
;
;KIXTART BBS http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic&f=12&t=000270
;
function ReadEventlog($eventlog, optional $eventid, optional $computer, optional $datetime, optional $username, optional $password)
dim $objLocator, $objWBEM, $objWMIResults, $namespace
dim $event, $item, $wqlQuery, $eventarray
dim $customwql, $customfields, $field
dim $rownumber, $arrayrows, $arraycolumns, $columnnumber
dim $byte, $datastring, $date, $time, $querydate, $querytime, $timezone
dim $objWMIService, $colItems, $objItem

$namespace = 'root\CIMV2'

if trim($eventlog)=''
exit 87
endif

; check to see whether we're connecting to a local or remote eventlog
$computer=trim($computer)
select
case $computer=@WKSTA
$computer='.'
case $computer
case 1
$computer='.'
endselect

if $username and $computer<>'.'
; create locator object for connection to a remote computer
$objLocator = CreateObject('WbemScripting.SWbemLocator')
if @ERROR
exit @ERROR
endif
; create a (credentialed, if username/password provided) connection to a remote computer
$objWBEM=$objLocator.ConnectServer($computer,$namespace,$username,$password)
if @ERROR
exit @ERROR
endif
; set the impersonation level
$objWBEM.Security_.ImpersonationLevel = 3
if @ERROR
exit @ERROR
endif
else
;set the impersonation level and make sure we have security permissions
if $eventlog='Security'
$objWBEM=GetObject('winmgmts:{impersonationLevel=impersonate, (Security)}!\\'+$computer+'\'+$namespace)
else
$objWBEM=GetObject('winmgmts:{impersonationLevel=impersonate}!\\'+$computer+'\'+$namespace)
endif
if @ERROR
exit @ERROR
endif
endif

; check to see whether we're looking for an event ID or if there's a custom query
if left($eventlog,6)='select'
$wqlquery=$eventlog
$arraycolumns=trim(substr($wqlquery,instr($wqlquery,' ')+1,instr($wqlquery,'FROM')-instr($wqlquery,' ')-2))
if instr($arraycolumns,'*')
$arraycolumns=16
$customwql=0
else
$customfields=split(trim($arraycolumns),',')
for $arraycolumns=0 to ubound($customfields)
$customfields[$arraycolumns]=trim($customfields[$arraycolumns])
next
$arraycolumns=ubound($customfields)+1
$customwql=1
endif
else
$customwql=0
$arraycolumns=16
$eventid=val($eventid)
$wqlQuery="SELECT * FROM Win32_NTLogEvent WHERE Logfile='"+$eventlog+"' AND EventCode="+val($eventID)

if $datetime
$colItems = $objWBEM.ExecQuery('Select CurrentTimeZone from Win32_ComputerSystem')
if @ERROR
exit @ERROR
endif

for each $objItem in $colItems
$timezone = $objItem.CurrentTimeZone
next

$objWMIService = 0
$colItems = 0
$objItem = 0

$datetime=trim($datetime)

select
case instr($datetime,' ')
$date=left($datetime,instr($datetime,' ')-1)
$time=substr($datetime,instr($datetime,' ')+1)
case instr($datetime,'/')
$date=$datetime
$time='00:00:00'
case instr($datetime,':')
$date=@DATE
$time=$datetime
case 1
$date=@DATE
$time=@TIME
endselect
if $date and $time
$datetime=join(split($date,'/'),'')+join(split($time,':'),'')+'.000000'+$timezone
else
$datetime=''
endif

$wqlQuery=$wqlQuery+' AND TimeGenerated>="'+$datetime+'"'
endif
endif

;------------------------------------------
;| beginning of problematic code fragment |
;------------------------------------------

; the following line would not work
; $objWMIResults = $objWBEM.ExecQuery($wqlQuery,'WQL',48)
; this line would work
; $objWMIResults = $objWBEM.ExecQuery($wqlQuery,'WQL',0)
; this line would work
$objWMIResults = $objWBEM.ExecQuery($wqlQuery)

;------------------------------------------
;| end of problematic code fragment |
;------------------------------------------


if @ERROR
exit @ERROR
endif

$rownumber = 0
$columnnumber = 0
; this is for semisyncronous queries
$arrayrows=0
for each $event in $objWMIResults
$arrayrows=$arrayrows+1
next
; otherwise, this could be used for non-semisynchronous queries
;$arrayrows = val($objWMIResults.Count)

if $arrayrows=0
$ReadEventlog=''
return
endif

redim $eventarray[$arrayrows-1,$arraycolumns-1]

for each $event in $objWMIResults
$columnnumber = 0
for each $item in $event.Properties_
if $customwql=0 or ascan($customfields,$item.name)+1
select
case $item.Name='Data'
$datastring=''
for each $byte in $event.Data
if $byte=0
$byte=46
endif
$datastring=$datastring+chr($byte)
next
$eventarray[$rownumber,$columnnumber]=$datastring
case $item.name='InsertionStrings'
$eventarray[$rownumber,$columnnumber]=join($item.Value,@CRLF)
case $item.name='TimeGenerated' or $item.Name='TimeWritten'
$time=left($item.Value,4)+'/'+substr($item.Value,5,2)+'/'+substr($item.Value,7,2)+' '
$time=$time+substr($item.Value,9,2)+':'+substr($item.Value,11,2)+':'+substr($item.Value,13,2)
$eventarray[$rownumber,$columnnumber]=$time
case 1
$eventarray[$rownumber,$columnnumber]=$item.Value
endselect
$columnnumber=$columnnumber+1
endif
next
$rownumber=$rownumber+1
next

$objWMIResults = 0
$objWBEM = 0
$objLocator = 0
$ReadEventlog = $eventarray
exit 0

endfunction



[ 02. March 2003, 01:35: Message edited by: sealeopard ]


LonkeroAdministrator
(KiX Master Guru)
2003-03-02 12:29 AM
Re: Problem with semisyncronous WMI query

could you mark out little more the lines that are failing, thanks.

Les
(KiX Master)
2003-03-02 12:31 AM
Re: Problem with semisyncronous WMI query

Jens,
Could you please clarify if this is a 4.21 issue or any 4.xx?


Sealeopard
(KiX Master)
2003-03-02 12:36 AM
Re: Problem with semisyncronous WMI query

I've tested it with KiXtart 4.20, 4.20-RC1, and 4.21-RC1. They all have the same problem. I think it might actually be the case that the KiXtart COM-implementation does nto support these semi-synchronus calls. However, that would be something that only Ruud could answer.

Also, if it is ccurrently not supported, then it should be implemented as soon as possible as I can achieve speed improvements of a factor of 3-4 when querying remote eventlogs and I can also return larger eventarrays without maxing out the memory on the remote computers.

Addendum: I added my test code to the first post.

[ 02. March 2003, 00:38: Message edited by: sealeopard ]


LonkeroAdministrator
(KiX Master Guru)
2003-03-02 12:43 AM
Re: Problem with semisyncronous WMI query

k, jens...
this looks like it must have something to do with the lacks of kixtart-com.

like, I'm unable to show properties of some shell-object properties even though there is no error.

so, I don't see this as "bug" but more like lack of support... feature.

indeed, we like to see them supported and I'm sure ruud is working on these.


Sealeopard
(KiX Master)
2003-03-02 12:51 AM
Re: Problem with semisyncronous WMI query

Yes, I'm curious to hear whether it's a bug or currently not supported.

Ruud van Velsen
(Hey THIS is FUN)
2003-03-03 08:03 AM
Re: Problem with semisyncronous WMI query

Well, I just tested with the semisynchronous sample in MSDN, and it appears to work fine:

$oSvc = GetObject("winmgmts:root\cimv2")
$oInstSet = $oSvc.ExecQuery("SELECT Name FROM Win32_Process",,48)

For Each $oInst in $oInstSet
? "Process: " + $oInst.name
Next

Let me know if this code works for you as well, and/or if you see any difference with your own script.

Kind regards,

Ruud


NTDOCAdministrator
(KiX Master)
2003-03-03 08:23 AM
Re: Problem with semisyncronous WMI query

Code worked for me with v4.20

Process: System Idle Process
Process: System
Process: SMSS.EXE
Process: CSRSS.EXE
Process: WINLOGON.EXE
Process: SERVICES.EXE
Process: LSASS.EXE
Process: svchost.exe
Process: spoolsv.exe
Process: ati2evxx.exe
Process: CTsvcCDA.exe
Process: cvpnd.exe
Process: defwatch.exe
Process: DKService.exe
Process: svchost.exe
Process: gearsec.exe
Process: gearsec.exe
Process: mdm.exe
Process: rtvscan.exe
Process: MSGSYS.EXE
Process: explorer.exe
Process: PGPsdkServ.exe
Process: regsvc.exe
Process: mstask.exe

etc....


Sealeopard
(KiX Master)
2003-03-03 04:19 PM
Re: Problem with semisyncronous WMI query

Very strange, Ruud's code works for me, too. However, it doesn't work inside the ReadEventlog() UDF. [Confused]

Chris S.
(MM club member)
2003-03-03 04:42 PM
Re: Problem with semisyncronous WMI query

What if you don't specify the strQueryLanguage, since it's an optional parameter and, if set, must be 'WQL', perhaps leaving it out will make it work as expected.

Sealeopard
(KiX Master)
2003-03-03 05:34 PM
Re: Problem with semisyncronous WMI query

I thnink I've found the problem. Run this script and wonder [Confused]
code:
$oSvc = GetObject("winmgmts:root\cimv2")
$oInstSet = $oSvc.ExecQuery("SELECT Name FROM Win32_Process",,48)

? 'Running it for the first time'
For Each $oInst in $oInstSet
? "Process 1: " + $oInst.name
Next

? 'Running it for the second time'

For Each $oInst in $oInstSet
? "Process 2: " + $oInst.name
Next

It looks to me as if there's soem kind of recordset pointer involved which doesn't get set back, thus if you run it a second time it's at the end of the recordset.

Now I just have to find a way to set it back to the start without creating a copy of the object.


Ruud van Velsen
(Hey THIS is FUN)
2003-03-03 05:40 PM
Re: Problem with semisyncronous WMI query

This is expected behaviour, as per the documentation of the 'wbemFlagForwardOnly' flag (value 32):

Causes a forward-only enumerator to be returned. Use this flag in combination with wbemFlagReturnImmediately to request semisynchronous access.

You can only iterate (as in a For Each statement) through a forward-only enumerator one time. The memory containing the instances is released by WMI so that the enumerator cannot be rewound. Therefore, the Count method cannot be used since it requires rewinding the enumerator.

Forward-only enumerators are generally much faster and use less memory than conventional enumerators, but they do not allow calls to Clone or Reset.

--Ruud


Sealeopard
(KiX Master)
2003-03-03 05:44 PM
Re: Problem with semisyncronous WMI query

Thanks, Ruud, found it, too, under http://www.microsoft.com/technet/treeview/default.asp?url=/technet/scriptcenter/scrguide/sas_wmi_krnh.asp

I guess it's back to the drawing board [Frown]


NTDOCAdministrator
(KiX Master)
2003-03-03 08:43 PM
Re: Problem with semisyncronous WMI query

Jens,

I've run into this with all kinds of WMI scripts. So far I've had to kill the object and query again. Have not found any other way around it. Seems it is designed to be that way.


Sealeopard
(KiX Master)
2003-03-03 11:15 PM
Re: Problem with semisyncronous WMI query

I think I've now solved all the issues with the ReadEventlog() UDF.

I circumvented the inability to retrieve the number of events in a direct way by first putting the event objects into a 1-D array which can be REDIMmed without problems before extracting the logs into a final 2-D array. The alternative of REDIMming a 2-D array and then transposing it in order to get the familiar row/column structure was too costly timewise.

The UDF in the UDF Forum has been updated accordingly.

[ 04. March 2003, 04:50: Message edited by: sealeopard ]