burnsc
(Starting to like KiXtart)
2004-04-15 12:48 AM
execution time

I have a fairly short script that takes a long time(approximately 2.5 minutes) to execute. I will be useing this as a logon script so I am obviously concerned about the execution time.
In the script I have 1 IF InGroup and approximately 5 If Exists. This script will grow to a MUCH LARGER size before I am finished.

I am wondering if this is an average execution time for this size script. Is it possible that most of my time is taken up with the group chacheing?

I did notice that if I 'short circuited' the If InGroup statement it took 1.5 minutes off my execution time. If InGroup is one of the culprits would I get better execution time using my own array for getting the groups?

Also, Any tips on increaseing speed will help drasticly. Even if it is rearrangeing the script in certain ways to help the exe 'compile' it quicker.

PS. This script is very likely to have tons of InGroups in it.

Sealeopard
(KiX Master)
2004-04-15 12:55 AM
Re: execution time

We can't really help you if you don't even show your script. However, 2 1/2 minutes is way to long.

Les
(KiX Master)
2004-04-15 12:56 AM
Re: execution time

First off, KiX does a pre-fetch so the delay is not commensurate with the number of InGroup() functions. That said, you have a problem with your network setup. It should not take that long!

ShaneEP
(MM club member)
2004-04-15 12:58 AM
Re: execution time

This is not what I would consider a normal time. We have a script that does about 200 If InGroups and it only takes seconds. What is the client u are testing with? Are you checking domain group or local group? It may speed it up if you put the domain name before the group name, especially if you have more than one domain.

If InGroup ("Domain\Group")
Do Stuff
Endif

If you can try to provide some more info about your environment and we will try to help. You may also want to post the contents of the script.

burnsc
(Starting to like KiXtart)
2004-04-15 01:03 AM
Re: execution time

The full script is as follows. I tended to think that this was very slow for normal operations. Thanks for all the help. Some of this is commented out because it will be comeing soon in our configuration, but we are not quite read for them yet.

;*************************************************************
; NAME: Clinton Login Script
;
; DATE : 4/14/04
;
; COMMENT:
;
;*************************************************************

;*************************************************************
; root variable intailization
;*************************************************************
$rootserver = "\\clnhfp01\"
$KixHome = $rootserver+"\ScriptTools\WKix32.exe"
$BatHome = $rootserver+"\ClintonLogon\bat\"
$ScriptHome = $rootserver+"\ClintonLogon\script\"

;*************************************************************
;Auto Install based Logic
; This is used for to install Applications automatically upon
; reboot
;*************************************************************

;*************************************************************
;Process M & N Drive Mappings
;*************************************************************
If (InGroup("TKNTOR\CLN__Clinton Users"))
$NTData = $RootServer+"NTData"
$NTSoftware = $RootServer+"NTSoftware"
Use M: $NTData
Use N: $NTSoftware
? "Drives should be connected"
endif

;*************************************************************
;Process Old Tor-Cl03 Drive Mappings
;*************************************************************
;If (InGroup("TKNTOR\CLN_Clinton Users"))
; Use q $RootServer+"Vol1\"
; Use u $RootServer+"NWData\"
; Use V $RootServer+"Vol4\"
; Use W $RootServer+"Vol3\"
; Use X $RootServer+"Vol2\"
; Use Y $RootServer+"Vol1\"
; Use Z $RootServer+"Sys\"
;endif

;*************************************************************
;Process User Home Drive Mappings
;*************************************************************
$UserHome = $rootServer+@UserID+"$"
if Exist($UserHome)
Use H $UserHome
endif

;*************************************************************
;Run login Batch if it exists
;*************************************************************
$AllUsersBat = $BatHome+"AllUsers.BAT"
If exist($AllUsersBat)
call $AllUsersBat
endif

$userbat = $BatHome+$uid+".BAT"
If exist($userbat)
call $userbat
endif

;*************************************************************
;Run login script if it exists
;*************************************************************
$AllUsersScr = $ScriptHome+"AllUsers.SCR"
If exist($AllUsersScr)
$KixHome $AllUsersScr
endif

$userScr = $ScriptHome+$uid+".SCR"
If exist($userScr)
$KixHome $userScr
EndIf



ShaneEP
(MM club member)
2004-04-15 01:16 AM
Re: execution time

Have you tried commenting out the portions that call other scripts? My first guess would be that this portion is really what is taking the longest. Try commenting everything below this section...

Code:
;*************************************************************
;Process M & N Drive Mappings
;*************************************************************
If InGroup("TKNTOR\CLN__Clinton Users")
$NTData = $RootServer+"NTData"
$NTSoftware = $RootServer+"NTSoftware"
Use M: $NTData
Use N: $NTSoftware
? "Drives should be connected" 
endif


Also you dont need the double parenthesis in the ingroup lines. Im not sure if that would cause any slow downs but just incase.

burnsc
(Starting to like KiXtart)
2004-04-15 01:21 AM
Re: execution time

I had to add the extra set of parenthesis earlier today to get it to work. I know that sounds strange, but that is what it required.

Also it took 1.5 mins to get to that serction of the script. That is why the ? line was there (to help me gauge time). The problem may be the prefetch.

ShawnAdministrator
(KiX Supporter)
2004-04-15 01:24 AM
Re: execution time

Think I know what it is, your resultant path names are incorrect, which is probably hangin-up the UNC's, for example in your script:

$rootserver = "\\clnhfp01\"
$KixHome = $rootserver+"\ScriptTools\WKix32.exe"

results in the string:

\\clnhfp01\\ScriptTools\WKix32.exe

two sets of double-back-whacks, which i think might be bad.

Les
(KiX Master)
2004-04-15 01:28 AM
Re: execution time

What OS is the test client?
What OSs are the other clients?
What OS are the DCs?
What version @KiX?
What protocols are on the network?
What network clients on the clients?
Do you have KXRPC service running?

ShaneEP
(MM club member)
2004-04-15 01:42 AM
Re: execution time

There is no reason why you should have double parenthesis to make ingroup work. I think you have some kind of network problems or something.

I would also agree with Shawn that the double \'s are bad...But if what you say about it taking 1.5 minutes to get to the "? "Drives should be connected"" line is true then this is occuring before the double \'s come into play.

Les
(KiX Master)
2004-04-15 01:55 AM
Re: execution time

Quote:

I think you have some kind of network problems.


Gee... where did I hear that before?

Also, verify your DNS, WINS, and node type. You might want to run a network sniff to see what is going wrong.

burnsc
(Starting to like KiXtart)
2004-04-15 05:00 AM
Re: execution time

Thanks for catching the double \s. This is a error I missed. Logically to me though at that point of the script they are just variables. They should not effect my speed in the early part of the script (unless the parser is prepareing it in a special part of memory or something like that.).

I agree that there should not have to be the double set of ( for the InGroup, but for some unknown reason it did not work until I placed the double set there. (This was suggested by one of our corporate contacts useing kixstart as well).

burnsc
(Starting to like KiXtart)
2004-04-15 05:05 AM
Re: execution time

What OS is the test client?
Most are Windows NT 4.0 SP6 machines
What OSs are the other clients?
same
What OS are the DCs?
DCs are windows NT 4.0 (will be moveing to win2k later this year when we go AD.
What version @KiX?
latest downloadable version
What protocols are on the network?
Currently TCP/IP, Netbios, IPX (IPX will be going later this year)
What network clients on the clients?
MS Windows, Netware (Netware will be going later this year)
Do you have KXRPC service running?
No, I dont know about the finer point of what this would add to the mix yet.

Thanks

Sealeopard
(KiX Master)
2004-04-15 05:21 AM
Re: execution time

You might want to read the FAQ Forum to learn about KXRPC.

Also, segemnt your code and test each segment, use DEBUG ON in appropriate locations to see why it is so slow. Basically, there's something seriosuly wrong with either your network or your script.

NTDOCAdministrator
(KiX Master)
2004-04-15 06:24 AM
Re: execution time

FYI

Quote:

What version @KiX?
latest downloadable version



Does not specifically tell us what version you're running because were not sure where you downloaded from.

I will assume KiXtart 4.22

Just try running a whole new script on a test account.

In user manager place something like this on the account for testing.

KIX32.EXE TEST.KIX /F

Then in the script just place this code...

Code:
Break On

Dim $SO,$x

$SO=SetOption('Explicit','On')

$SO=SetOption('NoVarsInStrings','On')





DIM $RootServer,$NTSoftware

'Welcome to test script - going to sleep for 3 seconds ' ?

sleep 3

'Welcome to test script - going to sleep for 5 seconds ' ?

sleep 5

'Welcome to test script - going to sleep for 10 seconds ' ?

sleep 10

'Okay sleep timing should have been okay.  Now we will try some other tasks.' ?

$RootServer = "\\clnhfp01\"

$NTSoftware = $RootServer + "NTSoftware"

'My NT Software location is: ' + $NTSoftware

'Will now attempt to map the N: drive to NTSoftware ' ?

Use N: $NTSoftware

'Mapping N drive error was: ' + @ERROR + ' - ' + @SERROR

get $x


This code should complete very quickly, if not then you will have to start digging into the way your Network is setup.

The get $x at the end of the code will cause the script to hold and wait until you press a key so that you can see all of this on screen.

burnsc
(Starting to like KiXtart)
2004-04-15 01:55 PM
Re: execution time

Yup it is 4.22 . sorry about not knowing the version. I was home and checking the board by that time so my work resources were not available at that time. I was also time constrained due to 'kid' issues
I will try that as a test here shortly.

burnsc
(Starting to like KiXtart)
2004-04-15 02:03 PM
Re: execution time

This script does not contain an InGroup. will kixstart still do the group pre-fetch? Or is the script system smart enough to know it wont need it so it wont pre-fetch it.
A little more information about our network structure:
TONS UPON TONS of groups(I would say 1k+, without counting)
BDC local to my plant
ISDN to PDC
Also, I am not testing this by actually logging in. I am executeing the command line that will be executed out of a .bat file.

Thanks

burnsc
(Starting to like KiXtart)
2004-04-15 02:08 PM
Re: execution time

ok... forgot I had a tool that could count the groups. I have 1400 global groups and 10 local groups on the domain controller.

ShawnAdministrator
(KiX Supporter)
2004-04-15 02:15 PM
Re: execution time

If I were you, would test things like this:

Go into registry and delete this key:

HKEY_CURRENT_USER\Software\KiXtart\TokenCache

This is where Kixtart caches it's groups, it will cache all your group memberships only when/if you use the INGROUP() function, then I would run this script:

Code:

break on



$start = @TICKS



if ingroup("domain users")



 ?"In domain users"



endif



?"elapsed=" @TICKS-$start



exit 1


THen show us the output strings from this script ... then RUN THIS SCRIPT AGAIN and you should see a much improved reponse time.

btw, we have thousands of groups and over a hundred InGroups in our login script, the script takes just a few seconds to run.


-Shawn

burnsc
(Starting to like KiXtart)
2004-04-15 02:38 PM
Re: execution time

The output I recieved was as follows :

In Domain Users
elapsed=91872

Actual script time to run was 1 min 32 seconds

BTW, thanks for the location of the cache. With the way this program caches the grops it will come in handy for another program I will be writing later.

ShawnAdministrator
(KiX Supporter)
2004-04-15 02:46 PM
Re: execution time

My goodness, my times look like this:

first run:

In domain users
elapsed=3785

second run:

In domain users
elapsed=271

the user i used is a member of 77 groups ... having said that, still not sure what your next step is ...

Are there any events log entries on your local machine, or on any of your servers pertaining to any of this ?

ShawnAdministrator
(KiX Supporter)
2004-04-15 02:50 PM
Re: execution time

btw - the userid your testing with, how many groups does it belong to ? how many entries in the tokencache ?

Les
(KiX Master)
2004-04-15 02:56 PM
Re: execution time

Quote:

What protocols are on the network?
Currently TCP/IP, Netbios, IPX (IPX will be going later this year)
What network clients on the clients?
MS Windows, Netware (Netware will be going later this year)
Do you have KXRPC service running?
No, I dont know about the finer point of what this would add to the mix yet.



Here is your problem!
You have to optimize your network bindings and provider order!

BTW, you don't need KXRPC unless you are supporting Win9x/ME.

burnsc
(Starting to like KiXtart)
2004-04-15 03:00 PM
Re: execution time

YEESH, yes. Thanks for pointing me to the event log. I have about 5 errors occuring.
They are as follows
The description for Event ID ( 53 ) in Source ( KIXTART ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. The following information is part of the event: UserGetInfo failed Error : The network path was not found. (0x35/53).

The description for Event ID ( 53 ) in Source ( KIXTART ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. The following information is part of the event: UserModalsGet failed Error : The network path was not found. (0x35/53).

The description for Event ID ( 1332 ) in Source ( KIXTART ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. The following information is part of the event: GetPG: LookupAccountSid failed Error : No mapping between account names and security IDs was done. (0x534/1332).

The description for Event ID ( 1332 ) in Source ( KIXTART ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. The following information is part of the event: GetPrimaryGroup failed Error : No mapping between account names and security IDs was done. (0x534/1332).

The description for Event ID ( 1722 ) in Source ( KIXTART ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. The following information is part of the event: UserGetLocalGroups failed Error : The RPC server is unavailable. (0x6ba/1722).

I am still processing what all this means at the moment though.

burnsc
(Starting to like KiXtart)
2004-04-15 03:09 PM
Re: execution time

the user is a member of 16 groups

Les
(KiX Master)
2004-04-15 03:53 PM
Re: execution time

So... did you try optimizing the network settings?
I ran a mixed shop of Windows and NetWare for years and speak from experience. Just got rid of NetWare this year.

burnsc
(Starting to like KiXtart)
2004-04-15 04:13 PM
Re: execution time

We are going down that road now as well. No I really did not optimize the network settings, but this had been done previously because of what appeared to be network latency issues. However, I did test this script on a win2k station I have. It ran in about 5 seconds there.

I wish I knew what I would have to do to fix this machine though. I changed domains and tried to preserve the registry. This may be where my problem is coming in. Since everything appears to be a SID based problem on this. I have not been deep enough in the registry to bring only 'username/sid' settings from a preserved registry for my new user into this registry.
Other than that I am going to have to consider my registry 'hosed'. At that point my next step is to reinstantiate the HKLM Hive that was created for my new user and take months to try to recreate reg keys.

This topic is becoming beyond the scope of this forum. I don't want to be too OT. But if anyone has any ideas, I would appreciate finishing this by email.

Les
(KiX Master)
2004-04-15 04:16 PM
Re: execution time

You are really barking up the wrong tree. It has nothing to do with SIDs and the registry and everything to do with the network settings. But hey! if you want to keep banging your head on that tree...

ShawnAdministrator
(KiX Supporter)
2004-04-15 04:23 PM
Re: execution time

Quote:


I changed domains and tried to preserve the registry.




Not too sure I understand what your trying to accomplish by "preserving the registry", sounds very unusual and "non-supported" whatever it is.

burnsc
(Starting to like KiXtart)
2004-04-15 04:24 PM
Re: execution time

If I am wrong about the network settings thing I am more than willing to admit it. I don't profess to be the smartest in that arena.

Does the event log entries still make you think that there is a problem with the network settings?

burnsc
(Starting to like KiXtart)
2004-04-15 04:42 PM
Re: execution time

Basicly the reg is a file. I copied this file to the new user.

ShawnAdministrator
(KiX Supporter)
2004-04-15 04:45 PM
Re: execution time

I found this link on the net:

Running kixtart with NT Domain logins

Its Ruud discussing issues around the UserModalsGet error (different error code though) and he states that

Quote:


Because of the way KiXtart retrieves information, if the UserModals information can't be retrieved, the user groups are also not retrieved.




Not sure if anything in that link sheds light on any of this.

Les
(KiX Master)
2004-04-15 04:56 PM
Re: execution time

Yes, the event logs indicate a problem with providers and protocols. Make sure that you do not bind the MS client to anything other than TCPIP and move MS to the top in provider order.

The network APIs that KiX calls are processing requests according to the bindings and provider order. If you were to run a network trace (sniffer), you would see all the failures that populate the event log.

burnsc
(Starting to like KiXtart)
2004-04-15 06:34 PM
Re: execution time

When you are talking about the MS client are you talking about the ms client for Netware? We are running the actual Netware client (v4.60.604 SP1).

Les
(KiX Master)
2004-04-15 07:00 PM
Re: execution time

No, M$ client for M$ networking.

In XP, on the bindings tab, it is:
File and Printer Sharing for Microsoft Networks
- Internet Protocol (TCP/IP)

Client for Microsoft Networks
- Internet Protocol (TCP/IP)


On the Provider Order tab, it is:
Network Providers
- Microsoft Windows Network
- Novell bla bla (going form memory)
- Other providers

burnsc
(Starting to like KiXtart)
2004-04-15 08:01 PM
Re: execution time

I dont have the MSClient installed All the bindable services I am showing (on the bindings tab)(all services view) is as follows:
Service->Bound To
3com Smartagent->(NIC)
NetBIOS->NWLink Netbios->IPX/SPX
NetBIOS->Wins->(NIC)
Novell Client->IPX/SPX
Novell Client->TCP/IP->(NIC)
Server->IPX/SPX
Server->NWLink Netbios->IPX/SPX
Server->WINS->(NIC)
Workstation->NWLink Netbios->IPX/SPX
Workstation->WINS->NIC

Sealeopard
(KiX Master)
2004-04-15 10:35 PM
Re: execution time

You'll need the MS Client.

Les
(KiX Master)
2004-04-15 10:45 PM
Re: execution time

Like I said, my view was on XP. YMMV

Your view looks like NT4. I no longer support NT4. NT4 is nearing EOL.

burnsc
(Starting to like KiXtart)
2004-04-16 12:30 AM
Re: execution time

What would be the verbage used for the MS client on NT. Would it be 'Simple TCP/IP services'?

Les
(KiX Master)
2004-04-16 12:58 AM
Re: execution time

There was a time I could recite every option on Win95 and NT4 but interestingly, when I stop supporting them, it just gets purged from my memory.
Now as I am running around the continent migrating divisions to AD, I wish I would have retained some of it.

Since I no longer have NetWare and no NT4 wkstas, I cannot refresh my memory.

Sorry

Why not just bounce around in there on a test machine and see what you can tweak.

burnsc
(Starting to like KiXtart)
2004-04-16 02:42 PM
Re: execution time

One other Question(for my own clarification).
When exactly does this prefetch occur?
Before Script Runs
At the point in the script where InGroup function is, but once per script(unless cache is older than specified)?

Thanks

Kdyer
(KiX Supporter)
2004-04-16 02:48 PM
Re: execution time

Can you clarify "Prefetch?" Are you talking about Token Caching with KiXtart? Or, are you talking about Prefetch from XP, for example?

If I go to Start/Run and type in Prefetch.. I see the programs that have been run. Prefetch tries to capture how programs are executed so that they attempt to run more efficiently. Have you dug around on http://msdn.microsoft.com for Prefetch and it's technology?

Thanks,

Kent

burnsc
(Starting to like KiXtart)
2004-04-16 02:51 PM
Re: execution time

This is the InGroup token cacheing.

Thanks for the xp tip though. Yet another nice tool to know about

Kdyer
(KiX Supporter)
2004-04-16 02:55 PM
Re: execution time

If we dig into the manual from KiXtart. It is on pp. 19-20 of the 4.22 version.

Kent


ShawnAdministrator
(KiX Supporter)
2004-04-16 03:09 PM
Re: execution time

The token cache is only populated when the first INGROUP function is executed. If you never execute a GROUP related function, then the cache never gets created. And of course, if the cache is already there and the cache age hasn't expired, then Kixtart will just use the cache instead of re-filling it everytime.

btw, think the manual is a tad mis-leading about how this works.

burnsc
(Starting to like KiXtart)
2004-04-16 03:25 PM
Re: execution time

This was not definitive enough for my implementations. "This group is filled once every kixstart session" does not tell me when during that session it is created. If 2 programs use this cache, and they run asynchronously it is very easy to have 'race' conditions.

Granted I am being more picky than an average user about this, but I am looking to use this cache as a resource for another program I am writeing too.

burnsc
(Starting to like KiXtart)
2004-04-16 03:28 PM
Re: execution time

Thanks for the information.

I think it is more vague than misleading. But, for most 'normal' implentations this is enough.

Les
(KiX Master)
2004-04-16 03:37 PM
Re: execution time

I don't see the point of concerning yourself with how the prefetch works since you cannot control its behaviour. There are a couple of discussions in the beta forum on this. Granted, the topic is macros but howards tests reveal more than just macros.

@Macros
More on @Macros

Back to the topic of protocols and provider order, build yourself a new image and leave the NetWare stuff out, and you will see the delays are no more. Then add NetWare and start tweaking. How hard can it be?

burnsc
(Starting to like KiXtart)
2004-04-16 03:44 PM
Re: execution time

My primary reason for that question was not as much control as it was to avoid the 'trial and error' work that someone had already done. It also adds information for the support spectrum of the program. Thanks for the links, I am sure they will help.

On the protocol setup, I am going to try to have one of my test machines available in a few days to try those kinds of setups, but unfortunately they are tied up with other projects at this exact moment.

Thanks for all the help.

ShawnAdministrator
(KiX Supporter)
2004-04-16 04:25 PM
Re: execution time

Lets seperate token caching from prefetching for a sec, they're two different thingies from my point of view. Based on the event log error messages your getting, the first message hints that the prefetching of @MACROS isn't working - this would happen (imho) for any script you run - regardless of whether you use INGROUP or not. Ruuds comments in the thread i linked you to suggests that if the pre-fetching of @MACROS fails, then the INGROUP caching will also fail which, based on the logs, holds water.

burnsc
(Starting to like KiXtart)
2004-04-16 04:37 PM
Re: execution time

I can run the timing script that was given. It produces no errors in the event log.
That script contains no @ macros that I can tell. But then neither does my logon script. Unless something is triggered in the InGroup commands processing.

Les
(KiX Master)
2004-04-16 04:52 PM
Re: execution time

I did say that it was a little off-topic, but the point is that certain macros must make a trip over the network to populate. Ruud tries to optimize by populating all the macros that would make the trip when the first macro is called and that is what I call the prefetch. He does the same thing with macros that do not make a network trip but those are populated at the start of the script regardless of whether a macro is called or not.

What is not entirely clear is whether a macro call the makes a network trip will also prefetch what InGroup() need to make a trip for or not. I have in the past run tests over a dialup link on all the macros to see which ones cause a network trip but I have misplaced the results. At the time, I was trying to optimize a script for dialup users and wanted to know what macros and functions I should avoid to minimize network trips.

If anyone wants to run those sort of tests, there are two ways to do it. One would be to run a network trace, and the other would be trial and error of macros and functions in various combinations over a dialup line, timing the results each time.