#186722 - 2008-04-04 08:17 PM
Re: Open all files in dir, find phrase, insert text, write file
[Re: Curious]
|
Glenn Barnas
KiX Supporter
   
Registered: 2003-01-28
Posts: 4401
Loc: New Jersey
|
Norwegian? Closest I can come is one word in Swedish!
Glad it's working! It's a perfect opportunity to illustrate RUN vs. SHELL, too.
Let's say that some external process delivers 50 Zip files each day to a network share by 1am. You need to expand and process them by 3am. It takes 40 minutes to unzip the files, one by one, using Shell. It then takes 90 minutes to process the files. Your boss is upset because the report comes out at 03:10 instead of before 03:00. You can't spead up the processing, or can you???
Well, this is where RUN can come in handy, creating a form of multi-threaded scripting. You create a script that runs at 1am and enumerates all of the zip files. For each zip file it finds, it runs an unzip command via RUN. It keeps a running count of the files, and, say, every 10 it processes, it sleeps 30 seconds. Now you have 10 CONCURRENT unzip processes running instead of one at a time. You realize that you can now unzip all of the files in under 10 minutes!! Knowing this, you can run the second script that processes the unzipped files, starting at 01:20, and completing at 02:50.
I use this type of multi-threading to fire off 25 Kix scripts at a time. These process logs from 300+ systems in about 12 minutes instead of the 3 hours it took doing them one at a time. In my environment, I actually keep track of the number of active subprocesses, firing off 25, and 5 more each time the count drops to 20 or less. Shawn used this concept recently with some impressive results.
Also, just something to consider - this was pulled directly from your code..
$WorkFile = $RootPath+$Workdir+'WorkFile.tmp'
$OutPutDir = $RootPath + 'OutDir\'
Which line is easier to read?  Having an open and clear (and consistent) format will go a long way in making your code more supportable. Just something to consider.
Glenn
_________________________
Actually I am a Rocket Scientist!
|
|
Top
|
|
|
|
Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-03-25 03:48 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Mart
|
2008-03-25 04:50 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
NTDOC
|
2008-03-25 06:29 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Mart
|
2008-03-25 08:24 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-03-26 11:06 AM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Mart
|
2008-03-26 01:35 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Glenn Barnas
|
2008-03-26 03:36 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-01 04:05 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-01 08:57 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-01 11:27 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Glenn Barnas
|
2008-04-02 12:32 AM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-03 01:20 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Glenn Barnas
|
2008-04-03 03:59 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-04 05:03 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Glenn Barnas
|
2008-04-04 08:17 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-05 12:11 AM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Glenn Barnas
|
2008-04-05 01:45 AM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-05 04:21 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Les
|
2008-04-03 05:06 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-01 09:32 AM
|
Re: Open all files in dir, find phrase, insert text, write file
|
NTDOC
|
2008-03-26 08:10 PM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Curious
|
2008-04-01 09:57 AM
|
Re: Open all files in dir, find phrase, insert text, write file
|
Glenn Barnas
|
2008-04-01 01:05 PM
|
|
Moderator: Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart
|
0 registered
and 1044 anonymous users online.
|
|
|