Page 1 of 2 12>
Topic Options
#170963 - 2006-12-04 06:26 AM Dirplus to gather files
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
I have an archive directory that I'm going to use dirplus and 7zip to create a zip file for every day of the year. This is a huge directory without subdirectories containing pdf's, csv's and tabs with this name format (RPT710.D.00001.ALL.20061203.0000001.CSV). What would be the best way to increment through every day (about two years worth of files) with a dirplus command? I've thought about creating a text file of dates and looping through that array and dirplus each element, but not sure if that would be very efficient.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#170964 - 2006-12-04 06:53 AM Re: Dirplus to gather files [Re: brewdude6]
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
If the date is always in the same place and in the same format, you can Split() the array that is returned by DirPlus, and then use that information to move forward.

Code:
$Counter2 = 0
For $Counter = 0 to UBound($DirPlus_Array)
  $Split_Name = Split($DirPlus_Array[$Counter],".")
  If $SPlit_Name[3] = $Specified_Date
    Redim Preserve $Build_Array[$Counter2]
    $Build_Array[$Counter2] = $DirPlus_Array[$Counter]
    $Counter2 = $Counter2 + 1
  EndIf
Next
_________________________
Today is the tomorrow you worried about yesterday.

Top
#170981 - 2006-12-04 04:15 PM Re: Dirplus to gather files [Re: Gargoyle]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
There are 1.5 million files in this directory. Should I break up the arrays created somehow?
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#170988 - 2006-12-04 07:07 PM Re: Dirplus to gather files [Re: brewdude6]
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
I do not know what the limitations of KiX is in that regards, while it may take time to search an array with 1.5 million entries, it is not like you are going to be doing this on every users workstation. It may be inconvienent for the person that ends up running the script.

Others may be able to answer if there is a limitation with KiX in regards to the size of the array's that you would be working with.
_________________________
Today is the tomorrow you worried about yesterday.

Top
#170990 - 2006-12-04 07:15 PM Re: Dirplus to gather files [Re: Gargoyle]
NTDOC Administrator Offline
Administrator
*****

Registered: 2000-07-28
Posts: 11571
Loc: CA
The memory of the machine and string length are the only two I can think of.

A few people have run some large array scripts similar and didn't seem to have an issue. Now if you're trying to put the array back together as a string then concatenated then yes you're going to bump into a string lenght issue pretty quick on that size.

But processing each element should be okay as long as there is enough memory in the system.

Top
#171024 - 2006-12-05 12:06 AM Re: Dirplus to gather files [Re: Gargoyle]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
How would I find the $Specified_Date variable? Would I need a separate array with those in them?
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#171028 - 2006-12-05 12:32 AM Re: Dirplus to gather files [Re: brewdude6]
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22334
Loc: OK
hmm...
think you could go the other way.

Code:
for $year=2004 to 2006
 for $month=1 to 12
  for $day=1 to 31
   $file=dir("folder\RPT*.ALL."+$year+right("0"+$month,2)+right("0"+$day,2)+".*.CSV")
   $files=""
   while $file and @error=0
    $files=$files+"\"+$file
    $file=dir()
   loop
   if 2<len($files)
    ? "make zip for " $year " " $month " " $day " consisting off:" ?
    for each $f in split($files,"\")
     "  " $f ?
    next
   endif
  next
 next
next


or something similar...
_________________________
!

download KiXnet

Top
#171037 - 2006-12-05 04:19 AM Re: Dirplus to gather files [Re: Lonkero]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
Thanks for the suggestion. If I try this it simply adds all the files to the zip file for the date fields present. I know I've got some variables in strings...just experimenting.

Code:

For $year=2004 to 2006
 For $month=1 to 12
  For $day=1 to 31
   $file=Dir("c:\admin1\*.*.*.*."+$year+Right("0"+$month,2)+Right("0"+$day,2)+".*.*")
   $files=""
   While $file And @error=0
    $files=$files+"\"+$file
    $file=Dir()
   Loop
   If 2<Len($files)
    ;? "make zip for " $year$month$day " consisting off:" ?
    For Each $f in Split($files,"\")
     $f ?
	Shell "%comspec% /c 7z.exe u -tzip c:\admin2\RPT$year$month$day c:\admin1\$f"
    Next
   EndIf
  Next
 Next
Next


Edited by brewdude6 (2006-12-05 05:44 AM)
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#171039 - 2006-12-05 06:55 AM Re: Dirplus to gather files [Re: brewdude6]
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22334
Loc: OK
you should use the right() trick I used for the days so all dates have the same length and you can know what is RPT2004111
currently, it can be eleventh of jan 2004 or first of nov 2004.

but, back to your post.
when you say "if I try this, it simply..."
does that mean that it somehow doesn't work or that it is simply brilliant?
as adding the files to daily archives was the task, right?
_________________________
!

download KiXnet

Top
#171051 - 2006-12-05 01:47 PM Re: Dirplus to gather files [Re: Lonkero]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
It was simply brilliant...pardon me for not clarifying that.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#171056 - 2006-12-05 04:09 PM Re: Dirplus to gather files [Re: brewdude6]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
Are you adding the "\" character to the $files variable so you'll have a delimiter for the split command?

What about the If 2<Len($files) line? I'm not quite sure what checking the length is doing for you here.

Would I need to use the "right" command to pull the string for the file name with the "$files" variable and not the "$file"?

I know this is all over the map, but I'm trying to make sure I understand what is going on.

Thanks for your help.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#171070 - 2006-12-05 09:00 PM Re: Dirplus to gather files [Re: brewdude6]
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22334
Loc: OK
yes, \ is delimiter.
and right(,2) is just for checking if there is files for that day. could have simply used len()

don't use right or anything, you get the filenames if you split with the delimiter like shown in my example.

I also see you removed all the matching characters from the dir:
$file=Dir("c:\admin1\*.*.*.*."+$year+Right("0"+$month,2)+Right("0"+$day,2)+".*.*")

even though, it now requires to have enough dots in the name, but at least leaving the constant values, if there are any should be considered.
if there is none, then it's fine as it is.
_________________________
!

download KiXnet

Top
#171089 - 2006-12-06 05:56 AM Re: Dirplus to gather files [Re: Lonkero]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
I need to create a file for each day ie: RPT20061204.zip with all the files with *.20061204.* in the file name added to that days zip file. Maybe I should just move the files to a directory for each day first to keep things simple, and then zip the folders later.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#171117 - 2006-12-06 03:31 PM Re: Dirplus to gather files [Re: brewdude6]
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22334
Loc: OK
what's wrong with the add method you used earlier?
_________________________
!

download KiXnet

Top
#171263 - 2006-12-08 09:56 PM Re: Dirplus to gather files [Re: Lonkero]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
I don't know how to correct it, but the method that I posted creates a file correctly, but adds every file in the directory to every zip file. I'd also like to pad a "0" to the single digit days of the month.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#171267 - 2006-12-08 10:49 PM Re: Dirplus to gather files [Re: brewdude6]
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22334
Loc: OK
well, then you need to use the right() trick I used with my example.

hmm...
if all files are added to all zips, does all of the files also show in "$f ?" listing?
if yes, sounds like kixtart does not support multiple * wildcards.
but that need to be studied before saying anything sure.
_________________________
!

download KiXnet

Top
#171271 - 2006-12-09 12:35 AM Re: Dirplus to gather files [Re: Lonkero]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
Thanks..the right() trick does work for the file name. The $f? does return the expected files. I'm sure it's the way I'm shelling out to 7zip. For some reason, every file is added to each zip. If I replace the shell line to echo the $f variable I get the correct list of files for each zip.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#171272 - 2006-12-09 12:55 AM Re: Dirplus to gather files [Re: brewdude6]
NTDOC Administrator Offline
Administrator
*****

Registered: 2000-07-28
Posts: 11571
Loc: CA
Not sure if it will help, but you can take a look at the script I ran here using 7Zip.

http://www.kixtart.org/forums/ubbthreads.php?ubb=showflat&Number=135983=#post135984

.

Top
#171273 - 2006-12-09 01:55 AM Re: Dirplus to gather files [Re: NTDOC]
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22334
Loc: OK
weird.
there is nothing special in the shell line that would make it behave like that.
well, except the lack of 2-digit month and day, like already said some days ago.
but, if you still with the right() hack experience this... well, guess, either the script needs to be made a lot more comp...
k, as I was typing I realised it.

change this:
For Each $f in Split($files,"\")

to this:
For Each $f in Split(substr($files,2),"\")

and the symptoms are gone.
_________________________
!

download KiXnet

Top
#171274 - 2006-12-09 02:06 AM Re: Dirplus to gather files [Re: Lonkero]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
SWEET! Thank you very much! I obviously would of never came up with that fix.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
Page 1 of 2 12>


Moderator:  Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Arend_, Mart 
Hop to:
Shout Box

Who's Online
1 registered (Bryce) and 183 anonymous users online.
Newest Members
gkustra, emnipetro, Hirze, thequeen, ameliaryan
17642 Registered Users

Generated in 0.042 seconds in which 0.01 seconds were spent on a total of 14 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org