#71155 - 2002-10-25 09:19 PM
udf-link collector
|
Lonkero
KiX Master Guru
Registered: 2001-06-05
Posts: 22346
Loc: OK
|
I made a script to collect the udf-library topics.
funny thing is that it says that the bloody page has no more than one topic and even that one does not have author.
can't get my head over this.
script is so simple and so easy and still, no can get it.
I made so debug lines like:
?substr($pages,15)
all of those return ""
at the first point, $page[$c] returns some stuff and later on, it's empty...
this is so weird.
code:
$udfLibrary = createobject("microsoft.xmlhttp")
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum&f=12&DaysPrune=1000&submit=Go",not 1)
$udfLibrary.send
$pages=$udfLibrary.responsetext
$pages=substr($pages,instr($pages,"Page: 1"))
$pages=left($pages,instr($pages,chr(10))-3)
$pages=substr($pages,instrrev($pages,">")+1)
$pages=val(trim(left($pages,instr($pages,"<")-1)))
dim $page[$pages-1]
for $c=0 to ubound($page)
$page[$c]=""
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum;f=12;hardset=1000;start_point="+($c*25) ,not 1)
$udfLibrary.send
$pages=$udfLibrary.responsetext
$place=instr($pages,'<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t=')
while $place
$pages=substr($pages,$place)
$page[$c]=$page[$c]+trim(left($pages,instr($pages,"</a>")+3))
?substr($pages,15)
$pages=substr($pages,instr($pages,'<font size="2" face="Verdana, Helvetica, sans-serif">')+54)
??$page[$c]
?substr($pages,15)
$page[$c]=$page[$c]+chr(9)+substr($pages,instr($pages,"</font>")-1)+chr(10)
?$page[$c]
?substr($pages,15)
$place=instr($pages,'<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t=')
get $ if $="y" exit 0 endif
loop
next
for each $pages in $page
$pages
next
get $
_________________________
!download KiXnet
|
|
Top
|
|
|
|
|
#71159 - 2002-10-26 02:17 AM
Re: udf-link collector
|
Lonkero
KiX Master Guru
Registered: 2001-06-05
Posts: 22346
Loc: OK
|
ok, don't know where I'm going wrong, but decided to cut the page it generates like bb does.
every page gets 25 "pages"?
it kinda starts ok, but it does not cut it...
code:
$udfLibrary = createobject("microsoft.xmlhttp")
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum&f=12&DaysPrune=1000&submit=Go",not 1)
$udfLibrary.send
$c=$udfLibrary.responsetext
$c=substr($c,instr($c,"Page: 1"))
$c=left($c,instr($c,chr(10))-3)
$c=substr($c,instrrev($c,">")+1)
$c=val(trim(left($c,instr($c,"<")-1)))
dim $pages[$c-1]
for $c=0 to ubound($pages)
$pages[$c]=""
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum;f=12;hardset=1000;start_point="+($c*25) ,not 1)
$udfLibrary.send
$page=$udfLibrary.responsetext
$place=instr($page,'<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t=')
while $place
$page=substr($page,$place)
$pages[$c]=$pages[$c]+trim(left($page,instr($page,"</a>")+3))
$page=substr($page,instr($page,'<font size="2" face="Verdana, Helvetica, sans-serif">')+54)
$pages[$c]=$pages[$c]+chr(9)+substr($page,instr($page,"</font>")-1)+"<br>"
$place=instr($page,'<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t=')
loop
next
del @scriptdir+"\list.htm"
for $c=0 to ubound($pages)
$=open(1,@scriptdir+"\list$c"+".htm",5)
$=writeline(1,$pages$[c])
$=close(1)
next
_________________________
!download KiXnet
|
|
Top
|
|
|
|
|
#71161 - 2002-10-26 02:47 AM
Re: udf-link collector
|
Lonkero
KiX Master Guru
Registered: 2001-06-05
Posts: 22346
Loc: OK
|
got a new one out of it and, surely it looks perfect...
just wonder where are all the topics ![[Big Grin]](images/icons/grin.gif)
code:
$udfLibrary = createobject("microsoft.xmlhttp")
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum&f=12&DaysPrune=1000&submit=Go",not 1)
$udfLibrary.send
$pages=$udfLibrary.responsetext
$pages=substr($pages,instr($pages,"Page: 1"))
$pages=left($pages,instr($pages,chr(10))-3)
$pages=substr($pages,instrrev($pages,">")+1)
$pages=val(trim(left($pages,instr($pages,"<")-1)))
for $c=0 to $pages-1
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum;f=12;hardset=1000;start_point="+($c*25) ,not 1)
$udfLibrary.send
$page=$udfLibrary.responsetext
$page=split($page,'<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t=')
for $=1 to ubound($page)
$page[$]='<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t='+substr($page[$],instr($page[$],"</a>")+3)
$page[$]=$page[$]+substr($page[$],instr($page[$],'<font size="2" face="Verdana, Helvetica, sans-serif">')+54)
$page[$]=left($page[$],instr($page[$],"</font>")-1)
next
$=open(1,@scriptdir+"\list$c"+".htm",5)
for each $line in $page
$=writeline(1,$line+"<br>")
next
$=close(1)
next
_________________________
!download KiXnet
|
|
Top
|
|
|
|
|
#71162 - 2002-10-26 03:45 AM
Re: udf-link collector
|
Lonkero
KiX Master Guru
Registered: 2001-06-05
Posts: 22346
Loc: OK
|
ok,
first really working collector.
les, this also can be pretty slow on your wksta.
50kbytes*11=5hours?
code:
$udfLibrary = createobject("microsoft.xmlhttp")
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum&f=12&DaysPrune=1000&submit=Go",not 1)
$udfLibrary.send
$pages=$udfLibrary.responsetext
$pages=substr($pages,instr($pages,"Page: 1"))
$pages=left($pages,instr($pages,chr(10))-3)
$pages=substr($pages,instrrev($pages,">")+1)
$pages=val(trim(left($pages,instr($pages,"<")-1)))
$=open(1,@scriptdir+"\listUDF"+".htm",5)
for $c=0 to $pages-1
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum;f=12;hardset=1000;start_point="+($c*25) ,not 1)
$udfLibrary.send
$page=$udfLibrary.responsetext
$page=split($page,'<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t=')
for $=1 to ubound($page)
$line='<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t='+left($page[$],instr($page[$],"</a>")+3)+chr(9)
if instr($line,";p=") $line="" endif
$page[$]=substr($page[$],instr($page[$],'<font size="2" face="Verdana, Helvetica, sans-serif">')+54)
if instr($page[$],"</font>")
$nul=writeline(1,left($line+$page[$],instr($line+$page[$],"</font>")-1)+"<br>")
else
$nul=writeline(1,$line)
endif
next
next
$=close(1)
_________________________
!download KiXnet
|
|
Top
|
|
|
|
|
#71163 - 2002-10-27 03:15 PM
Re: udf-link collector
|
Lonkero
KiX Master Guru
Registered: 2001-06-05
Posts: 22346
Loc: OK
|
yeah yeah yeah.
how to remove those un-udf topics?
can't mask with "()" nor "-" as all udf topic titles do not have them...
anyway, my current lister:
code:
$udfLibrary = createobject("microsoft.xmlhttp")
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum&f=12&DaysPrune=1000&submit=Go",not 1)
$udfLibrary.send
$pages=$udfLibrary.responsetext
$pages=substr($pages,instr($pages,"Page: 1"))
$pages=left($pages,instr($pages,chr(10))-3)
$pages=substr($pages,instrrev($pages,">")+1)
$pages=val(trim(left($pages,instr($pages,"<")-1)))
del @scriptdir+"\listUDF"+".htm"
$=open(1,@scriptdir+"\listUDF"+".htm",5)
for $c=0 to $pages-1
$udfLibrary.open("GET","http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=forum;f=12;hardset=1000;start_point="+($c*25) ,not 1)
$udfLibrary.send
$page=$udfLibrary.responsetext
$page=split($page,'<a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t=')
for $=1 to ubound($page)
$line=left($page[$],instr($page[$],"</a>")+3)+"</td><td>"
if instr($line,";p=") $line="" endif
$page[$]=substr($page[$],instr($page[$],'<font size="2" face="Verdana, Helvetica, sans-serif">')+54)
if instr($page[$],"</font>")
$list=$list+left($line+$page[$],instr($line+$page[$],"</font>")-1)+chr(10)
else
$list=$list+$line
endif
next
next
$list=split($list,chr(10))
;sorting routine
for $c=0 to ubound($list)
if instr($list[$c],'">')
$line=split($list[$c],'">')
$list[$c]=$line[1],$line[0]
$list[$c]=join($list[$c],'">')
endif
next
$list=qs($list)
for $c=0 to ubound($list)
if instr($list[$c],'">')
$line=split($list[$c],'">')
$list[$c]=$line[1],$line[0]
$list[$c]=join($list[$c],'">')
endif
next
;write to file
$c=0 ;count udf's too
$=writeline(1,'<table><th>UDF</th><th>Author</th>')
for each $line in $list
$c=$c+1
$=writeline(1,'<tr><td><a href="http://www.kixtart.org/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=12;t='+$line+'</td></tr>')
next
$=writeline(1,'</table><br>Total Count: $c')
$=close(1)
;QS() -quicksort function by brianTX
function qs($a)
DIM $b[32],$c[32],$d,$e,$f,$g,$h,$i,$j,$k,$l
$b[0]=0
$c[0]=UBOUND($a)
$d=0
While $d >=0
$e=$b[$d]
$f=$c[$d]
While $e < $f
$h=$e+($f-$e)/2
$k=$a[$e]
$A[$e]=$A[$h]
$A[$h]=$k
$i=$e+1
$j=$f
$l=0
Do
While ($i<$j) AND $A[$e] > $A[$i]
$i=$i+1
Loop
While ($j>=$i) AND $A[$j] > $A[$e]
$j=$j-1
Loop
IF $i>=$j
$l=1
ELSE
$k=$A[$i]
$A[$i]=$A[$j]
$A[$j]=$k
$j=$j-1
$i=$i+1
ENDIF
Until $l=1
$k=$a[$e]
$a[$e]=$a[$j]
$a[$j]=$k
$g=$j
If $g-$e <= $f - $g
If $g+1 < $f
$b[$d]=$g+1
$c[$d]=$f
$d=$d+1
Endif
$f=$g-1
Else
If $g-1 > $e
$b[$d]=$e
$c[$d]=$g-1
$d=$d+1
Endif
$e=$g+1
Endif
Loop
$d=$d-1
Loop
$qs=$a
Endfunction
added counter.
[ 27. October 2002, 15:42: Message edited by: Lonkero ]
_________________________
!download KiXnet
|
|
Top
|
|
|
|
Moderator: Glenn Barnas, NTDOC, Arend_, Jochen, Radimus, Allen, ShaneEP, Ruud van Velsen, Mart
|
0 registered
and 2082 anonymous users online.
|
|
|
![[Razz]](images/icons/tongue.gif){kind=icon}
![[Wink]](images/icons/wink.gif){kind=icon}
![[Frown]](images/icons/frown.gif){kind=icon}
