Page 1 of 1 1
Topic Options
#23112 - 2002-06-13 02:37 PM Getting HTML Source Code from IE & Send to Text File
Jayson75 Offline
Lurker

Registered: 2002-06-12
Posts: 2
Loc: Niagara Falls NY
Greetings All,

Hoping someone would know how to script a way to get the source code from a webpage and perhaps send it to a text file (we use IE).

I'm basically putting together a simple test script that will automatically check the source code and compare it to a "successful test". Unfortunately these webpages exist on our client servers which we don't have direct access to (and we often find the problem 1st). If someone could provide a way to just pull the html source code I can do the rest.

Thanks in advance!
Jason

Top
#23113 - 2002-06-13 02:59 PM Re: Getting HTML Source Code from IE & Send to Text File
Shawn Administrator Offline
Administrator
*****

Registered: 1999-08-13
Posts: 8611
Welcome Jayson,

I'm having a bit of trouble getting you exactly the script you may be looking for, but in the meantime, might want to check-out the Internet Stock Quote script - specifically read-up to the point in the script where it fetches the HTML - then you could just write it out to a file. Heres the link, post any questions you may have:

Stock Quotes

-Shawn

Top
#23114 - 2002-06-13 03:05 PM Re: Getting HTML Source Code from IE & Send to Text File
kasul Offline
Fresh Scripter
*****

Registered: 2001-12-03
Posts: 41
Loc: Hannover / Germany
Hi Jayson,

how about using some external cmd-line Tool named "url2file.exe" which loads a given url to a specific file.

klaus

[ 13 June 2002, 15:06: Message edited by: kasul ]
_________________________
german native-> poor english sysadmin-> poor german...

Top
#23115 - 2002-06-13 03:26 PM Re: Getting HTML Source Code from IE & Send to Text File
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22346
Loc: OK
my codeparser takes sourcecode and parses it.
about it is in http://81.17.37.55/cgi-bin/ultimatebb.cgi?ubb=get_topic&f=13&t=000099&p=

older kix code
hmm...
here http://www.gwspikval.com/jooel/scripts/BBCodeParser/Older%20versions/2.0.1/BBCodeP arser2.kix

all the commands to get a page is:
$httpObj = createobject("winhttp.winhttprequest.5")
$httpObj.open("GET",$page)
$httpObj.send()
$data = $httpObj.Responsetext

the $data has the page's html source inside.

for this you need to have winhttp5.dll which is included in bbcodeparser and it can be downloaded from microsoft website http://msdn.microsoft.com/downloads/default.asp?URL=/code/sample.asp?url=/MSDN-FILES/027/001/655/msdncompositedoc.xml
from there you get the whole sdk.

cheers,

[ 13 June 2002, 15:27: Message edited by: Lonkero ]
_________________________
!

download KiXnet

Top
#23116 - 2002-06-13 03:31 PM Re: Getting HTML Source Code from IE & Send to Text File
Shawn Administrator Offline
Administrator
*****

Registered: 1999-08-13
Posts: 8611
Think Lonkero is right, the winhttp object does a better, quicker job of returning HTML.
Top
#23117 - 2002-06-13 04:51 PM Re: Getting HTML Source Code from IE & Send to Text File
BrianTX Offline
Korg Regular

Registered: 2002-04-01
Posts: 895
Hmm.. I fiddled around with it and the previous one works fine:

code:
break on
$URL = 'http://www.yahoo.com'
$ie = createobject("internetexplorer.application")
$ie.navigate($URL)
while $ie.readystate <> 4 and $ie.busy and @error = 0 loop
while $ie.document.readystate <> "complete" and @error = 0 loop
$html = $ie.document.documentelement.innerhtml

$html does include the complete text of the file. However, I think there is another way to do this other than using IE or the winhttp.... i'm looking into it.

Brian

Top
#23118 - 2002-06-13 05:00 PM Re: Getting HTML Source Code from IE & Send to Text File
Shawn Administrator Offline
Administrator
*****

Registered: 1999-08-13
Posts: 8611
The "uncomplete" text string thingy to do with the string size limitations of KiXtart (I think) ... which I'm still having a hard time trying to figure out. Since 4.0, I've seen instances where this wasn't a problem, and others were it is a problem ...
`
{EDIT}

oops - you said complete - not incomplete. Still trying to figure that one out though.

[ 13 June 2002, 17:01: Message edited by: Shawn ]

Top
#23119 - 2002-06-13 05:07 PM Re: Getting HTML Source Code from IE & Send to Text File
BrianTX Offline
Korg Regular

Registered: 2002-04-01
Posts: 895
I'm still messing with it, too. I'm not sure about all websites, but I saved the result to a file and opened it and it is a perfect .html of the yahoo website.... the file is also 24,784 long. I was fiddling around with other ways to do this, but I don't see an easy way to write the source code to a file with IE other than the one previously shown. I was able to get it to write a file that was interpreted html, but not the raw source code, although I must admit I'm a novice controlling IE with this method.)

Brian

(I suppose string length becomes an issue at > 32000 characters... so this wouldn't be an adequate test.. On that note, though, shouldn't there be a way to save the document object as source via object method without importing the whole string into an object in KiXtart?)

[ 13 June 2002, 17:17: Message edited by: BrianTX ]

Top
#23120 - 2002-06-14 04:12 AM Re: Getting HTML Source Code from IE & Send to Text File
Bryce Offline
KiX Supporter
*****

Registered: 2000-02-29
Posts: 3167
Loc: Houston TX
something like this??

code:
break on

$htmlfile = "http://isorg.net"
$outfile = "html.txt"

$ie=createobject("internetexplorer.application")


? "Downloading html file"
$ie.navigate($htmlfile)
while $ie.busy and $ie.readystate <> 4 and @error = 0 sleep(1) "." loop

if $ie.document
$doc = $ie.document
$data = $doc.documentelement.innerhtml
$nul = open(1,$outfile,5)
$nul = writeline(1,$data)
endif

as far as the string size limit.... i found that as long as your don't try to display large string's, Kix can still handle them.

using the above method i was downloading a inventory file that was on average 170k parsing the information and saving it to the hard disk with out a problem.

Bryce

Top
#23121 - 2002-06-14 04:17 AM Re: Getting HTML Source Code from IE & Send to Text File
Radimus Moderator Offline
Moderator
*****

Registered: 2000-01-06
Posts: 5187
Loc: Tampa, FL
he's back.....
_________________________
How to ask questions the smart way <-----------> Before you ask

Top
#23122 - 2002-06-14 04:18 AM Re: Getting HTML Source Code from IE & Send to Text File
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22346
Loc: OK
the reason my newist codeparser still used winhttp was that ie was not able to return from here (kixtart.org/bb) other than page header.

winhttp seems to work in pretty different way and it works fine with cgi/php (server side worked stuff) which ie.application didn't seem to comfort with.

dunno...
_________________________
!

download KiXnet

Top
#23123 - 2002-06-14 04:19 AM Re: Getting HTML Source Code from IE & Send to Text File
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22346
Loc: OK
yeah rad. and at right time.
right time is when bb is silent for some 60 minutes and you are almost falling a sleep.

[ 14 June 2002, 04:20: Message edited by: Lonkero ]
_________________________
!

download KiXnet

Top
#23124 - 2002-06-14 04:20 AM Re: Getting HTML Source Code from IE & Send to Text File
Bryce Offline
KiX Supporter
*****

Registered: 2000-02-29
Posts: 3167
Loc: Houston TX
shhhhh

I have been lurking, but normally yall beat me to the problems [Razz]

Top
#23125 - 2002-06-19 01:56 PM Re: Getting HTML Source Code from IE & Send to Text File
Jayson75 Offline
Lurker

Registered: 2002-06-12
Posts: 2
Loc: Niagara Falls NY
WoW thanks for the quick responses! I'll work in the above examples next week and let you know how it goes. Good news is the pages I need the source code from aren't very large. Thanks again!
Top
Page 1 of 1 1


Moderator:  Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart 
Hop to:
Shout Box

Who's Online
0 registered and 1044 anonymous users online.
Newest Members
StuTheCoder, M_Moore, BeeEm, min_seow, Audio
17884 Registered Users

Generated in 0.064 seconds in which 0.024 seconds were spent on a total of 12 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org