Page 1 of 1 1
Topic Options
#139791 - 2005-05-16 08:51 PM FYI -- File locking woes for noobies (like me)
thepip3r Offline
Hey THIS is FUN
*****

Registered: 2005-03-02
Posts: 350
If any of this is incorrect or incomplete, please correct me or add the information so others may see.

In working on my current project involving KiX, I've been working with the forum on good coding practices for writing error logs to a network file location. In this endeavor, I've come across a peculiar problem; Windows appeared to be "locking" files when multiple people were writing to the same file at once.

In my case, when a user logs onto the network, this script runs. If at any of the key points it fails, it writes an error log to a network share in the format of "current_date.xls". The information that was logged was the information contained in @ERROR and @SERROR along with the computer name, user, and time of the error. When I was writing the file to .doc's and .xls files, I'd have massive extra log files in my directory outside of the current date. It seems that when using any office extension to write your log file, Windows locks the file while it's in use so no one else can write to it.

So... in playing around with different formats, it appears this problem only pertains to file extensions where that extension is configured to open up with an Office Suite program. If you use an extension like .log or .txt, one that is not commonly opened by Office apps, your log file should work fine. I'm using .log and have yet to get an extra file to appear.

Top
#139792 - 2005-05-16 09:01 PM Re: FYI -- File locking woes for noobies (like me)
Howard Bullock Offline
KiX Supporter
*****

Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
There have been serveral threads that cover this topic. My solution is write a log file for each computer name then collect and consolidate the data.
_________________________
Home page: http://www.kixhelp.com/hb/

Top
#139793 - 2005-05-16 09:07 PM Re: FYI -- File locking woes for noobies (like me)
Howard Bullock Offline
KiX Supporter
*****

Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
Here is one of those links: writing to log file on remote server
_________________________
Home page: http://www.kixhelp.com/hb/

Top
#139794 - 2005-05-16 09:27 PM Re: FYI -- File locking woes for noobies (like me)
thepip3r Offline
Hey THIS is FUN
*****

Registered: 2005-03-02
Posts: 350
I read over your thread and having a different log file for each computer seems a little excessive. I also read that you have a script to consolidate the files but why not just write the log to an extension where it only writes a log file with the computer in it's name if it can't write to the initial file.

My current writing process is working for a possible 3000 users and I haven't gotten an excess file in my logging directory yet since I've changed the file extension to .log...

Top
#139795 - 2005-05-16 09:36 PM Re: FYI -- File locking woes for noobies (like me)
Howard Bullock Offline
KiX Supporter
*****

Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
I really do not think that the file extensions has much to do with file contention. There are 35,000 computers here that do cause file contention during the peak logon hours. The individual log files which are picked up and written to an Oracle DB works fine for me. Once written, I haven't touched it in years.
_________________________
Home page: http://www.kixhelp.com/hb/

Top
#139796 - 2005-05-16 09:55 PM Re: FYI -- File locking woes for noobies (like me)
thepip3r Offline
Hey THIS is FUN
*****

Registered: 2005-03-02
Posts: 350
well like I said, when I was using .xls/.doc, I was getting excess log files like crazy, after changing the extension, no extra log files. I don't know what the reason is either and I couldn't find an answer on the Internet.

If someone does know the reason, please share; that was the purpose of this thread...

Top
#139797 - 2005-05-16 10:04 PM Re: FYI -- File locking woes for noobies (like me)
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
My guess is that if you were using COM components to write to these files, they would take a lot longer to write to than simple text files, hence greater opportunity for contention.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#139798 - 2005-05-16 10:14 PM Re: FYI -- File locking woes for noobies (like me)
Chris S. Offline
MM club member
*****

Registered: 2002-03-18
Posts: 2368
Loc: Earth
You can still run into file contention with text files. Run this script on two different computers at the same time...

Code:

Break On

IF Open( 3 , "\\someserver\someshare\TEMP\LOG.TXT" , 5 ) = 0
Do
$x = WriteLine( 3 , @WKSTA + ": " + @TIME + @CRLF )
Sleep 1
Until @ERROR
ELSE
BEEP
? "failed to open file, error code : [" + @ERROR + "]"
ENDIF


Top
#139799 - 2005-05-16 10:41 PM Re: FYI -- File locking woes for noobies (like me)
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Chris,
I don't dispute that. It's like shooting a machine gun through the path of a rotating prop... as long as the timing is right, there is no problem.

As for his claim "I haven't gotten an excess file in my logging directory", that is totally bogus. The extra files would be a side effect of what creates them, not what extension is used.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#139800 - 2005-05-16 11:10 PM Re: FYI -- File locking woes for noobies (like me)
thepip3r Offline
Hey THIS is FUN
*****

Registered: 2005-03-02
Posts: 350
well it may be "bogus" Les but all I have to do is change the file extension in the kix script to start writing to an .xls extension instead of .log and my output directory gets multiple files with computer names in them and they are the same computers that are showing up in my multi-line .log file which logs them fine. and i'm not looping until the file is accessible as in Chris's example...
Top
#139801 - 2005-05-16 11:38 PM Re: FYI -- File locking woes for noobies (like me)
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Well... since we haven't seen a lick of code, all we can do is surmise. My Carnac hat quit working when Johnny died!
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#139802 - 2005-05-16 11:43 PM Re: FYI -- File locking woes for noobies (like me)
thepip3r Offline
Hey THIS is FUN
*****

Registered: 2005-03-02
Posts: 350
well it's the same code I was asking about the other day:

Code:
function ErrorLog($ErrorMsg)
DIM $LogFile,$LogError,$AltLogFile,$error,$serror,$ErrorString

$error = @ERROR
$serror = @SERROR
$ErrorString = "@WKSTA,@USERID,@DATE,@TIME,$error,$serror,$ErrorMsg,@CRLF"
$LogFile = "\\server\SoftwarePushes\sms\logging\@Year_@MonthNo_@MDayNo.xls"
$AltLogFile = "\\server\SoftwarePushes\sms\logging\@Year_@MonthNo_@MDayNo-@WKSTA.xls"
$LogError = OPEN (5,$LogFile,5)

If $LogError=0
$ = WRITELINE(5,$ErrorString)
$ = CLOSE(5)
Else
$ = OPEN (6,$AltLogFile,5)
$ = WRITELINE(6,$ErrorString)
$ = CLOSE (6)
EndIf
endfunction



Edited by thepip3r (2005-05-16 11:47 PM)

Top
#139803 - 2005-05-16 11:48 PM Re: FYI -- File locking woes for noobies (like me)
Kdyer Offline
KiX Supporter
*****

Registered: 2001-01-03
Posts: 6241
Loc: Tigard, OR
How can you do a WRITELINE to an EXCEL file? Excel is not a raw text file.

Kent
_________________________
Utilize these resources:
UDFs (Full List)
KiXtart FAQ & How to's

Top
#139804 - 2005-05-16 11:59 PM Re: FYI -- File locking woes for noobies (like me)
thepip3r Offline
Hey THIS is FUN
*****

Registered: 2005-03-02
Posts: 350
I don't know. I just specified .xls and an .xls file popped up in my logging directory. it just writes all of the information to the first columns cell when I open the actual file. I've since changed the extension to .csv. would that be better kyder?
Top
#139805 - 2005-05-17 12:05 AM Re: FYI -- File locking woes for noobies (like me)
Kdyer Offline
KiX Supporter
*****

Registered: 2001-01-03
Posts: 6241
Loc: Tigard, OR
CSV is a text file. EXCEL (.XLS) is usually generated by EXCEL and is a binary.

You should be careful just changing file extensions around if you are not sure what they are.

WRITELINE/READLINE will work in the CSV file.

Kent
_________________________
Utilize these resources:
UDFs (Full List)
KiXtart FAQ & How to's

Top
#139806 - 2005-05-17 12:14 AM Re: FYI -- File locking woes for noobies (like me)
NTDOC Administrator Offline
Administrator
*****

Registered: 2000-07-28
Posts: 11628
Loc: CA
I agree with Howard, unless you're ready to use a Database solution and write directly to the database then I would log the data as @WKSTA+'.TXT'

Then use another process either KiX or batch and copy all the log files into a single file?

Depends on how you want to process and report on this data.

The datase solution is not without potential errors either. There are many, many things that can cause a write or connection failure. Detecting and dealing with that within KiXtart is not a trivial task for a new user.

Aside from that EVERY solution aside from a database that writes to the same file can potentially have locking issues except the method described by Howard.

Top
#139807 - 2005-05-17 12:23 AM Re: FYI -- File locking woes for noobies (like me)
thepip3r Offline
Hey THIS is FUN
*****

Registered: 2005-03-02
Posts: 350
Thanx for the suggestion NTDOC. I currently am using a database solution for that script I was telling you about and this error reporting is specifically set up to track database connectivity and write errors for that project. I'm still looking into how file locking works but will defer to Mr Bullock's example for my current situation.

Edited by thepip3r (2005-05-17 12:30 AM)

Top
#139808 - 2005-05-17 12:32 AM Re: FYI -- File locking woes for noobies (like me)
NTDOC Administrator Offline
Administrator
*****

Registered: 2000-07-28
Posts: 11628
Loc: CA
File locking works because each user that attempts to access the same file for updating opens it for write access. The OS then locks this file to the handle that opened it first and will not allow another handle to open it for write access until it has been closed by the orignal opening handle.

With a database it actually is the same except that a databse often has different methods of handling mutliple writes to the same field. However since each user would rarely write to the same fields at the same time then that type of lock is often a non issue.

Top
#139809 - 2005-05-17 12:45 AM Re: FYI -- File locking woes for noobies (like me)
Kdyer Offline
KiX Supporter
*****

Registered: 2001-01-03
Posts: 6241
Loc: Tigard, OR
You may also look into the use of WRITELOG() or WRITELOG2() UDFs which attempt to address file locking.

Kent
_________________________
Utilize these resources:
UDFs (Full List)
KiXtart FAQ & How to's

Top
#139810 - 2005-05-17 02:12 AM Re: FYI -- File locking woes for noobies (like me)
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
I think the higher incidence of file locking has to do with antivirus settings.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
Page 1 of 1 1


Moderator:  Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart 
Hop to:
Shout Box

Who's Online
0 registered and 874 anonymous users online.
Newest Members
StuTheCoder, M_Moore, BeeEm, min_seow, Audio
17884 Registered Users

Generated in 0.076 seconds in which 0.027 seconds were spent on a total of 11 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org