Page 1 of 3 123>
Topic Options
#110848 - 2003-12-25 06:17 PM 4.5 alpha
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
A couple of observations...

There is no option to specify the resultant filename or extension which defaults to .KX and then .KX is not searched for and run like .KiX is. One must rename it after tokenizing if wanting the search to work. Any thought of adding .KX to the list of extensions searched?

When running the tokenized version, there is no possibility of running it in debug mode. Is this by design? I realize that with DEBUG ON the script would no longer be obfuscated and understand why the /D option is disabled, but should the in-line DEBUG ON also be disabled? Some people embed DEBUG ON in conditionals which will no longer work. 'Suggestions' might be a better place to discuss this but while I'm here... perhaps adding a password option for DEBUG.

Is there going to be an option to reverse the tokenization, that is to return it to text format? I presume all the comments and indenting would be lost... well the commenting anyway, maybe the indenting could be made more consistent.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#110849 - 2003-12-27 01:40 AM Re: 4.5 alpha
Howard Bullock Offline
KiX Supporter
*****

Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
The tokenized version of my logon script code turned out to be half the size of the original non-tokenized scripts.

Just begining to test...
_________________________
Home page: http://www.kixhelp.com/hb/

Top
#110850 - 2003-12-29 08:42 AM Re: 4.5 alpha
Anonymous
Unregistered


Excellent observations!
I'll consider adding an option to specify the resultant filename.

As for adding ".KX" to the auto-search list: I considered this and did not add this as it adds yet another network roundtrip to the startup time of KiXtart. Any thoughts on this are welcome (Maybe I can drop ".SCR" from the auto-search list? Also, if I add ".KX", should it be first in the list?)

As for debugging a KX-file: this is disabled on purpose. Without the source, all KiXtart has is the tokenized stream, and the current implementation of debugging does not deal with that.

Lastly, on 'de-tokenizing': any thoughts on this are appreciated. It's certainly possible, and it probably will only be a matter of time before someone does it. The question is whether I should provide it from day 1.

Ruud

Top
#110851 - 2003-12-29 08:03 PM Re: 4.5 alpha
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Ruud,
I do not have a clear understanding of how KiX searches for the script. The manual says that \Netlogon is searched first, the location where KiX32 started from second, and the current directory third. If it makes its rounds of these locations for each extension before trying another extension then yes, it would add significant delay but if it searched on all possible extensions at each location, I believe the delay could be less if a match is found. Maybe you could provide a search order switch as you do for the RPC.

As for the .SCR extension, I am all for dropping it, since that one is already associated with screensavers. None the less, it is bound to break some implementations.

A primary reason to pre-tokenize might be for speed and bandwidth so putting .KX ahead of .KiX makes sense since it would have a slight leg up on speed and traffic.

As for the debugging, I wasn't sure if you were de-tokenizing in 4.22 while debugging, if you kept both the clear text and tokenized in memory, or if you tokenized line-by line while in debug mode. Some of the challenges that you would face with error reporting if you INCLUDE or MAKE several files into one .KX file, would be the same with in-line debugging. Namely, that you need to represent in clear text, the line and/or line number.

Lastly, a de-tokenizer utility is something that someone may build anyway through reverse engineering. If the intent is to obfuscate the code, the de-tokenizer would make quick work of exposing it which, contrary to Martha, would not be a good thing. If you made available a de-tokenizer, there would have to be some sort of encryption/decryption key to maintain obfuscation. The question is would the benefit of a de-tokenizer outweigh the reduced complexity of cracking the key versus having to crack the built-in key AND tokenization algorithm? Since presumably, the key would have to be embedded in the script, it may also make it easier to crack.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#110852 - 2003-12-30 04:16 AM Re: 4.5 alpha
Sealeopard Offline
KiX Master
*****

Registered: 2001-04-25
Posts: 11164
Loc: Boston, MA, USA
I'd also drop the .SCR extension and give priority for the .KX over the .KIX extension.

De-tokenization is not important to me but it would be nice if there would be a detokenizer. This would keep KiXtart an open system as people would not be able to distribute UDFs in a tokenized way without others being able to see the source code.
_________________________
There are two types of vessels, submarines and targets.

Top
#110853 - 2003-12-30 04:17 PM Re: 4.5 alpha
Bryce Offline
KiX Supporter
*****

Registered: 2000-02-29
Posts: 3167
Loc: Houston TX
where can i get my hands on the 4.5 alpha? It seems that my email/domain went down for about a week or so... did Ruud email it out like usual?

Bryce

Top
#110854 - 2003-12-30 04:19 PM Re: 4.5 alpha
Sealeopard Offline
KiX Master
*****

Registered: 2001-04-25
Posts: 11164
Loc: Boston, MA, USA
Yes, he did.
_________________________
There are two types of vessels, submarines and targets.

Top
#110855 - 2003-12-30 04:51 PM Re: 4.5 alpha
Radimus Moderator Offline
Moderator
*****

Registered: 2000-01-06
Posts: 5187
Loc: Tampa, FL
don't feel bad... I didn't get it either

:-(
_________________________
How to ask questions the smart way <-----------> Before you ask

Top
#110856 - 2003-12-30 04:58 PM Re: 4.5 alpha
Jochen Administrator Offline
KiX Supporter
*****

Registered: 2000-03-17
Posts: 6380
Loc: Stuttgart, Germany
you got mail
_________________________



Top
#110857 - 2003-12-31 07:58 PM Re: 4.5 alpha
MCA Offline
KiX Supporter
*****

Registered: 2000-04-28
Posts: 5152
Loc: Netherlands, EU
Dear,

Some testings show us
- overall the tokenized scripts are shorter
- debugging mode not possible. our alternative with kixstrip still works in a proper way.
- error messages generated by kix32/wkix32 are the same for both versions.
First impression shows no problems with it. All our scripts work in a proper way.
Second impression: a very good extension to kixtart to get more secure scripts.

Our suggestions:
- don't introduce a backdoor to recreate original source.
- for people who want something like a backdoor we suggest that they should tokenize their scripts with a password. only by specifying this password someone can recreate his source.

greetings.
_________________________
email scripting@wanadoo.nl homepage scripting@wanadoo.nl | Links | Summary of Site Site KiXforms FAQ kixtart.org library collection mirror MCA | FAQ & UDF help file UDF kixtart.org library collection mirror MCA | mirror USA | mirror europe UDF scriptlogic library collection UDFs | mirror MCA

Top
#110858 - 2004-01-02 02:24 AM Re: 4.5 alpha
Sealeopard Offline
KiX Master
*****

Registered: 2001-04-25
Posts: 11164
Loc: Boston, MA, USA
Hmm, great, no ability to un-tokenize a script. Let me just write a little UDF that is really helpful with a small side effect of destroying the computer it is run on via a timebomb. Easy to make and impossible to detect with a tokenized script/UDF. The end-result is that you cannot trust tokenized scripts that have not been tokenized by yourself as you have no way of verifying the content, functionality, and integrity of the script.

Just think of an employee making a little change to the login script right before he's been shown the door. It's not such a far-fetched idea as is already happened in real-life.

Anyway, I'd prefer if there would be a way to untokenize the script.
_________________________
There are two types of vessels, submarines and targets.

Top
#110859 - 2004-01-02 02:39 AM Re: 4.5 alpha
ShaneEP Moderator Offline
MM club member
*****

Registered: 2002-11-29
Posts: 2125
Loc: Tulsa, OK
I would rather not have a way to un-tokenize.
Top
#110860 - 2004-01-02 02:43 AM Re: 4.5 alpha
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
I knew this could be a contentious issue between the right to know versus the right to protect intellectual property. If you can de-tokenize, you no longer have obfuscation, which has been a long-standing request. I guess one has to ask what the main intent was for tokenization, performance or obfuscation. Since Ruud does tokenization in 4.22 and obfuscation in 4.5 alpha, I have to assume obfuscation.

When it comes to blind faith, we have plenty of it when we download and use compiled programs that are freeware, shareware, or commercial. That is the risk we take when doing so.

There is always the possibility of an unwelcome going away gift, and that risk must be mitigated through policy and monitoring. There is nobody forcing you to use obfuscated scripts.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#110861 - 2004-01-02 03:48 AM Re: 4.5 alpha
NTDOC Administrator Offline
Administrator
*****

Registered: 2000-07-28
Posts: 11623
Loc: CA
To assume that no one will write a method of their own sooner or later to dechipher the code back to an original readable format is in my opinion to attempt to hide from reality.

If you're on a ship that just hit something that ripped a large hole in it and was now sinking and the Captain and Crew are now asking you to man life boats and you just hide in your stateroom pretending it is not happening, is about the same type of mentality as I see it.

Also similar reasons that Jens has stated.

1. You write a cool logon script and it is working great then you accidentally drop the original readable version into a file shredder program along with some other files. Now you no longer have the original and have to start all over from scratch. Or some other Admin deletes that original accidentally as you Company mandates that all shared code is kept on one location. What ever the case, this is just an idea to get a point accross.

2. One of your Admins is laid off or let go but is now PISSED OFF and leaves a time-bomb like a script that schedules a task on all systems desktop and server that now on a certain day and time deletes every thing it can in the registry and overwrites all files it can with 0-byte size files. I don't care how nice of an organized shop you run, IF something like that were to happen it would take a LONG time to recover from the damage inflicted. However, if the code was in readable format it would be much harder to hide such an act.

3. You are hired on at a new Company and the VP says he really LOVES the script the last guy wrote and all it working great just the way they want it, but he wants to add just a couple little items to the original. Well, without the original or without being able to restore to readable format your kind of stuck.

Perhaps allowing something like during setup of the program it is "signed" or pass-phrase etc first before the EXE can be used. Then only Admins with this pass-phrase can decipher the tokenized code. Then if an Admin is let go, the process can re-code a new EXE and replace the file on the Servers so that that Admin won't know the new pass-phrase. I myself don't view this as a "SECURITY TOOL" so much as I see it as an added measure of protection to prevent non admins from knowing what is going on in the script if wanted. Just because KiX will now support Obfuscation does not mean all Companies will use it either.

It also may be possible that in some Countries it "might" be illegal to support Obfuscation without a means to restore the original source. Possibly opening a means of litigation against Ruud. (not that they would have any legal recourse against him, but something to ponder I guess)

I'm sure this is a subject that as with most other subjects that not EVERYONE will agree with each other.

Regardless of what or how Ruud moves forward with supporting this hopefully no one runs into any real problems with it in the future.

Top
#110862 - 2004-01-02 03:55 AM Re: 4.5 alpha
Sealeopard Offline
KiX Master
*****

Registered: 2001-04-25
Posts: 11164
Loc: Boston, MA, USA
However, the main question is, whether tokenization the way Ruud implements it is actually reversible in such a way that the original script is more or less recreated. It might not even necessarily be a one-to-one recreation of the original script down to the variable names or comment fields.
_________________________
There are two types of vessels, submarines and targets.

Top
#110863 - 2004-01-02 04:49 AM Re: 4.5 alpha
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Since Ruud holds the secret to how the tokenizing works, he is the best one to answer just how much of the original would be restored. I have run some experiments to see if the var names are tokenized or not. I thought they too would have been tokenized to really optimize the code but if I use longer var names, the tokenized file is larger. I think var names should also be converted to tokens for best performance.

Comments seem to not make it through the tokenization since changing the amount of comments does not change the resultant file size.

I can see where some companies may try to ban the use of KiX if we have obfuscation without reversal. Very often, it is not mainstream programmers that code scripts but rather sysadmins. I work for divisional IT and constantly run into red tape by corporate IT that wants DivIT to be nothing more than PC monkeys yet CorpIT cannot provide the level of service that my users need. There are a lot of companies that ban KiX because it is viewed as ShareWare without full support. Add to that, the argument that sysadmins, who may not follow rigid procedures of version and change control with the potential to build obfuscated code and not protect the source, will have the potential to create a vast legacy of unsupportable scripts.

There are a lot of people that are pinning their hopes on obfuscation being a panacea of encryption to not only protect their code as proprietary, but also as a secure(?) way to delegate admin rights through their scripts. It is not now and probably will never be truly secure, at least not as long as it runs in the security context of the user.

If the emphasis is more on simple obfuscation, Brian's suggestion to use compression after the tokenization might suffice. Still, it is a matter of time before someone cracks either the simple or the more complex obfuscation and then publishes it on the web.

I just can't think of how one can have obfuscation and the guaranteed ability to reverse it to clear text without a backdoor password that would soon enough be in the public domain. About the only thing I can think of is to embed the domain name and SID in the script and to only allow a Domain Admin to use the backdoor password. While the domain name could be duplicated, the SID would not be the same.


Edited by Les (2004-01-02 04:51 AM)
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#110864 - 2004-01-02 05:35 AM Re: 4.5 alpha
Radimus Moderator Offline
Moderator
*****

Registered: 2000-01-06
Posts: 5187
Loc: Tampa, FL
Here is my 2 cents...

Since it would seem to be impossible (or just really difficult) to encrypt the tokenized file for the really hard core security guys, don't try.. save that for a compiler version

I don't know how much detail Ruud put into the tokenizer routine... but he makes enough efforts to call it a tokenizer, and not a compiler

If possible, make a detokenizer for reading a tokenized file. Stripped comments are fine, varibles names aren't that big of a deal (find/replace)

However, Ruud is not resposible if your shop loses your source code... that is your problem.

I really like the make-exe concept
_________________________
How to ask questions the smart way <-----------> Before you ask

Top
#110865 - 2004-01-02 04:44 PM Re: 4.5 alpha
ShaneEP Moderator Offline
MM club member
*****

Registered: 2002-11-29
Posts: 2125
Loc: Tulsa, OK
I dont see any reason yet that would convince me to want to be able to un-tokenize a script. All the problems that are being pointed out would still exist if tokenized was not in the picture at all.

If your login scripts accidentally get deleted and there are no tokenized scripts you are still in the same boat arent you? I dont think the purpose for the tokenized scripts are in any way a means of backup.

And I would hope that any scripts or scheduled tasks that suddenly and mysteriously appeared on servers/clients around the same time an angry ex-employee leaves would be removed whether you were able to read them or not.

I would only opt for the de-tokenizing if there was some kind of unique password protection or something like that. The whole point of it for me would be so that other admins/users couldn't get in there and "improve" login scripts that they did not write. None of our scripts are so big that the comporession aspect plays a big role. Its mostly the obfuscation that Im looking forward to. And if anyone simply has to find the correct command line switch to convert it back to clear text then I dont see any point in the obfuscation to begin with.

Just my 2 cents on the subject.

Top
#110866 - 2004-01-05 11:39 AM Re: 4.5 alpha
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
Quote:

I have run some experiments to see if the var names are tokenized or not. I thought they too would have been tokenized to really optimize the code but if I use longer var names, the tokenized file is larger. I think var names should also be converted to tokens for best performance.




Variable names must be maintained.

Consider two scenarios where tokenising the variable name will fail:
  1. When calling an external script.
    Variables which are shared between the scripts are unlikely to have the same "tokenised" names so data cannot be passed. Worse, unrelated variables are likely to have the same tokenised names, so will be passed/overwritten accidentally.
  2. Executed code.
    Consider this somewhat contrived example:
    Code:
    $foo="bar"
    $bar="test"
    $=Execute('"The value of $$$$'+$foo+' is "+$$'+$foo+' ?')


    This will never work if the variable names are tokenised.

Top
#110867 - 2004-01-08 02:31 PM Re: 4.5 alpha
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22346
Loc: OK
rich, nope...
have to disagree on that one.

example tokenization of:
$foo="bar"
$bar="test"
$=Execute('"The value of $$$$'+$foo+' is "+$$'+$foo+' ?')

should lead to something like:
$1="bar"
$2="test"
$3=Execute('"The value of $$$$'+$1+' is "+$$'+$1+' ?')


what comes to called scripts, if they are to share same namespace, they will have same names.
if they are executed via run or shell, their namespace is own and then it does have no effect.
know this from my expirements with kixtart-compressor.
_________________________
!

download KiXnet

Top
Page 1 of 3 123>


Moderator:  ShaneEP, Arend_, Jochen, Radimus, Glenn Barnas, Allen, Ruud van Velsen, Mart 
Hop to:
Shout Box

Who's Online
0 registered and 507 anonymous users online.
Newest Members
gespanntleuchten, DaveatAdvanced, Paulo_Alves, UsTaaa, xxJJxx
17864 Registered Users

Generated in 0.078 seconds in which 0.029 seconds were spent on a total of 13 queries. Zlib compression enabled.