#206767 - 2013-02-23 02:36 PM
ASCII Chars 128 to 255 can stretch a string by more than one character
|
applitude
Fresh Scripter
Registered: 2013-02-14
Posts: 18
Loc: Cologne, Germany
|
While working on a RC4 based encryption function, i was faced with a strange problem. FYI, i'm not familar with encrytion/decryption, in fact the only encryption function i really understand is rot13. But for such rookies as i am, the www and Google was made 
To implement a RC4 function you mostly need the operators XOR (^) and Mod, as well as the functions Asc(), Chr() and Len(). Because i never get the correct encryption result, i decided to do some testing on Chr() and Len()
dim $OutString, $i
for $i=128 to 143 ; -> 16 Chars
$OutString = $OutString + chr($i)
next
; Big Surprise
messagebox("Length of should be 16 but it is " + len($Outstring), "String Length")
; Now we split the loop to detect where the additional char is hiding
dim $OutString1, $OutString2, $OutString3, $OutString4
for $i=128 to 131 ; -> 4 Chars
$OutString1 = $OutString1 + chr($i)
next
for $i=132 to 135; -> 4 Chars
$OutString2 = $OutString3 + chr($i)
next
for $i=136 to 139; -> 4 Chars
$OutString3 = $OutString3 + chr($i)
next
for $i=140 to 143; -> 4 Chars
$OutString4 = $OutString4 + chr($i)
next
; Bigger Surprise :-)
messagebox("Length of $$Outstring should be 16 but it is " + len($Outstring) + @crlf +
"Length of $$Outstring1 should be 4 but it is " + len($Outstring1) + @crlf +
"Length of $$Outstring2 should be 4 but it is " + len($Outstring2) + @crlf +
"Length of $$Outstring3 should be 4 but it is " + len($Outstring3) + @crlf +
"Length of $$Outstring4 should be 4 but it is " + len($Outstring4), "String Length")
; Let's take a look on the content of $Outstring
for $i=1 to len($Outstring)
messagebox("This is the Asc-Value of Char #" + $i + " inside $$OutString: " + Asc(substr($Outstring, $i, 1)) + " (should be " + cstr($i + 127) + ")", "Asc()")
next
Ok, i'll never get the encryption function to work correctly, if there is no "1 to 1"-relation between Asc() and Chr() and the length of the inputstring equals the length of the outputstring
Any ideas? -- Andre
_________________________
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ( Bertrand Russell)
|
Top
|
|
|
|
#206769 - 2013-02-23 05:05 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: ShaneEP]
|
ShaneEP
MM club member
   
Registered: 2002-11-29
Posts: 2127
Loc: Tulsa, OK
|
For $i=65 to 90
$OutString = $OutString + chr($i)
Next
$nul = messagebox("Length of should be 26 and it is " + len($Outstring), "String Length")
|
Top
|
|
|
|
#206770 - 2013-02-23 07:30 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: ShaneEP]
|
applitude
Fresh Scripter
Registered: 2013-02-14
Posts: 18
Loc: Cologne, Germany
|
Yes, the bug (???) applies only to characters up to 127. But meanwhile i've done some more testing. And it becomes more strange Take a look at this:messagebox(VBChr(158), "VBChr(158)")
messagebox(Chr(158), "Chr(158)")
; Putting the Chr(158) into two strings
$OutputStr1 = "A" + Chr(158) + "B"
$OutputStr2 = Chr(158) + Chr(158) + Chr(158)
messagebox($OutputStr1, "$OutputStr1")
; Now, the Chr(158) is shown as the same as VBChr
messagebox($OutputStr2, "$OutputStr2")
Function VBChr($i_Code)
Dim $sc
$sc = CreateObject("ScriptControl")
$sc.Language = "VBScript"
$VBChr=$sc.Eval('chr($i_Code)')
EndFunction
_________________________
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ( Bertrand Russell)
|
Top
|
|
|
|
#206772 - 2013-02-23 10:19 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: applitude]
|
Allen
KiX Supporter
   
Registered: 2003-04-19
Posts: 4557
Loc: USA
|
? chr(158)
$RC=setoption("Ascii","on")
? chr(158)
? VBChr(158)
You also get different results when displaying to the console. However, it seems to me this has been discussed in the past, and I could be wrong, but I don't think it was considered a bug. However, I have to say it does seem odd.
|
Top
|
|
|
|
#206774 - 2013-02-24 10:08 AM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: Lonkero]
|
applitude
Fresh Scripter
Registered: 2013-02-14
Posts: 18
Loc: Cologne, Germany
|
I already thought, that different codepages have something to do with it. And as far as i can see, the characters 128 up to 159 are the problem in KiXtart. The Windows-1252 codepage coincides with ISO-8859-1 for all codes except the range 128 to 159 (hex 80 to 9F), where the little-used C1 controls are replaced with additional characters including all the missing characters provided by ISO-8859-15. But IMHO this does not really explains, why the characters are interpreted in different ways, depending on which other character is followed.
Andre
_________________________
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ( Bertrand Russell)
|
Top
|
|
|
|
#206779 - 2013-02-24 05:22 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: Lonkero]
|
applitude
Fresh Scripter
Registered: 2013-02-14
Posts: 18
Loc: Cologne, Germany
|
That's right, the function call asc(chr($i)) always returns the correct value $i. The problem comes up only if you concatenate strings. Some characters will stretch, others will shorten the resulting string as you can see in this code snippetdim $OutString1, $OutString2, $OutString3, $OutString4, $i
for $i=128 to 131 ; -> 4 Chars
$OutString1 = $OutString1 + chr($i)
next
for $i=132 to 135; -> 4 Chars
$OutString2 = $OutString3 + chr($i)
next
for $i=136 to 139; -> 4 Chars
$OutString3 = $OutString3 + chr($i)
next
for $i=140 to 143; -> 4 Chars
$OutString4 = $OutString4 + chr($i)
next
messagebox("Length of $$Outstring1 should be 4 but it is " + len($Outstring1) + @crlf +
"Length of $$Outstring2 should be 4 but it is " + len($Outstring2) + @crlf +
"Length of $$Outstring3 should be 4 but it is " + len($Outstring3) + @crlf +
"Length of $$Outstring4 should be 4 but it is " + len($Outstring4), "String Length") Let's take a look at your first example, but now we only examine the chars 128 to 159$cnt=0
For $i=128 to 158
$cnt=$cnt+1
$OutString = $OutString + chr($i)
Next
$nul = messagebox("Length of should be $cnt and it is " + len($Outstring), "String Length")
_________________________
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ( Bertrand Russell)
|
Top
|
|
|
|
#206780 - 2013-02-24 06:02 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: applitude]
|
Glenn Barnas
KiX Supporter
   
Registered: 2003-01-28
Posts: 4400
Loc: New Jersey
|
Andre,
Please download and test your strings with my cipher UDF. http://www.innotechcg.com/downloads/cipher.zip
It's a "closed source" UDF to protect the algorithms, so be sure to use either the 4.53 or 4.61 versions depending on whether you use the 4.5x or 4.6x Kix executable. If the logic returns the correct data each time, I can share the character translation part of the logic with you.
The cipher UDF is based on the German Enigma technology of WWII but mixed with some methods from early Mission:Impossible episodes and applied to computer technology. You can specify between 1 and 32768 "codewheels". The 254 possible character codes are "wrapped" around each of the wheels. (Note that zero isn't possible in Kix, so only 254 of the 255 possible codes are usable.) The characters on the wheels are aligned to the source text, then each is rotated an amount between 0 and 254 (positive or negative) based on an internal algorithm. Thus, "cat" could conceivably encipher to "rrr", making decoding based on character frequency impossible. With a single codewheel, you get a traditional Caesar code, where one character directly translates to another - easy to demonstrate but very low security (similar to ROT13). The use of more than 32 (and up to 32K) codewheels makes it extremely difficult to crack.
Anyway - this UDF had been tested and published a few years ago. The tests compared the original content and size with that which was enciphered and then deciphered. The enciphered data was the same byte-count but in display appears to have a different number of characters when non-ASCII and ASCII-Control codes are employed. I don't remember how many languages were tested, but I know it was more than just English.
Glenn
_________________________
Actually I am a Rocket Scientist!
|
Top
|
|
|
|
#206781 - 2013-02-24 06:30 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: Glenn Barnas]
|
Glenn Barnas
KiX Supporter
   
Registered: 2003-01-28
Posts: 4400
Loc: New Jersey
|
Hmm.. this might well be a bug! Try this:Dim $OutString, $Cc, $Sv, $C, $Lc
For $Sv = 128 to 143 Step 4
$OutString = ''
For $Cc = $Sv to $Sv + 3 ; -> 4 Chars
$C = Chr($Cc)
$Lc = Len($C)
$I ' length: ' $Lc ?
If $Lc > 1
'Expecting ' $Cc ' but got ' Asc(Left($C, 1)) ' / ' Asc(Right($C, 1))
EndIf
$OutString = $OutString + $C
Next
Next a small variation on your code. I did this to expand the intermediate steps into specific variables that could be examined.
When I run this using Kix32 v4.62, I get (ICWD011) - H:\>x
length: 2
Expecting 128 but got 172 / 32 length: 1
length: 2
Expecting 130 but got 26 / 32 length: 2
Expecting 131 but got 146 / 1 length: 2
Expecting 132 but got 30 / 32 length: 2
Expecting 133 but got 38 / 32 length: 2
Expecting 134 but got 32 / 32 length: 2
Expecting 135 but got 33 / 32 length: 2
Expecting 136 but got 198 / 2 length: 2
Expecting 137 but got 48 / 32 length: 2
Expecting 138 but got 96 / 1 length: 2
Expecting 139 but got 57 / 32 length: 2
Expecting 140 but got 82 / 1 length: 1
length: 2
Expecting 142 but got 125 / 1 length: 1 Not really sure WHAT that is. But, if I run the following(ICWD011) - H:\>kix32_4.53 x.kix
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1
length: 1 it works as expected. Clearly, it is not working the same on 4.53 as 4.6x. I tested this on 4.61, 4.62, and 4.63 and all return the same results. 4.53 seems happy.
Andre, the best thing to do is put the sample into the Beta section, where Ruud will spot it and either provide an answer or fix the problem. You found it so you should make the post.
Glenn
_________________________
Actually I am a Rocket Scientist!
|
Top
|
|
|
|
#206782 - 2013-02-24 06:50 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: Lonkero]
|
applitude
Fresh Scripter
Registered: 2013-02-14
Posts: 18
Loc: Cologne, Germany
|
there is something funky about the control characters[...]making $OutString just an empty string does not fix the behavior, it actually needs to contain ascii val. in this case the letter a. This may help me! It's maybe possible to prefix the string before encryption with an ascii-char and then after the string encryption to subtract the needless characters. I'll give it a try! Thanks a lot! -- Andre
_________________________
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ( Bertrand Russell)
|
Top
|
|
|
|
#206783 - 2013-02-24 08:28 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: Glenn Barnas]
|
applitude
Fresh Scripter
Registered: 2013-02-14
Posts: 18
Loc: Cologne, Germany
|
Hmm.. this might well be a bug! Glenn, you're my hero!  I've already thought i'm going mad! Your posts are detailed and always helpful to me! Andre, the best thing to do is put the sample into the Beta section, where Ruud will spot it and either provide an answer or fix the problem. You found it so you should make the post. To much honor to me - I'm a newbie "Fresh Scripter"  I think you've detected and confirmed this bug. I've only an idea that "maybe" something is wrong. Also it's always a pain for me to write in English . IMO, it would be the easiest, if an Admin move this whole thread into the Beta section. The cipher UDF is based on the German Enigma technology of WWII but mixed with some methods from early Mission:Impossible episodes and applied to computer technology. Thanks for this UDF! I'll test it! But in fact i really like to implement an RC4 encryption. It's because i'm writing a lot of - especially GUI-based - programs in AutoIt3, where an UDF exists which do this. So i will be able to read/write the encrypted strings from my KiX-Scripts as well. And, as a german, i do not really like Enigma based encryption: Hey, this was cracked by Turing and others from Bletchley Park (Just kidding!) For those of you, who are interested in the RC4 AutoIt Function (it's really similar to KiXtart!) here's the code:Func _StringEncrypt($i_Encrypt, $s_EncryptText, $s_EncryptPassword, $i_EncryptLevel = 1)
If $i_Encrypt <> 0 And $i_Encrypt <> 1 Then
SetError(1, 0, '')
ElseIf $s_EncryptText = '' Or $s_EncryptPassword = '' Then
SetError(1, 0, '')
Else
If Number($i_EncryptLevel) <= 0 Or Int($i_EncryptLevel) <> $i_EncryptLevel Then $i_EncryptLevel = 1
Local $v_EncryptModified
Local $i_EncryptCountH
Local $i_EncryptCountG
Local $v_EncryptSwap
Local $av_EncryptBox[256][2]
Local $i_EncryptCountA
Local $i_EncryptCountB
Local $i_EncryptCountC
Local $i_EncryptCountD
Local $i_EncryptCountE
Local $v_EncryptCipher
Local $v_EncryptCipherBy
If $i_Encrypt = 1 Then
For $i_EncryptCountF = 0 To $i_EncryptLevel Step 1
$i_EncryptCountG = ''
$i_EncryptCountH = ''
$v_EncryptModified = ''
For $i_EncryptCountG = 1 To StringLen($s_EncryptText)
If $i_EncryptCountH = StringLen($s_EncryptPassword) Then
$i_EncryptCountH = 1
Else
$i_EncryptCountH += 1
EndIf
$v_EncryptModified = $v_EncryptModified & Chr(BitXOR(Asc(StringMid($s_EncryptText, $i_EncryptCountG, 1)), Asc(StringMid($s_EncryptPassword, $i_EncryptCountH, 1)), 255))
Next
$s_EncryptText = $v_EncryptModified
$i_EncryptCountA = ''
$i_EncryptCountB = 0
$i_EncryptCountC = ''
$i_EncryptCountD = ''
$i_EncryptCountE = ''
$v_EncryptCipherBy = ''
$v_EncryptCipher = ''
$v_EncryptSwap = ''
$av_EncryptBox = ''
Local $av_EncryptBox[256][2]
For $i_EncryptCountA = 0 To 255
$av_EncryptBox[$i_EncryptCountA][1] = Asc(StringMid($s_EncryptPassword, Mod($i_EncryptCountA, StringLen($s_EncryptPassword)) + 1, 1))
$av_EncryptBox[$i_EncryptCountA][0] = $i_EncryptCountA
Next
For $i_EncryptCountA = 0 To 255
$i_EncryptCountB = Mod(($i_EncryptCountB + $av_EncryptBox[$i_EncryptCountA][0] + $av_EncryptBox[$i_EncryptCountA][1]), 256)
$v_EncryptSwap = $av_EncryptBox[$i_EncryptCountA][0]
$av_EncryptBox[$i_EncryptCountA][0] = $av_EncryptBox[$i_EncryptCountB][0]
$av_EncryptBox[$i_EncryptCountB][0] = $v_EncryptSwap
Next
For $i_EncryptCountA = 1 To StringLen($s_EncryptText)
$i_EncryptCountC = Mod(($i_EncryptCountC + 1), 256)
$i_EncryptCountD = Mod(($i_EncryptCountD + $av_EncryptBox[$i_EncryptCountC][0]), 256)
$i_EncryptCountE = $av_EncryptBox[Mod(($av_EncryptBox[$i_EncryptCountC][0] + $av_EncryptBox[$i_EncryptCountD][0]), 256)][0]
$v_EncryptCipherBy = BitXOR(Asc(StringMid($s_EncryptText, $i_EncryptCountA, 1)), $i_EncryptCountE)
$v_EncryptCipher &= Hex($v_EncryptCipherBy, 2)
Next
$s_EncryptText = $v_EncryptCipher
Next
Else
For $i_EncryptCountF = 0 To $i_EncryptLevel Step 1
$i_EncryptCountB = 0
$i_EncryptCountC = ''
$i_EncryptCountD = ''
$i_EncryptCountE = ''
$v_EncryptCipherBy = ''
$v_EncryptCipher = ''
$v_EncryptSwap = ''
$av_EncryptBox = ''
Local $av_EncryptBox[256][2]
For $i_EncryptCountA = 0 To 255
$av_EncryptBox[$i_EncryptCountA][1] = Asc(StringMid($s_EncryptPassword, Mod($i_EncryptCountA, StringLen($s_EncryptPassword)) + 1, 1))
$av_EncryptBox[$i_EncryptCountA][0] = $i_EncryptCountA
Next
For $i_EncryptCountA = 0 To 255
$i_EncryptCountB = Mod(($i_EncryptCountB + $av_EncryptBox[$i_EncryptCountA][0] + $av_EncryptBox[$i_EncryptCountA][1]), 256)
$v_EncryptSwap = $av_EncryptBox[$i_EncryptCountA][0]
$av_EncryptBox[$i_EncryptCountA][0] = $av_EncryptBox[$i_EncryptCountB][0]
$av_EncryptBox[$i_EncryptCountB][0] = $v_EncryptSwap
Next
For $i_EncryptCountA = 1 To StringLen($s_EncryptText) Step 2
$i_EncryptCountC = Mod(($i_EncryptCountC + 1), 256)
$i_EncryptCountD = Mod(($i_EncryptCountD + $av_EncryptBox[$i_EncryptCountC][0]), 256)
$i_EncryptCountE = $av_EncryptBox[Mod(($av_EncryptBox[$i_EncryptCountC][0] + $av_EncryptBox[$i_EncryptCountD][0]), 256)][0]
$v_EncryptCipherBy = BitXOR(Dec(StringMid($s_EncryptText, $i_EncryptCountA, 2)), $i_EncryptCountE)
$v_EncryptCipher = $v_EncryptCipher & Chr($v_EncryptCipherBy)
Next
$s_EncryptText = $v_EncryptCipher
$i_EncryptCountG = ''
$i_EncryptCountH = ''
$v_EncryptModified = ''
For $i_EncryptCountG = 1 To StringLen($s_EncryptText)
If $i_EncryptCountH = StringLen($s_EncryptPassword) Then
$i_EncryptCountH = 1
Else
$i_EncryptCountH += 1
EndIf
$v_EncryptModified &= Chr(BitXOR(Asc(StringMid($s_EncryptText, $i_EncryptCountG, 1)), Asc(StringMid($s_EncryptPassword, $i_EncryptCountH, 1)), 255))
Next
$s_EncryptText = $v_EncryptModified
Next
EndIf
Return $s_EncryptText
EndIf
EndFunc ;==>_StringEncrypt Again, Thanks a lot to all of you who helped me! -- Andre
_________________________
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ( Bertrand Russell)
|
Top
|
|
|
|
#206787 - 2013-02-24 09:02 PM
Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re: Lonkero]
|
Lonkero
KiX Master Guru
   
Registered: 2001-06-05
Posts: 22346
Loc: OK
|
here is basic example with chr(158):
"if a string contains 158, it's ok:"
$string=chr(158)+"word"
? 'len of chr(158)+"word":' len($string) ?
? 'substr(that,2):' substr($string,2) ?
? 'len of substr(chr(158)+"word",2):' len(substr($string,2)) ?
? "but if it is initialised with it alone, it's BROKEN:"
$string=chr(158)
? 'len of chr(158):' len($string) ?
? 'substr(that,2):' substr($string,2) ?
? 'len of substr(chr(158),2):' len(substr($string,2)) ?
?
get $
with 4.62
_________________________
!download KiXnet
|
Top
|
|
|
|
Moderator: ShaneEP, Arend_, Jochen, Radimus, Glenn Barnas, Allen, Ruud van Velsen, Mart
|
0 registered
and 581 anonymous users online.
|
|
|