| 
| 
| 
| #206767 - 2013-02-23 02:36 PM  ASCII Chars 128 to 255 can stretch a string by more than one character |  
| applitude   Fresh Scripter
 
   Registered:  2013-02-14
 Posts: 18
 Loc:  Cologne, Germany
 | 
While working on a RC4 based encryption function, i was faced with a strange problem. FYI, i'm not familar with encrytion/decryption, in fact the only encryption function i really understand is rot13. But for such rookies as i am, the www and Google was made    
 To implement a RC4 function you mostly need the operators XOR (^) and Mod, as well as the functions Asc(), Chr() and Len(). Because i never get the correct encryption result, i decided to do some testing on Chr() and Len()
 
 Ok, i'll never get the encryption function to work correctly, if there is no "1 to 1"-relation between Asc() and Chr() and the length of the inputstring equals the length of the outputstringdim $OutString, $i
for $i=128 to 143  ; -> 16 Chars
  $OutString = $OutString + chr($i)
next
; Big Surprise
messagebox("Length of should be 16 but it is " + len($Outstring), "String Length")
; Now we split the loop to detect where the additional char is hiding
dim $OutString1, $OutString2, $OutString3, $OutString4
for $i=128 to 131 ; -> 4 Chars
  $OutString1 = $OutString1 + chr($i)
next
for $i=132 to 135; -> 4 Chars
  $OutString2 = $OutString3 + chr($i)
next
for $i=136 to 139; -> 4 Chars
  $OutString3 = $OutString3 + chr($i)
next
for $i=140 to 143; -> 4 Chars
  $OutString4 = $OutString4 + chr($i)
next
; Bigger Surprise :-)
messagebox("Length of $$Outstring should be 16 but it is " + len($Outstring) + @crlf +
           "Length of $$Outstring1 should be 4 but it is " + len($Outstring1) + @crlf +
           "Length of $$Outstring2 should be 4 but it is " + len($Outstring2) + @crlf +
           "Length of $$Outstring3 should be 4 but it is " + len($Outstring3) + @crlf +
           "Length of $$Outstring4 should be 4 but it is " + len($Outstring4), "String Length")
; Let's take a look on the content of $Outstring
for $i=1 to len($Outstring)
  messagebox("This is the Asc-Value of Char #" + $i + " inside $$OutString: " + Asc(substr($Outstring, $i, 1)) + " (should be " + cstr($i + 127) + ")", "Asc()")
next
 Any ideas?
 --
 Andre
 
 
 
_________________________ 
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. (Bertrand Russell )
 |  
| Top |  |  |  |  
| 
| 
| #206769 - 2013-02-23 05:05 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  ShaneEP] |  
| ShaneEP   MM club member
 
       
   Registered:  2002-11-29
 Posts: 2127
 Loc:  Tulsa, OK
 | 
For $i=65 to 90
   $OutString = $OutString + chr($i)
Next
$nul = messagebox("Length of should be 26 and it is " + len($Outstring), "String Length") |  
| Top |  |  |  |  
| 
| 
| #206770 - 2013-02-23 07:30 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  ShaneEP] |  
| applitude   Fresh Scripter
 
   Registered:  2013-02-14
 Posts: 18
 Loc:  Cologne, Germany
 | 
Yes, the bug (???) applies only to characters up to 127. But meanwhile i've done some more testing. And it becomes more strange  Take a look at this: messagebox(VBChr(158), "VBChr(158)")
messagebox(Chr(158), "Chr(158)")
; Putting the Chr(158) into two strings
$OutputStr1 = "A" + Chr(158) + "B"
$OutputStr2 = Chr(158) + Chr(158) + Chr(158)
messagebox($OutputStr1, "$OutputStr1")
; Now, the Chr(158) is shown as the same as VBChr
messagebox($OutputStr2, "$OutputStr2")
Function VBChr($i_Code)
  Dim $sc
  $sc = CreateObject("ScriptControl")
  $sc.Language = "VBScript"
  $VBChr=$sc.Eval('chr($i_Code)')
EndFunction
_________________________ 
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. (Bertrand Russell )
 |  
| Top |  |  |  |  
| 
| 
| #206772 - 2013-02-23 10:19 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  applitude] |  
| Allen   KiX Supporter
 
       
 Registered:  2003-04-19
 Posts: 4562
 Loc:  USA
 | 
? chr(158)
$RC=setoption("Ascii","on")
? chr(158)
? VBChr(158)
 You also get different results when displaying to the console.  However, it seems to me this has been discussed in the past, and I could be wrong, but I don't think it was considered a bug.  However, I have to say it does seem odd.
 |  
| Top |  |  |  |  
| 
| 
| #206774 - 2013-02-24 10:08 AM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  Lonkero] |  
| applitude   Fresh Scripter
 
   Registered:  2013-02-14
 Posts: 18
 Loc:  Cologne, Germany
 | 
I already thought, that different codepages have something to do with it. And as far as i can see, the characters 128 up to 159 are the problem in KiXtart.  But IMHO this does not really explains, why the characters are interpreted in different ways, depending on which other character is followed.The Windows-1252 codepage coincides with ISO-8859-1 for all codes except the range 128 to 159 (hex 80 to 9F), where the little-used C1 controls are replaced with additional characters including all the missing characters provided by ISO-8859-15.
 Andre
 
_________________________ 
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. (Bertrand Russell )
 |  
| Top |  |  |  |  
| 
| 
| #206779 - 2013-02-24 05:22 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  Lonkero] |  
| applitude   Fresh Scripter
 
   Registered:  2013-02-14
 Posts: 18
 Loc:  Cologne, Germany
 | 
That's right, the function call asc(chr($i)) always returns the correct value $i. The problem comes up only if you concatenate strings. Some characters will stretch, others will shorten the resulting string as you can see in this code snippetLet's take a look at your first example, but now we only examine the chars 128 to 159dim $OutString1, $OutString2, $OutString3, $OutString4, $i
for $i=128 to 131 ; -> 4 Chars
  $OutString1 = $OutString1 + chr($i)
next
for $i=132 to 135; -> 4 Chars
  $OutString2 = $OutString3 + chr($i)
next
for $i=136 to 139; -> 4 Chars
  $OutString3 = $OutString3 + chr($i)
next
for $i=140 to 143; -> 4 Chars
  $OutString4 = $OutString4 + chr($i)
next
messagebox("Length of $$Outstring1 should be 4 but it is " + len($Outstring1) + @crlf +
           "Length of $$Outstring2 should be 4 but it is " + len($Outstring2) + @crlf +
           "Length of $$Outstring3 should be 4 but it is " + len($Outstring3) + @crlf +
           "Length of $$Outstring4 should be 4 but it is " + len($Outstring4), "String Length")$cnt=0
For $i=128 to 158
   $cnt=$cnt+1
   $OutString = $OutString + chr($i)
Next
$nul = messagebox("Length of should be $cnt and it is " + len($Outstring), "String Length")
_________________________ 
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. (Bertrand Russell )
 |  
| Top |  |  |  |  
| 
| 
| #206780 - 2013-02-24 06:02 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  applitude] |  
| Glenn Barnas   KiX Supporter
 
       
   Registered:  2003-01-28
 Posts: 4401
 Loc:  New Jersey
 | 
Andre,
 Please download and test your strings with my cipher UDF. http://www.innotechcg.com/downloads/cipher.zip
 
 It's a "closed source" UDF to protect the algorithms, so be sure to use either the 4.53 or 4.61 versions depending on whether you use the 4.5x or 4.6x Kix executable. If the logic returns the correct data each time, I can share the character translation part of the logic with you.
 
 The cipher UDF is based on the German Enigma technology of WWII but mixed with some methods from early Mission:Impossible episodes and applied to computer technology. You can specify  between 1 and 32768 "codewheels". The 254 possible character codes are "wrapped" around each of the wheels. (Note that zero isn't possible in Kix, so only 254 of the 255 possible codes are usable.) The characters on the wheels are aligned to the source text, then each is rotated an amount between 0 and 254 (positive or negative) based on an internal algorithm. Thus, "cat" could conceivably encipher to "rrr", making decoding based on character frequency impossible. With a single codewheel, you get a traditional Caesar code, where one character directly translates to another - easy to demonstrate but very low security (similar to ROT13). The use of more than 32 (and up to 32K) codewheels makes it extremely difficult to crack.
 
 Anyway - this UDF had been tested and published a few years ago. The tests compared the original content and size with that which was enciphered and then deciphered. The enciphered data was the same byte-count but in display appears to have a different number of characters when non-ASCII and ASCII-Control codes are employed. I don't remember how many languages were tested, but I know it was more than just English.
 
 Glenn
 
_________________________Actually  I am  a Rocket Scientist!    |  
| Top |  |  |  |  
| 
| 
| #206781 - 2013-02-24 06:30 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  Glenn Barnas] |  
| Glenn Barnas   KiX Supporter
 
       
   Registered:  2003-01-28
 Posts: 4401
 Loc:  New Jersey
 | 
Hmm.. this might well be a bug! Try this:a small variation on your code. I did this to expand the intermediate steps into specific variables that could be examined.Dim $OutString, $Cc, $Sv, $C, $Lc
For $Sv = 128 to 143 Step 4
  $OutString = ''
  For $Cc = $Sv to $Sv + 3 ; -> 4 Chars
    $C = Chr($Cc)
    $Lc = Len($C)
    $I ' length: ' $Lc ?
    If $Lc > 1
      'Expecting ' $Cc ' but got ' Asc(Left($C, 1)) ' / ' Asc(Right($C, 1))      
    EndIf
    $OutString = $OutString + $C
  Next
Next
 When I run this using Kix32 v4.62, I get
 Not really sure WHAT that is. But, if I run the following(ICWD011) - H:\>x
 length: 2
Expecting 128 but got 172 / 32 length: 1
 length: 2
Expecting 130 but got 26 / 32 length: 2
Expecting 131 but got 146 / 1 length: 2
Expecting 132 but got 30 / 32 length: 2
Expecting 133 but got 38 / 32 length: 2
Expecting 134 but got 32 / 32 length: 2
Expecting 135 but got 33 / 32 length: 2
Expecting 136 but got 198 / 2 length: 2
Expecting 137 but got 48 / 32 length: 2
Expecting 138 but got 96 / 1 length: 2
Expecting 139 but got 57 / 32 length: 2
Expecting 140 but got 82 / 1 length: 1
 length: 2
Expecting 142 but got 125 / 1 length: 1it works as expected. Clearly, it is not working the same on 4.53 as 4.6x. I tested this on 4.61, 4.62, and 4.63 and all return the same results. 4.53 seems happy.(ICWD011) - H:\>kix32_4.53 x.kix
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 length: 1
 Andre, the best thing to do is put the sample into the Beta section, where Ruud will spot it and either provide an answer or fix the problem. You found it so you should make the post.
 
 Glenn
 
_________________________Actually  I am  a Rocket Scientist!    |  
| Top |  |  |  |  
| 
| 
| #206782 - 2013-02-24 06:50 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  Lonkero] |  
| applitude   Fresh Scripter
 
   Registered:  2013-02-14
 Posts: 18
 Loc:  Cologne, Germany
 | 
 This may help me! It's maybe possible to prefix the string before encryption with an ascii-char and then after the string encryption to subtract the needless characters.there is something funky about the control characters[...]making $OutString just an empty string does not fix the behavior, it actually needs to contain ascii val. in this case the letter a.I'll give it a try! Thanks a lot!
 --
 Andre
 
_________________________ 
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. (Bertrand Russell )
 |  
| Top |  |  |  |  
| 
| 
| #206783 - 2013-02-24 08:28 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  Glenn Barnas] |  
| applitude   Fresh Scripter
 
   Registered:  2013-02-14
 Posts: 18
 Loc:  Cologne, Germany
 | 
 Glenn, you're my hero!Hmm.. this might well be a bug!  I've already thought i'm going mad! Your posts are detailed and always helpful to me!
 To much honor to me - I'm a newbie "Fresh Scripter"Andre, the best thing to do is put the sample into the Beta section, where Ruud will spot it and either provide an answer or fix the problem. You found it so you should make the post.  I think you've detected and confirmed this bug. I've only an idea that "maybe" something is wrong. Also it's always a pain for me to write in English
  . IMO, it would be the easiest, if an Admin move this whole thread into the Beta section.
 Thanks for this UDF! I'll test it! But in fact i really like to implement an RC4 encryption. It's because i'm writing a lot of - especially GUI-based - programs in   AutoIt3, where an UDF exists which do this. So i will be able to read/write the encrypted strings from my KiX-Scripts as well. And, as a german, i do not really like Enigma based encryption: Hey, this was cracked by Turing and others from Bletchley Park (Just kidding!)The cipher UDF is based on the German Enigma technology of WWII but mixed with some methods from early Mission:Impossible episodes and applied to computer technology.For those of you, who are interested in the RC4 AutoIt Function (it's really similar to KiXtart!) here's the code:
 Func _StringEncrypt($i_Encrypt, $s_EncryptText, $s_EncryptPassword, $i_EncryptLevel = 1)
	If $i_Encrypt <> 0 And $i_Encrypt <> 1 Then
		SetError(1, 0, '')
	ElseIf $s_EncryptText = '' Or $s_EncryptPassword = '' Then
		SetError(1, 0, '')
	Else
		If Number($i_EncryptLevel) <= 0 Or Int($i_EncryptLevel) <> $i_EncryptLevel Then $i_EncryptLevel = 1
		Local $v_EncryptModified
		Local $i_EncryptCountH
		Local $i_EncryptCountG
		Local $v_EncryptSwap
		Local $av_EncryptBox[256][2]
		Local $i_EncryptCountA
		Local $i_EncryptCountB
		Local $i_EncryptCountC
		Local $i_EncryptCountD
		Local $i_EncryptCountE
		Local $v_EncryptCipher
		Local $v_EncryptCipherBy
		If $i_Encrypt = 1 Then
			For $i_EncryptCountF = 0 To $i_EncryptLevel Step 1
				$i_EncryptCountG = ''
				$i_EncryptCountH = ''
				$v_EncryptModified = ''
				For $i_EncryptCountG = 1 To StringLen($s_EncryptText)
					If $i_EncryptCountH = StringLen($s_EncryptPassword) Then
						$i_EncryptCountH = 1
					Else
						$i_EncryptCountH += 1
					EndIf
					$v_EncryptModified = $v_EncryptModified & Chr(BitXOR(Asc(StringMid($s_EncryptText, $i_EncryptCountG, 1)), Asc(StringMid($s_EncryptPassword, $i_EncryptCountH, 1)), 255))
				Next
				$s_EncryptText = $v_EncryptModified
				$i_EncryptCountA = ''
				$i_EncryptCountB = 0
				$i_EncryptCountC = ''
				$i_EncryptCountD = ''
				$i_EncryptCountE = ''
				$v_EncryptCipherBy = ''
				$v_EncryptCipher = ''
				$v_EncryptSwap = ''
				$av_EncryptBox = ''
				Local $av_EncryptBox[256][2]
				For $i_EncryptCountA = 0 To 255
					$av_EncryptBox[$i_EncryptCountA][1] = Asc(StringMid($s_EncryptPassword, Mod($i_EncryptCountA, StringLen($s_EncryptPassword)) + 1, 1))
					$av_EncryptBox[$i_EncryptCountA][0] = $i_EncryptCountA
				Next
				For $i_EncryptCountA = 0 To 255
					$i_EncryptCountB = Mod(($i_EncryptCountB + $av_EncryptBox[$i_EncryptCountA][0] + $av_EncryptBox[$i_EncryptCountA][1]), 256)
					$v_EncryptSwap = $av_EncryptBox[$i_EncryptCountA][0]
					$av_EncryptBox[$i_EncryptCountA][0] = $av_EncryptBox[$i_EncryptCountB][0]
					$av_EncryptBox[$i_EncryptCountB][0] = $v_EncryptSwap
				Next
				For $i_EncryptCountA = 1 To StringLen($s_EncryptText)
					$i_EncryptCountC = Mod(($i_EncryptCountC + 1), 256)
					$i_EncryptCountD = Mod(($i_EncryptCountD + $av_EncryptBox[$i_EncryptCountC][0]), 256)
					$i_EncryptCountE = $av_EncryptBox[Mod(($av_EncryptBox[$i_EncryptCountC][0] + $av_EncryptBox[$i_EncryptCountD][0]), 256)][0]
					$v_EncryptCipherBy = BitXOR(Asc(StringMid($s_EncryptText, $i_EncryptCountA, 1)), $i_EncryptCountE)
					$v_EncryptCipher &= Hex($v_EncryptCipherBy, 2)
				Next
				$s_EncryptText = $v_EncryptCipher
			Next
		Else
			For $i_EncryptCountF = 0 To $i_EncryptLevel Step 1
				$i_EncryptCountB = 0
				$i_EncryptCountC = ''
				$i_EncryptCountD = ''
				$i_EncryptCountE = ''
				$v_EncryptCipherBy = ''
				$v_EncryptCipher = ''
				$v_EncryptSwap = ''
				$av_EncryptBox = ''
				Local $av_EncryptBox[256][2]
				For $i_EncryptCountA = 0 To 255
					$av_EncryptBox[$i_EncryptCountA][1] = Asc(StringMid($s_EncryptPassword, Mod($i_EncryptCountA, StringLen($s_EncryptPassword)) + 1, 1))
					$av_EncryptBox[$i_EncryptCountA][0] = $i_EncryptCountA
				Next
				For $i_EncryptCountA = 0 To 255
					$i_EncryptCountB = Mod(($i_EncryptCountB + $av_EncryptBox[$i_EncryptCountA][0] + $av_EncryptBox[$i_EncryptCountA][1]), 256)
					$v_EncryptSwap = $av_EncryptBox[$i_EncryptCountA][0]
					$av_EncryptBox[$i_EncryptCountA][0] = $av_EncryptBox[$i_EncryptCountB][0]
					$av_EncryptBox[$i_EncryptCountB][0] = $v_EncryptSwap
				Next
				For $i_EncryptCountA = 1 To StringLen($s_EncryptText) Step 2
					$i_EncryptCountC = Mod(($i_EncryptCountC + 1), 256)
					$i_EncryptCountD = Mod(($i_EncryptCountD + $av_EncryptBox[$i_EncryptCountC][0]), 256)
					$i_EncryptCountE = $av_EncryptBox[Mod(($av_EncryptBox[$i_EncryptCountC][0] + $av_EncryptBox[$i_EncryptCountD][0]), 256)][0]
					$v_EncryptCipherBy = BitXOR(Dec(StringMid($s_EncryptText, $i_EncryptCountA, 2)), $i_EncryptCountE)
					$v_EncryptCipher = $v_EncryptCipher & Chr($v_EncryptCipherBy)
				Next
				$s_EncryptText = $v_EncryptCipher
				$i_EncryptCountG = ''
				$i_EncryptCountH = ''
				$v_EncryptModified = ''
				For $i_EncryptCountG = 1 To StringLen($s_EncryptText)
					If $i_EncryptCountH = StringLen($s_EncryptPassword) Then
						$i_EncryptCountH = 1
					Else
						$i_EncryptCountH += 1
					EndIf
					$v_EncryptModified &= Chr(BitXOR(Asc(StringMid($s_EncryptText, $i_EncryptCountG, 1)), Asc(StringMid($s_EncryptPassword, $i_EncryptCountH, 1)), 255))
				Next
				$s_EncryptText = $v_EncryptModified
			Next
		EndIf
		Return $s_EncryptText
	EndIf
EndFunc   ;==>_StringEncryptAgain, Thanks a lot to all of you who helped me!
 --
 Andre
 
_________________________ 
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. (Bertrand Russell )
 |  
| Top |  |  |  |  
| 
| 
| #206787 - 2013-02-24 09:02 PM  Re: ASCII Chars 128 to 255 can stretch a string by more than one character
[Re:  Lonkero] |  
| Lonkero   KiX Master Guru
 
       
 Registered:  2001-06-05
 Posts: 22346
 Loc:  OK
 | 
here is basic example with chr(158):
 
"if a string contains 158, it's ok:"
$string=chr(158)+"word"
? 'len of chr(158)+"word":' len($string) ?
? 'substr(that,2):' substr($string,2) ?
? 'len of substr(chr(158)+"word",2):' len(substr($string,2)) ?
? "but if it is initialised with it alone, it's BROKEN:"
$string=chr(158)
? 'len of chr(158):' len($string) ?
? 'substr(that,2):' substr($string,2) ?
? 'len of substr(chr(158),2):' len(substr($string,2)) ?
?
get $
 with 4.62
 
_________________________! download KiXnet |  
| Top |  |  |  |  
 Moderator:  ShaneEP, Arend_, Jochen, Radimus, Glenn Barnas, Allen, Ruud van Velsen, Mart
 
 | 
| 
 
| 0 registered
and 360 anonymous users online. 
 | 
 |  |