It_took_my_meds
(Hey THIS is FUN)
2006-02-03 12:04 AM
Enhance CDbl to accept string representations of hexadecimal numbers

Hi All,

In order to avoid integer overflows by using val to convert large string representations of hexadecimal numbers, I suggest that CDbl should be enhanced to accept them. There are currently 3 UDF's that need to be utilised to process such hexadecimal numbers before you can pass the result into CDbl to get the decimal number. I have said before that I would prefer enhancements that deliver functionality that can't be realised by using UDF's but I believe that this is an exception.

While I'm at it, I would like to strongly suggest once more that COM event handling is also added.

Cheers,

Richard


NTDOCAdministrator
(KiX Master)
2006-02-03 12:42 AM
Re: Enhance CDbl to accept string representations of hexadecimal numbers

Quote:

COM event handling




Think Microsoft is heading towards ".NET event handling"


It_took_my_meds
(Hey THIS is FUN)
2006-02-03 01:08 AM
Re: Enhance CDbl to accept string representations of hexadecimal numbers

Both would be good!

Richard H.Administrator
(KiX Supporter)
2006-02-03 10:09 AM
Re: Enhance CDbl to accept string representations of hexadecimal numbers

As you know you can already pass hex numbers directly:
Code:
; Biggest "positive" hex.
CDbl(&7FFFFFFF) ? ; = 2147483647

; Biggest hex that can be handled before things get screwy.
CDbl(&FFFFFF7F) ? ; = -129



The limit on the size of the hex number suggests that an integer conversion is taking place. This in turn implies that the conversion is done when the script is parsed, then the converted number is passed to CDbl().

KiXtart could be enhanced to convert hex numbers to double rather than int, but that would probably break many existing scripts. Perhaps an enhancement to support the "0x0" format of hex numbers which converted them internally to a larger data type could work.

I think that a general purpose built-in base converter function would be a better solution than enhancing CDbl().

The only problem with using doubles is that they are an approximation (I think that's the right word) of the number, which can lead to strange fractional parts appearing when you perform maths on them. However this is true of any use of data types which store the datum using similar methods, and is not just restricted to base conversion.

In the mean time, if you want to cut down on the number of UDFs required, this will do it for you:
Code:
udfHexToDBL("FFFFFFFFFFFFFFFF")

Function udfHexToDBL($s)
$udfHexToDBL=1.0
While $s<>""
$udfHexToDBL=$udfHexToDBL*Execute("Exit &"+Right($s,1))
$s=Left($s,-1)
Loop
EndFunction



The example results in "6.56840835571289E+018", which is a fair sized number.