Page 1 of 1 1
Topic Options
#141721 - 2005-06-15 07:49 PM Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
What am I doing wrong?
Code:
 $Val = 14082 * (CINT($RV)/100) 



Where $RV is a value read from the registry.

I have confirmed that it is reading the registry correctly, but where is my math wrong?
_________________________
Today is the tomorrow you worried about yesterday.

Top
#141722 - 2005-06-15 08:00 PM Re: Feeling really stupid
Chris S. Offline
MM club member
*****

Registered: 2002-03-18
Posts: 2368
Loc: Earth
Have you tried testing the VarType of $RV? Most values returned are strings.
Top
#141723 - 2005-06-15 08:04 PM Re: Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
It is returning as string. Expected but the use of CINT should fix that should it not?
_________________________
Today is the tomorrow you worried about yesterday.

Top
#141724 - 2005-06-15 08:04 PM Re: Feeling really stupid
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Well, since you don't say what $RV is to begin with and you don't say what your result is vs. the expected result, what are we to suggest?
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#141725 - 2005-06-15 08:07 PM Re: Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
Les,
The registry value is a #. At this time that # is 80 but changes.

So the result of 0 is incorrect as I was expecting 11265 using that mathmatical formula.
_________________________
Today is the tomorrow you worried about yesterday.

Top
#141726 - 2005-06-15 08:13 PM Re: Feeling really stupid
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Hmmm... not getting this???

the integer of 80/100 is 0
$Val = 14082 * (0) = 0

What did you expect it to be?
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#141727 - 2005-06-15 08:17 PM Re: Feeling really stupid
maciep Offline
Korg Regular
*****

Registered: 2002-06-14
Posts: 947
Loc: Pittsburgh
$Val = 14082.0 * (cdbl($RV)/100)
Top
#141728 - 2005-06-15 08:18 PM Re: Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
So my problem is that I want it return .8 So I should use ?? instead

I tried doing it without the CINT and I get an error in expression.

And I do realize now why it was returning a 0, but still am not sure how to convert the string to a number


Edited by Gargoyle (2005-06-15 08:22 PM)

Top
#141729 - 2005-06-15 08:32 PM Re: Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
The code that Maciep gave works wonderfully. However I went back to the documentation and reread CDBL, and unless you already know what that does, it really doesn't say that you use it to convert a string to a number.

Not complaining or anything, but I just didn't understand from the documentation that it would do that.

Also (and I am sure it is because I don't understand CDBL), why do you have to include the .0 on the original value? I tried it without and the returned value was 14082.

Thanks for the help
_________________________
Today is the tomorrow you worried about yesterday.

Top
#141730 - 2005-06-15 08:39 PM Re: Feeling really stupid
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Without the .0 it would be an integer, and since you start with an integer, you end with an integer.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#141731 - 2005-06-15 08:50 PM Re: Feeling really stupid
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
Try this:
Code:
'1 + 1.5   = '    1 + 1.5 ?
'"1" + 1.5 = ' "1" + 1.5 ?
'1.0 + 1.5 = ' 1.0 + 1.5 ?



Once you understand this, truly you will be enlightened

Top
#141732 - 2005-06-16 12:08 AM Re: Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
Ok, I understand the second and third line, but not the first.
_________________________
Today is the tomorrow you worried about yesterday.

Top
#141733 - 2005-06-16 12:42 AM Re: Feeling really stupid
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Like I said, if you start with an integer, you end with an integer. Why is that so hard to understand?
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#141734 - 2005-06-16 01:10 AM Re: Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
I understand the Interger part. But I thought that it would always round down 1.5 would = 1, but the results returned 3 as if it rounded up.
_________________________
Today is the tomorrow you worried about yesterday.

Top
#141735 - 2005-06-16 10:05 AM Re: Feeling really stupid
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
Read the manual section on CInt() - this gives a reasonable explanation of what's going on. I'll highlight the most important points:
  • Values are rounded to the nearest even number when the fractional part is exactly 0.5
  • If you are after the integer part rather than the nearest integer, use Int() which does not round the value but simply drops the fractional part.

Top
#141736 - 2005-06-16 05:19 PM Re: Feeling really stupid
Gargoyle Offline
MM club member
*****

Registered: 2004-03-09
Posts: 1597
Loc: Valley of the Sun (Arizona, US...
I just reread the CINT. And it states that it will round down 0.5 would = 0 and 1.5 would = 1. Where the example shown rounds up.

Is this a difference in versions? I am using the manual from 4.22.
_________________________
Today is the tomorrow you worried about yesterday.

Top
#141737 - 2005-06-16 05:31 PM Re: Feeling really stupid
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
I've just downloaded 4.22 to check, and the remarks for CInt() read as follows:
Quote:

Remarks
CInt differs from the Fix and Int functions, which truncate, rather than round, the fractional part of a number. When the fractional part is exactly 0.5, the CInt function always rounds it to the nearest even number. For example, 0.5 rounds to 0, and 1.5 rounds to 2.




I use an older printed version (4.20) and the remarks are the same.

Top
Page 1 of 1 1


Moderator:  Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart 
Hop to:
Shout Box

Who's Online
0 registered and 918 anonymous users online.
Newest Members
batdk82, StuTheCoder, M_Moore, BeeEm, min_seow
17885 Registered Users

Generated in 0.141 seconds in which 0.103 seconds were spent on a total of 12 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org