floating point - ActionScript: Number.toExponential returns incorrect results for some values -


i found @ least 2 numbers actionscript's number.toexponential(20) returns incorrect results.

the obvious value 0.

trace(number(0).toexponential(20)); // 0.00000000000000000000e-16 trace(number(0).toexponential(2));  // 0.00e-16 trace(number(0).toexponential(1));  // 0.0e-16 trace(number(0).toexponential(0));  // 1e-15 - worse! 

i've made class double playing around numbers (which ieee 754 double-precision binary floating point format numbers). takes number , extracts bits sign, bits , significand (the latter printed including implicit leading bit), or can generate number provided sign, bits , significand. there methods next , prior nearest representable numbers. here's 0 looks internally:

[double number=0.00000000000000000000e-16 sign=1 exponent=-1023 iszero significand=00000000000000000000000000000000000000000000000000000] 

the other incorrect results nearest representable number greater number.max_value / 2 , number.max_value / 4.

// number.max_value / 2. trace(number(8.98846567431157854072e+307).toexponential(20)); // 8.98846567431157854072e+307 // nearest representable number greater number.max_value / 2. printed incorrectly! trace(number(8.98846567431157953864e+307).toexponential(20)); // 0.e+327 // number 2 ulps greater number.max_value / 2. trace(number(8.98846567431158153448e+307).toexponential(20)); // 8.98846567431158153448e+307 // nearest representable number greater number.max_value / 4. printed incorrectly! trace(number(4.49423283715578976932e+307).toexponential(20)); // 0.e+327 

here's these 4 numbers internally:

[double number=8.98846567431157854072e+307 sign=1 exponent=1022  significand=11111111111111111111111111111111111111111111111111111] [double number=0.e+327 sign=1 exponent=1023  significand=10000000000000000000000000000000000000000000000000000] [double number=8.98846567431158153448e+307 sign=1 exponent=1023  significand=10000000000000000000000000000000000000000000000000001] [double number=0.e+327 sign=1 exponent=1022  significand=10000000000000000000000000000000000000000000000000000] 

so far 3 numbers printed incorrectly have 0 significand (with implicit leading 1), , exponent values of 1023 (emax normal numbers), 1022 (emax - 1), or -1023 (special exponent value ±zero , subnormal numbers, emin - 1).

looks toexponent doesn't correctly handle @ least these 3 cases . there other numbers print incorrectly?

trying check if toexponential produce correct results other precision values.

trace(number(8.98846567431157953864e+307).toexponential(2)); // 0.e+309 trace(number(8.98846567431157953864e+307).toexponential(1)); // 0.e+308 trace(number(8.98846567431157953864e+307).toexponential(0)); // 1e+308 - less bad! 

if interested in double class post it. i've written faster implementations next(x:number) , prior(x:number) functions work pretty fast without messing internal bits of numbers.

references: https://en.wikipedia.org/wiki/double-precision_floating-point_format


Comments

Popular posts from this blog

python - No exponential form of the z-axis in matplotlib-3D-plots -

php - Best Light server (Linux + Web server + Database) for Raspberry Pi -

c# - "Newtonsoft.Json.JsonSerializationException unable to find constructor to use for types" error when deserializing class -