Quantcast
Channel: NPOI
Viewing all articles
Browse latest Browse all 1621

Commented Unassigned: Incorrect value of calculation with INT operator [13924]

$
0
0
In the following example I try to calculate different formulas, but the results of calculation are sometimes incorrects (and really understandables) :

```
var workbook = new HSSFWorkbook();
var sheet = workbook.CreateSheet();
var row = sheet.CreateRow(0);
var cell = row.CreateCell(0);
var evaluator = workbook.GetCreationHelper().CreateFormulaEvaluator();

// should be 6.0 => OK
cell.SetCellFormula("INT(6.99)");
var cellValue = evaluator.Evaluate(cell);
var result = cellValue.NumberValue;

// should be 6.0 => KO : 7.0 is returned
var cell1 = row.CreateCell(1);
cell1.SetCellFormula("INT(6.991)");
var cellValue1 = evaluator.Evaluate(cell1);
var result1 = cellValue1.NumberValue;

// should be 6.0 => OK
var cell2 = row.CreateCell(2);
cell2.SetCellFormula("INT(6.9900000000000005)");
var cellValue2 = evaluator.Evaluate(cell2);
var result2 = cellValue2.NumberValue;

// should be 6.0 => KO : 7.0 is returned
var cell3 = row.CreateCell(3);
cell3.SetCellFormula("INT(6.9900000000000007)");
var cellValue3 = evaluator.Evaluate(cell3);
var result3 = cellValue3.NumberValue;
```

Am I doing something wrong ?
Comments: ** Comment from web user: ebyrob **

This is controlled in the source code here:

main\SS\Formula\Functions\Int.cs

public class Int : OneArg
{

public override double Evaluate(double d)
{
if (d > 0)
return Math.Round(d - 0.49);
else
return Math.Round(d - 0.5);
}

}

Should be Math.Floor(d) instead of Round I think.


Viewing all articles
Browse latest Browse all 1621

Trending Articles