[Commits] Rev 4117: MDEV-5962: EITS: value "position" calculated incorrectly for CHAR(n) columns in file:///home/psergey/dev2/10.0/

Sergey Petrunya psergey at askmonty.org
Thu Mar 27 12:55:30 EET 2014


At file:///home/psergey/dev2/10.0/

------------------------------------------------------------
revno: 4117
revision-id: psergey at askmonty.org-20140327105529-8ogxytk3nnt30b1e
parent: psergey at askmonty.org-20140327090800-l3je8eqjrfx21iki
committer: Sergey Petrunya <psergey at askmonty.org>
branch nick: 10.0
timestamp: Thu 2014-03-27 14:55:29 +0400
message:
  MDEV-5962: EITS: value "position" calculated incorrectly for CHAR(n) columns
  - Dont substract unsigned numbers, use correct calculations.
  - (there is no testcase because effort is required to come up with it)
=== modified file 'sql/field.cc'
--- a/sql/field.cc	2014-03-26 21:25:38 +0000
+++ b/sql/field.cc	2014-03-27 10:55:29 +0000
@@ -1175,6 +1175,15 @@ inline ulonglong char_prefix_to_ulonglon
   return uint8korr(src); 
 }
 
+/*
+  Compute res = a - b, without losing precision and taking care that these are
+  unsigned numbers.
+*/
+static inline double safe_substract(ulonglong a, ulonglong b)
+{
+  return (a > b)? double(a - b) : -double(b - a);
+}
+
 
 /**
   @brief
@@ -1227,10 +1236,10 @@ double Field::pos_in_interval_val_str(Fi
   minp= char_prefix_to_ulonglong(minp_prefix);
   maxp= char_prefix_to_ulonglong(maxp_prefix);
   double n, d;
-  n= mp - minp;
+  n= safe_substract(mp, minp);
   if (n < 0)
     return 0.0;
-  d= maxp - minp;
+  d= safe_substract(maxp, minp);
   if (d <= 0)
     return 1.0;
   return MY_MIN(n/d, 1.0);



More information about the commits mailing list