The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘Algorithms’ Category

Next Entries »

Table with the Numeric Data Types in SQL Server

Posted by jpluimers on 2013/10/22

I couldn’t find a table with numeric data types in SQL Server 2012 on MSDN, but since they have not changed since SQL Server 2008,  I copied the table from Understand the 9 Numeric Data Types in SQL Server 2008, added an entry for bit, and links to the relevant SQL Server 2012 pages at MSDN.

Edit: somehow the WordPress editing system suppressed all the superscripts (for the powers of 2 and 10), so I replaced them with caret signs and powers of 2 and 10 to make it more clear and verified them against Floating point numbers and these Wikipedia pages:

Data Type Range of Values Storage Space

Data Type Range of Values Storage Space
tinyint 0 to 255 1 byte
smallint –32,768 to 32,767 2 bytes
int –2^31 to 2^31–1 4 bytes
bigint –2^63 to 2^63–1 8 bytes
decimal(p,s)
numeric(p,s)		 –10^38+1 to 10^38–1 5 to 17 bytes
smallmoney –214,748.3648 to 214,748.3647 4 bytes
money –922,337,203,685,477.5808 to 922,337,203,685,477.5807 8 bytes
real –3.4*10^38 to –1.18*10^38, 0, and 1.18*10^38 to 3.4*10^38 4 bytes
float(n) –1.79*10^308 to –2.23*10^308, 0, and 2.23*10^308 to 1.79*10^308 4 bytes or 8 bytes
bit 0 to 1 0+ bytes

Later I found an even more complete table at SQL Server Data Types Reference – ConnectionStrings.com.

–jeroen

via: Understand the 9 Numeric Data Types in SQL Server 2008.

Posted in Algorithms, Database Development, Development, Floating point handling, Software Development, SQL Server, SQL Server 2008, SQL Server 2008 R2, SQL Server 2012 | Leave a Comment »

A small table that shows differences between decimal, double and float (Single)

Posted by jpluimers on 2013/10/16

Though you programmers all should have read What Every Computer Scientist Should Know About Floating-Point Arithmetic.

But I know you don’t, so below is a small difference on which floating point comparisons fail when using float (f, Single), double (d) and decimal (m, money) values in C#:

0.1 fd_cast_none fd_cast_up fm_cast_none dm_cast_none
0.3 fd_cast_none fd_cast_up fm_cast_none dm_cast_none
0.7 fd_cast_none fd_cast_up fm_cast_none dm_cast_none
0.5 fm_cast_none dm_cast_none
0.9 fd_cast_none fd_cast_up fm_cast_none dm_cast_none

The can help you decide what kind of floating point type you want to use, for instance to answer c# – When should I use double instead of decimal? – Stack Overflow.

I specifically choose the values 0.1, 0.3, 0.5, 0.7 and 0.9 to stress the difference between binary and decimal representations. Apart from the decimal type, you cannot store these decimal values in a binary representation. You can see the decimal representation for a double using the DoubleConverter class (thanks Jon Skeet!)

If you are have a problem that isn’t suite for floating point, then don’t use it. Use rational types, IntX, BigInteger or Complex from the System.Numerics namespace, or arbitraty precision floating point numbers.

The failing methods are part of a bigger DecimalDoubleSingleTestProject, for which you will find the source at BeSharp.net repository (you can browse the sources, and access it through SVN and TFS).

That project contains more checks (see the table at the end which includes 0.100000000001 and 0.1222222222222222222221 based on the accuracy you can get with float/double/decimal) and the failing/succeeding methods are the same.

This is what the failing fd_cast_none, fd_cast_up, fm_cast_none and dm_cast_none methods do:

//...
    [TestClass]
    public class UnitTestBase
    {
        protected float f { get; private set; }
        protected double d { get; private set; }
        protected decimal m { get; private set; }
//...
        public void TestMethod_fd_cast_none()
        {
                Assert.AreEqual(f, d);
        }
//...
        public void TestMethod_fd_cast_up()
        {
                Assert.AreEqual((double)f, d);
        }
//...
        public void TestMethod_fm_cast_none()
        {
                Assert.AreEqual(f, m);
        }
//...
        public void TestMethod_dm_cast_none()
        {
                Assert.AreEqual(d, m);
        }
    }

The methods seem to succeed, but that is just the deceiving part: when you carefully select the values you compare, all checks will eventually fail.

The table at the end shows some more literals that fail other tests. It is caused by all these types have different storage formats.

<h3>Conclusion</h3>

When comparing floating point literals, make sure they are of the same type, and select the type according to what precision or representation features you need.

Note this gets even worse when you start calculating with floating points. You will almost always loose accuracy, watch rounding errors and you cannot even do direct AreEqual comparisons any more. Read the articles by Eric Lippert tagged floating+point+arithmetic – Fabulous Adventures In Coding.

--jeroen

Full table:

0.1 fd_cast_none fd_cast_up fm_cast_none dm_cast_none
0.3 fd_cast_none fd_cast_up fm_cast_none dm_cast_none
0.5 fm_cast_none dm_cast_none
0.7 fd_cast_none fd_cast_up fm_cast_none dm_cast_none
0.9 fd_cast_none fd_cast_up fm_cast_none dm_cast_none
0.100000000001 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none
0.300000000003 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none
0.500000000005 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none
0.700000000007 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none
0.900000000009 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none
0.1222222222222222222221 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up
0.3222222222222222222223 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up
0.5222222222222222222225 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up
0.7222222222222222222227 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_down dm_cast_up
0.9222222222222222222229 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up
144444444444444444444.1 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_down dm_cast_up
344444444444444444444.3 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up
544444444444444444444.5 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up
744444444444444444444.7 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up
944444444444444444444.9 fd_cast_none fd_cast_up fm_cast_none fm_cast_up dm_cast_none dm_cast_up

Posted in .NET, .NET 1.x, .NET 2.0, .NET 3.0, .NET 3.5, .NET 4.0, .NET 4.5, Algorithms, C#, C# 1.0, C# 2.0, C# 3.0, C# 4.0, C# 5.0, Development, Floating point handling, Jon Skeet, Software Development | Leave a Comment »

.NET/C#: suffixes for compiler number literals (via: Stack Overflow)

Posted by jpluimers on 2013/10/15

A long while I avoided using suffixes to force the C# compiler into a specific type and – like Marc Gravell – used type-catst like (decimal)3.1415. I found out the hardway: it doesn’t work all the time.

(decimal)1.000000000000001 evaluates to 1 whereas
1.000000000000001m evaluates to 1.000000000000001.
In the former case, the computer parses the literal as a double and then casts it to a decimal. – Joe Albahari Apr 15 ’11 at 6:02

My main reason for using the casts was that I kept forgetting the suffixes needed to force a literal to be of a specific type. Hence this post.

The suffixes and casts are not limited to use in consts: any place where a numeric literal is used, you can use a suffix to force a compile time type.

Though documented in sections 2.4.4.2 Integer literals (C#) and 2.4.4.3 Real literals (C#) of the C# standard and appendix C. Grammar (C#). That standard does not contain a comprehensive list, much after I wrote this post I found Value Types Table (C# Reference).

This post only lists the C# suffixes. Abel Braaksma published a blog entry Overview of type suffixes in C# and VB.Net.

StackOverflow user  sixlettervariables has this (slightly) edited list:

var y = 0f; // y is single/float
var z = 0d; // z is double
var r = 0m; // r is decimal
var i = 0U; // i is unsigned int
var j = 0L; // j is long (note capital L for clarity)
var k = 0UL; // k is unsigned long (note capital L for clarity)

Some more background info: Read the rest of this entry »

Posted in .NET, .NET 1.x, .NET 2.0, .NET 3.0, .NET 3.5, .NET 4.0, .NET 4.5, Algorithms, C#, C# 1.0, C# 2.0, C# 3.0, C# 4.0, C# 5.0, Development, Floating point handling, Software Development | Leave a Comment »

Paint.NET v3.5.11 is now available (via: Paint.NET Blog)

Posted by jpluimers on 2013/08/17

Programmers scale: time versus project completeness

Programmers scale: time versus project completeness

I totally agree that Paint.NET is the best free image and photo editor on Windows.

Writing quality software takes time, not only when writing it in spare time like Rick Brewster does. Getting things “right” is a tedious process and often will set you back: just watch the graph on the right.

So I’m not surprised that it took a very long time after the first Paint.NET 4.0 idea in 2008 to get “close” to a release.

And indeed, it looks like Rick has become much closer which will please many people waiting for Paint .NET 4.

I’m really glad with the announcement that Paint.NET v3.5.11 BETA is now available – Paint.NET Discussion & Questions – Paint.NET Forum.

Edit: while writing this, the final Paint.NET v3.5.11 came out.

It paves the way for Paint .NET 4.0 update in the future, and fixes/improves quite a few things.

A few quotes from it:

This is probably not the update you were expecting I need to push out an update to v3.5 in preparation for the eventual release of v4.0 […] releasing a “beta” today […] I’ll be pushing out the Final/RTM in a few days.

The primary goal of this update is preparing for the v4.0 release: v3.5.10 will not be able to offer the v4.0 update, but v3.5.11 will. […]

Here are the changes for this release:

  • Fixed: The Gaussian Blur effect was incorrectly calculating alpha values for non-opaque pixels. (http://forums.getpaint.net/index.php?/topic/18483-gaussian-blur-mistreats-alpha/)
  • Improved performance of the Sharpen effect by about 25%
  • Improved performance of the Median effect by about 30%
  • Improved performance of the Fragment effect by about 40%
  • Improved performance of the Unfocus effect by about 100%
  • Reduced memory usage when many selection manipulation operations are in the history/undo stack (the undo data is now saved to disk)
  • The built-in updater now supports upgrading to paint.net 4.0 (once it’s available)

There have been rumors floating around that Paint.NET is “dead.” This is not true!

–jeroen

via: Paint.NET Blog | The best free image and photo editor. By Rick Brewster..

Posted in .NET, .NET 3.5, .NET 4.0, Algorithms, Development, Floating point handling, Power User, Software Development, Windows, Windows 7, Windows 8, Windows Server 2000, Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2, Windows Vista, Windows XP | Tagged: , , | Leave a Comment »

.NET/C# duh moment of the day: “A char can be implicitly converted to ushort, int, uint, long, ulong, float, double, or decimal (not the other way around; implicit != implicit)”

Posted by jpluimers on 2012/11/20

A while ago I had a “duh” moment while calling a method that had many overloads, and one of the overloads was using int, not the char I’d expect.

The result was that a default value for that char was used, and my parameter was interpreted as a (very small) buffer size. I only found out something went wrong when writing unit tests around my code.

The culprit is this C# char feature (other implicit type conversions nicely summarized by Muhammad Javed):

A char can be implicitly converted to ushort, int, uint, long, ulong, float, double, or decimal. However, there are no implicit conversions from other types to the char type.

Switching between various development environments, I totally forgot this is the case in languages based on C and Java ancestry. But not in VB and Delphi ancestry  (C/C++ do numeric promotions of char to int and Java widens 2-byte char to 4-byte int; Delphi and VB.net don’t).

I’m not the only one who was confused, so Eric Lippert wrote a nice blog post on it in 2009: Why does char convert implicitly to ushort but not vice versa? – Fabulous Adventures In Coding – Site Home – MSDN Blogs.

Basically, it is the C ancestry: a char is an integral type always known to contain an integer value representing a Unicode character. The opposite is not true: an integer type is not always representing a Unicode character.

Lesson learned: if you have a large number of overloads (either writing them or using them) watch for mixing char and int parameters.

Note that overload resolution can be diffucult enough (C# 3 had breaking changes and C# 4 had breaking changes too, and those are only for C#), so don’t make it more difficult than it should be (:

Below a few examples in C# and VB and their IL disassemblies to illustrate their differnces based on asterisk (*) and space ( ) that also show that not all implicits are created equal: Decimal is done at run-time, the rest at compile time.

Note that the order of the methods is alphabetic, but the calls are in order of the type and size of the numeric types (integral types, then floating point types, then decimal).

A few interesting observations:

  • The C# compiler implicitly converts char with all calls except for decimal, where an implicit conversion at run time is used:
    L_004c: call valuetype [mscorlib]System.Decimal [mscorlib]System.Decimal::op_Implicit(char)
    L_0051: call void CharIntCompatibilityCSharp.Program::writeLineDecimal(valuetype [mscorlib]System.Decimal)
  • Same for implicit conversion of byte to the other types, though here the C# and VB.NET compilers generate slightly different code for run-time conversion.
    C# uses an implicit conversion:
    L_00af: ldloc.1
    L_00b0: call valuetype [mscorlib]System.Decimal [mscorlib]System.Decimal::op_Implicit(uint8)
    L_00b5: call void CharIntCompatibilityCSharp.Program::writeLineDecimal(valuetype [mscorlib]System.Decimal)
    VB.NET calls a constructor:
    L_006e: ldloc.1
    L_006f: newobj instance void [mscorlib]System.Decimal::.ctor(int32)
    L_0075: call void CharIntCompatibilityVB.Program::writeLineDecimal(valuetype [mscorlib]System.Decimal)

Here is the example code: Read the rest of this entry »

Posted in .NET, Agile, Algorithms, C#, C# 1.0, C# 2.0, C# 3.0, C# 4.0, C# 5.0, C++, Delphi, Development, Encoding, Floating point handling, Java, Software Development, Unicode, Unit Testing, VB.NET | 1 Comment »

Delphi XE2 x64 Extended floating point support: CodeCentral 28488 TExtendedX87: FPU-backed 80-bit Extended type for Win64

Posted by jpluimers on 2011/09/14

in the Delphi 64 world, there is no official support for the Extended data type for various reasons.

If you really need it, then you can use the TExtendedX87 unit by Philipp M. Schlüter as mentioned in this Embarcadero forum thread.

–jeroen

Via: 28488 TExtendedX87: FPU-backed 80-bit Extended type for Win64.

Posted in Algorithms, Delphi, Delphi x64, Development, Floating point handling, Software Development | 2 Comments »

Why is the result of RoundTo(87.285, -2) => 87.28 – Stack Overflow

Posted by jpluimers on 2011/03/08

Programmers on all sorts of platforms get this wrong all the time (I admit having done this in bad ways myself too).

In short: Don’t expect floating point values in a computer to be represented as decimals.

Rob Kennedy wrote a very nice answer on this:

The exact value 87.285 is not representable as a floating-point value in Delphi. A page on my Web site shows what that value really is, as Extended, Double, and Single:

87.285 = + 87.28500 00000 00000 00333 06690 73875 46962 12708 95004 27246 09375

87.285 = + 87.28499 99999 99996 58939 48683 51519 10781 86035 15625

87.285 = + 87.28500 36621 09375

And David Heffernan points to the best link you can get on this topic:

The classic reference on floating point is What Every Computer Scientist Should Know About Floating-Point Arithmetic.

For currency based calculations, if indeed this is, you should use a base 10 number type rather than base 2 floating point. In Delphi that means `Currency`.

–jeroen

via delphi – Why is the result of RoundTo(87.285, -2) => 87.28 – Stack Overflow.

Posted in .NET, Algorithms, C#, Delphi, Development, Floating point handling, Software Development | 14 Comments »

More Delphi x64 bits – Allen Bauer (kylix_rd) on Twitter

Posted by jpluimers on 2010/11/01

To save people from browsing Twitter history:

danny_thorpe:

@davidheff @kylix_rd x64 has SSE2 for FP ops. 8 directly addressable 64bit XMM FP registers. 8087 FP ops emulated in microcode, deprecated

28 Oct

@kylix_rd You mean critical mass isn’t church service on Christmas and Easter?

27 Oct

@kylix_rd Bah. What do they know about floating point? :P MSVC turns off all the FP hardware (exceptions) by default already!

27 Oct

@kylix_rd Yes, we did. The reason was Extendeds don’t exist at all in CLR. x64 FP ops only supporting 64 bit floats is justification

27 Oct

@kylix_rd Ok, so your cause/effect statement was incomplete. :P Shame to see extendeds go. How to count all atoms in the universe now?

27 Oct

@kylix_rd That doesn’t follow. You could still pass extendeds on the stack (not using the x87 register stack)

27 Oct

kylix_rd:

@davidheff Yes. Alignment is critical. Even the stack must remain properly aligned. Its all part of the ABI.

27 Oct

@davidheff SSE instructions and the xmm0:xmm15 registers.

27 Oct

@danny_thorpe That and the fact that MS strongly discourages the use of the FP coproc on 64 bit Windows.

27 Oct

@danny_thorpe Extendeds don’t align well, FP ops would dumb them down, sub-optimal codegen, are other reasons to drop them.

27 Oct

http://goo.gl/D0Kv. “The x87 register stack is unused. … must be considered volatile across function calls” So, Extended = Double.

27 Oct

As speculated, the Tag property will become a NativeInt.

27 Oct

@malcolmgroves @seanbdurkin And if we change the underlying implementation, don’t complain… mkay?

27 Oct

The elephants in the room. Max 64bit PE image size 4GB. Extended = Double (since xmm0-xmm4 are for FP param passing).

27 Oct

Number of calling conventions in x64 – 1. pascal, register, cdecl, stdcall… gone, treated as nop.

26 Oct

Wow… just a few tweets about D64 and I get flooded with new followers. Welcome to all my new followers from the last 24 hours.

26 Oct

Most common 64bit data models, LP64 and LLP64. Windows = LLP64, Linux, OSX = LP64. D64/Windows = LLP64.

26 Oct

64bit gotcha: SizeOf(THandle/HWND/HMODULE/Hxxxx) = SizeOf(Pointer) = 8. This isn’t valid: Value := Integer(Handle).

26 Oct

Type sizes for 64: SizeOf(Integer)=4, SizeOf(NativeInt)=SizeOf(Pointer)=8,

26 Oct

If you must… Value := Integer(TList[x]) -> Value := NativeInt(TList[x]). Even better, TList.

26 Oct

via Allen Bauer (kylix_rd) on Twitter
Danny Thorpe (danny_thorpe) on Twitter

Posted in 8087, Algorithms, Delphi, Development, Floating point handling, kylix_rd, SocialMedia, Software Development, Twitter | Leave a Comment »

Delphi – Michael Justin had strange floating point results when his 8087 FPU Control Word got hosed

Posted by jpluimers on 2009/05/06

Two days ago, [Wayback] Michael Justin (who just released version 1.7 of the [Wayback] Habari Active MQ Client components) posted a blog entry about a strange circumstance [Wayback] when 1.99 would not compare equally to 1.99.
He tracked it down to the [Wayback] 8087 (more formally: Intel [Wayback] FPU) Control Word being hosed on his system.

I could not reproduce his particular case, but since I have seen similar issues in the past, I wrote the DUnit test case below which shows you what can happen by manually setting the 8087 Control Word.

The difference between the 8087 Control Word values $1372 (default) and $1272 (failure) is the internal mantissa precision (see the [Wayback] “Art of Assembly Language” and the [Wayback] Intel FPU Control Word documentation on this).
Edit: Found a [Wayback] much more complete description of the bits in the FPU Control word.

It changes from 64 bits to 53 bits, which is enough to make 1.99 not equal to 1.99.

I have seen behaviour like this in the past with some networking stacks in the Turbo Pascal 7 era, with some C++ DLL’s in the Delphi 1-3 era, and some printer drivers in the Delphi 5-7 era.
Let me know in the comments (or using the contact form) where you have bumped into this.

The code below makes use of the Jcl8087 unit which is part of the JCL ([Wayback] JEDI Code Library) at [Wayback] SourceForge.
Add the unit to any DUnit test project you created and observe the results.
Read the rest of this entry »

Posted in 8087, Algorithms, Delphi, Development, Floating point handling, Software Development | 8 Comments »

Next Entries »