Daniël Tulp
Daniël Tulp

Reputation: 1845

Visual Studio 2012 adds numbers to my double values if I enter them in vb.net

A very strange, presumably meant to be helpfull behaviour of Visual Studio 2012. When I enter a double in vb.net like:

Dim myD as Double = 1.4

When I hit enter of move my focus another way, the formatting kicks in and changes the above to:

Dim myD as Double = 1.39999999999999998

or 1.6 as

Dim myD as Double = 1.6000000000000001

This behaviour does not appear to happen for all doubles. 1.3, 1.5, 1.7 and 1.8 See this youtube movie for the behaviour in action: http://youtu.be/afw4jg58-aU

Why, and more important, how can I prevent this?

Edit:

Extensions installed are: enter image description here

Second Edit

The behaviour seems to have gone away. I do not know what has caused this so for future reference this is useless to anyone, but for now, I'm happy that I don't have to go troubleshooting as suggested.

Upvotes: 2

Views: 555

Answers (2)

Pragmateek
Pragmateek

Reputation: 13396

I had the exact same issue with Visual Studio 2012.

The "solution" if it happens to you is:

  • write your number in your code

  • let VS screw it up, e.g. when you move to another line

  • CTRL-Z to revert VS mess

  • continue writing your code

Upvotes: 0

Hans Passant
Hans Passant

Reputation: 942109

The exact same issue is also reported in this question. Which applied to VS2010, otherwise without a usable answer.

This is an environmental problem, code is getting loaded into Visual Studio that messes with the FPU control word on your machine. It is a processor register that determines how floating point operations work, it looks like this:

enter image description here

The Rounding Control bits are a good source of trouble like this, they determine how the internal 80-bits precision floating point value is truncated to 64-bits. Options are round-up, round-down and round-to-nearest. The Precision Control bits are also a good candidate, options are full 64-bit precision, 53 and 24 bits.

Both VS2012 and the .NET Framework rely on the operating system default, with the expectation that this will not change afterwards. Pretty hard to diagnose trouble arises when code actually does change it, your observation strongly fits the pattern. The most common troublemakers are:

  • code that uses DirectX without the D3DCREATE_FPU_PRESERVE option. DirectX reprograms the precision and rounding control bits to squeeze out a bit more perf.

  • code that was written in an older Borland language product. Its runtime library initializes the FPU control word in a non-standard way. Otherwise a generic problem with software that relies on old runtime library or an old legacy initialization that was carried through in later releases.

  • in general, any code that uses a media codec or media api. Such code tends to reprogram the FPU to squeeze out perf for the same reasons that DirectX does. Especially notorious in a product I worked on which uses such codecs heavily. The codebase was peppered with calls that reset the FPU control word after making a call into external code.

Finding and eliminating such code can be very difficult. DLLs get injected into another process by a large variety of well-intended malware. The SysInternals' Autoruns utility can be very useful, it shows all the possible ways code can be injected with an easy way to disable it. Be prepared to be shocked at what you see and readily disable stuff that doesn't carry a Microsoft copyright.

For dynamic injection, you'll need a debugger to see what is loaded into VS. Start VS again and use Tools + Attach to Process to attach to the first one, selecting the unmanaged debugger. Debug + Windows + Modules shows you what DLLs are loaded. Do beware that the DLL can be transient, a shell dialog like File + Open + File will dynamically load shell extensions into VS and unload them again afterwards. Good luck with it, you'll need it and sometimes the only fix is a rather drastic one.

Upvotes: 5

Related Questions