There are several FAQs that show up in the forums and newsgroups regarding floating point issues. This article should address most of them:

1. Why does the accuracy of my calculations drop when I'm using DirectX?
   For speed DirectX switched the Floating Point Unit (FPU) on your processor into single precision mode when you create the device. .Net and other runtime libraries have no clue that this has happened, so even though you may have written all your code to use doubles everything will actually be using singles. Normally this is fine but if you have some high precision simulations or physics then you may notice a problem. The only supported solution is to use the correct CreateFlag when you create the device. There will be a performance hit though it is undocumented how much. If you entertain thoughts of modifying the FPU control bit yourselves then you should heed these warnings (See "Manipulating the Float-Point Control Word") for why its a bad idea.
2. I'm trying to animate an object at position which is a large number and I notice that I am losing accuracy. I can only move the object in large steps.
   -or -I set a float to XXXXXX but when I read it back its value has changed to YYYYYY
   Generally this happens when people try to model which has a wide range of distances. e.g. trees on a planet are 0.01 apart and planets are 100,000 apart. At the 100,000 distances they notice that they only see animation if they move things by +/-10 whereas they can move the 0.01 things by +/-0.0001. See this thread for an example.
   This problem occurs because of a lack of understanding how floating point numbers are stored, especially if your FPU is in single precision mode. In general the larger the floating point number gets the less accurate the number it can store. For a bunch of interesting posts on floating point see Eric Lipperts 6 part series 1, 2, 3, 4, 5, 6. You may also find the floating point calculator useful for seeing which numbers can be accurately represented and which are approximated.