To be safe one should add (and change) a query parameter after their Silverlight application to prevent the file from being cached.
The HTML would look something like:
<param name="source" value="ClientBin/MySilverlightApplication.xap?revision=2012011801"/>
And the revision needs to be changed after every new build that is published.