The best way to prevent others from using a download URL is to generate a tokenized URL using GetOptimalURLs. This will create a URL to download a file in your account that can be set to expire at a specific period of time as well as being restricted to a specific IP address.
Alternately, you could look at System.security.loadPolicyFile in Flash. This
method lets you select a specific crossdomain.xml in a directory other
than the root. I haven't experimented with this in all versions but
its worth a look to see if it will let you setup the permissions you
are looking for.