Here's a fun fact I've just spent half an hour learning. Hopefully knowledge of it can save you some time in the future by avoiding the same issue.
"Resources" is a special (reserved) name when it comes to including things in a XAP file. If you name a file "resources" it will never be included in the XAP. (Noted thanks to this answer on StackOverflow)
Additionally (and this is the bit that caught me out) any file that ends ".resources" is also not included.
I had a library named like "MyApp.Resources.dll" that is referenced by my app but just wasn't being included in the XAP and so was leading to a spurious FileNotFoundException when running the code.
Fortunately it was just a case of renaming the outputted assembly to "MyApp.Res.dll" and all was fine again.
I hope this knowledge can save someone else from being caught out by this same issue.