Well I have figured out why the size matters (Yes ladies, it does ;\) ).
Apparently it is a memory limit.
To fix this, you have to feed the bytes to the System.Security.Cryptography object in blocks using the .TransFormBlocks and eventually the .TransFormFinalBlocks property of the object before using the ComputeHash property.
The only problem I see is to get the stream reader to split it up in chucks of less then 200 MB.
If I get more time I'll try to sort it out.