Recently, many research works have been reported about how physical cryptanalysis can be carried out on cryptographic devices by exploiting any possible leaked information through side channels. In this paper, we demonstrate a new type of safe-error based hardware fault cryptanalysis which is mounted on a recently reported countermeasure against simple power analysis attack. This safe-error based attack is developed by inducing a temporary random computational fault other than a temporary memory fault which was explicitly assumed in the first published safe-error based attack (in which more precisions on timing and fault location are assumed) proposed by Yen and Joye. Analysis shows that the new safe-error based attack proposed in this paper is powerful and feasible because the cryptanalytic complexity (especially the computational complexity) is quite small and the assumptions made are more reasonable. Existing research works considered many possible countermeasures against each kind of physical cryptanalysis. This paper and a few previous reports clearly show that a countermeasure developed against one physical attack does not necessarily thwart another kind of physical attack. However, almost no research has been done on dealing the possible mutual relationship between different kinds of physical cryptanalysis when choosing a specific countermeasure. Most importantly, in this paper we wish to emphasize that a countermeasure developed against one physical attack if not carefully examined may benefit another physical attack tremendously. This issue has never been explicitely noticed previously but its importance can not be overlooked because of the attack found in this paper. Notice that almost all the issues considered in this paper on a modular exponentiation also applies to a scalar multiplication over an elliptic curve.