This paper considers amplify-and-forward (AF) relaying systems with multiple relay nodes between the source and the destination node, where all nodes are equipped with multiple antennas. Using multiple relay nodes to transmit a message, a distributed diversity gain can be obtained. In this paper, we address the optimization problem of relay weighting matrices which minimizes the mean-square-error (MSE). For general relaying systems which have an arbitrary number of relay nodes, the given problem has a non-convex cost function. Thus, we propose an iterative scheme which identifies a local optimal relay precoder by deriving the gradient of the MSE and applying the gradient descent algorithm. Simulation results show that the proposed iterative scheme outperforms the conventional schemes for multiple MIMO relay systems with a small number of iterations in terms of both the MSE and the bit error rate.