With the motivation of utilizing bi-directional contextual dependence in acoustic modeling, in this paper, a bidirectional hidden Markov modeling approach for speech recognition is studied and the importance of bi-directional contextual dependence for speech recognition is identified by a series of comparative experiments. Furthermore, hidden Markov random field based acoustic modeling techniques using our previously proposed contextual vector quantization method and iterated conditional modes algorithm which is very suitable for the parallel processing implementation are also attempted. Their viability is confirmed by a series of preliminary experiments in a speaker independent isolated English letter recognition task.