A task-independent filler modeling for robust key-phrase detection and verification is proposed. Instead of assuming task-specific lexical knowledge, our model is designed to characterize phrases depending on the speaking-style, thus can be trained with large corpora of different but similar tasks. We present two implementations of the portable and general model. The dialogue-style dependent model trained with the ATIS corpus is used as a filler and shown to be effective in detection-based speech understanding on different dialogue applications. The lecture-style dependent filler model trained with transcriptions of various oral presentations also improves the verification of key-phrases uttered during lectures.