This paper discusses difficulty factors affecting learners` listening. Regarding Korean education, we could evaluate a learner`s listening proficiency only by spoken questions. Consequently, many studies are biased towards listening to questions; and detailed studies about listening processes instead of results, and passage comprehension instead of question comprehension, could not be successful. Hence, we focused on listening processes and passage comprehension, and analyzed learners` living body singles instead of answers. We found that learners` attention and tension levels affected their listening proficiency. Also, the more spoken the language, the easier to understand. Unfamiliar and detailed information was more difficult to listen, but passage`s length had no significant effect. Additionally, if learner`s topic familiarly was high, response was active; otherwise it was passive. Based on these results, this paper suggests how to organize listening passages to improve learners` proficiency.