Most students who learn English as a second language do it the hard way. They learn in school. Schools focus on test scores rather than anything practical. Grammar is overemphasized whereas modern spoken English is somehow dismissed.
Native speakers in any language have a large vocabulary but in daily life use only a small fraction of it. There are so many ways to express ideas and the simplest ways of expressing ideas are the ones that usually prevail. ESL students often think in their own language and then translate word for word into English. This rarely works. They even have a hard time understanding a simple movie.
The strange part is that vocabulary is often not the problem at all. It is the little things. Native English speakers learn phrasal verbs when they are young. They do not learn them in school, but rather from the surrounding environment. I did not even know what a phrasal verb was until I started teaching ESL. Ironically, I learned this term from my students who wanted to learn more. I started paying attention to my own speech as well as to the speaking of English speakers around me and on TV. It is hard to listen to 3 sentences of native speakers without hearing at least one phrasal verb. When ESL students are learning, they should pay more attention to how native speakers actually speak. Watch modern movies and TV. You will see what I am talking about. It will improve the English learning speed dramatically.