I stopped producing videos nearly 4 years ago and I was never a "major player", but I thikn the issues I had were similar to others. For the most part, the models were professional, on time, etc. I worked with professionals and amateurs and the professionals (Reya Fet, Bailey Paige, ect) were great. But, on occasion, I worked with models (mostly amateur) who were very demanding, often late (if they showed at all) and, in some cases, not fit for being on camera.
The other issue I had was that I couldn't "enjoy" the work. I was more concerned with the camera angles, lighting, model comfort, etc. I would get DMs from people telling me I was so lucky to do this, but I was working and this wasn't a date.
The biggest problem, which I think all producers can relate to was piracy. After my 1st year producing, my revenues went down almost 80%. I would see my clips on other sites, regularly. Eventually, I just gave up. I may come back with new material, but I can't say for sure.