you are far too nice - my feeling would be: you want to see your daughter, you come and get her, and you return her at the pre-arranged time. End of story.
After all, as her mother, you are doing all the bringing up, cleaning, nursing when ill - and even if he does give you a food pension for...