I would maybe continue feeding on the side he left off when you change his nappy. It takes a good few weeks for the milk to be properly established and in the first few weeks alot of the watery milk comes first, then the fattier milk and then whats the equivalent of gold top milk comes last. It can take babies a good 45 mins to get all the milk on one side so it might be worth trying that same sife for a bit longer before moving to the next side. Do you ever get that weird feeling near the end of the feed where baby's tongue sort of vibrates your nipple? Apparently thats a good sign that he's getting the really fatty milk as its the equivalent of the baby 'licking the plate clean'.
Also breastfed babies are just as likely to suffer from trapped gas and things like reflux but there are certain things you can try to help relieve those ailments e.g. not having much gassy food in your diet, changing the position you feed in so baby is more upright and also things like infacol can help.
Otherwise if youre still a bit concerned are there any bf support groups you could go too which could check the latch as sometimes a small adjustment to the latch and positioning could help things like excess wind getting in.
Good luck though chick, 4 weeks breastfeeding and surviving a newborn is amazing!xx
