Its always a good idea to have him checked out and if he is given the all clear then this may be due to him showing over reactive submissive behaviour. In other words he is a little wary and is showing you that he is a bit worried by urinating. He might be worried because he does not understand what it is he has done wrong or just that he can't cope with your raised tone of voice. You might benefit with a session from a qualified, recommended canine behaviourist who will be able to show you how you can show your displeasure without him over reacting with fear. This will involve taking something away like a toy that he likes rather than confronting him face on. Make sure they are qualified as anyone can say they are a dog behaviourist and your vet should be able to recommend one and if you are insured your insurance might even cover the costs.
Meanwhile don’t use such a cross voice. If he does something you don’t like just get up and walk out of the room or take what he has away firmly and calmly.
Stop picking him up so much as it is not a doggy behaviour to go around picking each other up unless they are trying to cause come sort of damage. Just because he is little doesn’t mean he likes being picked up. He might get used to it in time so just a few tiny pick ups a day and slowly increase this as he gets used to it.
Hope that helps somewhat, it is difficult to give advice without seeing the dog so get some one to one advice as soon as you can.
Best wishes
Rosie Barclay