Bigger men usually spell trouble for smaller individuals because they have physics on their side. For example, when an object with less mass collides with an object of significantly more mass, the smaller object would receive significant recoil and likely bounce back. If it was the other way around, the object of more mass would transfer a significant portion of its energy to the smaller mass, which would move in response to the force.
This is why keeping bigger individuals physically restrained can be difficult even if they have the same relative strength. However, the neck is always vulnerable no matter how large a person is, and can be used to restrain a much bigger person as a last resort.