The reason is provided by Dr Christopher Hollingsworth, a member of the NYC Surgical Associates. He stated:
The navel stimulates not only the skin, but also the fibres of the “inner lining” of your abdomen, so when you put your finger in your belly button, it sends a signal to the deeper fibres in the abdominal cavity and to your spinal cord. Your spinal cord at this level also transmits signals coming from your bladder and urethra; it's almost the same sensation, and you interpret it as discomfort in your bladder.
This nerve response is therefore read by your body as a need to pee, which then transmits this unpleasant sensation. You learn something new every day!