Urine is not sterile, not while in the bladder and certainly not when it comes out.
Contrary to dogma that urine is sterile in the absence of a clinical urinary tract infection (UTI); our research team and others have recently shown the existence of a urinary microbiota in individuals with and without lower urinary tract symptoms
From "Bacteria isolated from blood, stool and urine of typhoid patients in a developing country" (note that while the paper itself is about people with "Overactive Bladder syndrome", the conclusion about urine not being sterile holds true for the control group in this study as well.
However, it is very unlikely you could get or give someone else some kind of disease through the kind of hypothetical contact you are talking about in your question. The South Australian health ministry for example lists only one disease potentially spread through urine : Cytomegalovirus [source]. But while the test for this infection can be done through analysing urine, I could not find any evidence that there has ever been an infection that resulted from exposure to urine as opposed to saliva, semen or one of the other transmission routes.
Another disease found in urine is typhoid :
Thirteen (14.1%) out of 92 urine samples were positive for bacterial growth
From ["Bacteria isolated from blood, stool and urine of typhoid patients in a developing country"]. Whether the level in these samples would be enough to lead to infection even if ingested is unclear to me.
No substance in your urine is actively dangerous to you just by touching it. Even drinking it is generally considered to be safe, unless done too often (it does contain products your body is trying to get rid off).
while urine’s purported healing properties have yet to be confirmed by rigorous research, drinking small amountsof your own urine is unlikely to produce serious harm if, for some reason, you are so inclined.[source]
Drying will possible reduce the amount of bacteria in the urine, just as it does for saliva.
In summary, no, urine is not sterile, but it also doesn't seem to be a "usual" route of infection for many (or maybe any) diseases. Furthermore, having a spot of it dry on your clothes and a hypothetical person then touching it is bringing the infection risk down even more.