Do we get sick when we get wet in rain while nothing happens when we stand in the shower?
'Cold' is just a name given to the rhinoviruses that cause cold symptoms, the idea that going out into the cold would cause a cold was just a traditional idea, not necessarily based on anything other than biased observations.
All this said, it is true that colds, and more specifically flu, are more prevalent in winter months, and that this is true, even in opposite hemispheres of the planet, where winter occurs at opposite times of year. This provable seasonality of flu doesn't show that the cold temperature causes flu, though. It could be that cold weather aligns with flu season because, for instance, people stay indoors, together, incubating viruses during this time.