While clearly, warm beverages will warm your body up first, we also need to look at this from a medical point of view. There is evidence that warm beverages do cool you down more than cool beverages.
Deutsche Welle has an easy-to-read media article that quotes some experts:
According to Professor Peter McNaughton, a neuroscientist at the University of Cambridge, consuming hot beverages, such as tea or hot water, will raise your core body temperature. And this makes you to sweat at an increased rate.
Nerves in our mouths and in our upper digestive tract respond to the heat of the beverage, stimulating the brain to produce more sweat. And as it evaporates, the sweat effectively cools you down.
A scientific study from 2012 performed an experiment getting 9 exercising men to drink water at different temperatures, and measured their body heat:
Under conditions permitting full sweat evaporation, body heat storage is lower with warm water ingestion, likely because of disproportionate modulations in sweat output arising from warm-sensitive thermosensors in the esophagus/stomach. Local temperature changes of the rectum following fluid ingestion exacerbate the previously identified error of thermometric heat storage estimations.
The keyword is disproportional, i.e. the body is overcompensating the seemingly too high core temperature. The regulation is done in the usual way (i.e. sweating more).