The warning, from 18:00 GMT on Christmas Day until 12:00 GMT on 27 December, comes with predictions for icy mornings and freezing nights.
The UKHSA said the warning impact would be low, with minor impacts on health and social care services.
Those at most risk were the elderly with multiple health conditions and on multiple medications, it added.
Met Office meteorologist Becky Mitchell confirmed many would be waking up to a frosty 25 December morning.
She said: “It will turn colder from Christmas Eve, with a strong wind across much of England and Wales, so temperatures on the thermometer will be a little bit below average: around 6C or 7C [43F or 45F] as a max, generally.
“And then, when you add on that wind chill of an easterly wind, it will probably feel more like freezing from Christmas Eve onwards.”
Ms Mitchell said clouds could accumulate in some southern counties on Boxing Day but elsewhere would remain largely dry with sunny spells.
She said: “It’s a bit colder than average, not unusual for the time of year, but, in terms of averages, it would be a bit below.
“The weather’s all very similar really, out towards New Year’s Eve. “