In part because of this thread, I recently learned two related facts that I absolutely cannot believe a nerd like me didn't learn until I was 50!
1) It turns out that the length of a "day" is actually not exactly 24 hours each day, if you define a day not by the human-imposed time measurement, but rather by the time it takes for the earth to actually make one full rotation (relative to the sun). [This relates to the earth following an elliptical rather than circular orbit.]
2) This actually causes the sunrise time to continue to get later (in terms of our human-imposed time) for quite a while after the winter solstice. This is because the entire window of daylight is drifting later. (Reason: our 24-hour day and the astronomical day aren't the same length.) So right after the winter solstice, both the sunrise and sunset times get later (and the length of daylight is also increasing).
Depending on your latitude, this can continue either for just a few days, or well into January.
So if you were thinking "I thought the days were getting longer, why does it seem like the sun is coming up later?" -- the answer is that both were happening!