What could be wrong with my 2 year old son? He has had a fever ranging from 101-104 the last three days.?

My son started daycare this past summer. Until then he has pretty much been healthy. (1-2 fevers a year, as well as a cold) Since the start of daycare he just seems sick all the time. He has probably had 4-5 fevers in the last four months. The fever is normally accompanied by other systems. His latest started this past Saturday. He has had a fever now for the last four days ranging from 101-104. Tonight it was 101.5 but we are now soon going on day 4. He does have a runny nose and sounds really congested when he sleeps. His doctor is saying it is nothing to worry about, it is just him getting used to all the germs at daycare. My question is, is it? Is it just him going to school for the first time or is something else wrong. I am a very nervous parent. I hate seeing my kids sick. Anyone have anything similar happen?