With a toddler I would stay on the boards. Think about it beach, bathroom, bed all in 20 yards. If your worried about noise the higher the street # the quieter it is. I say the boards simply for the conveniences. Your condo/hotel is a close restroom for a toddler, plus lunch and naps if he/she still takes them. Plus night time entertainment is easy, walking up & downs the boards with a stroller (assuming they use one) would be easier if you could roll out of your room onto the boards. No loading a car with toddler stuff.
We love the boardwalk and thats what we do every night when we are in OC. Last year when we stayed on 51 st it was a pain to load the kids in that car, drive to boardwalk, find parking, etc. And our girls are old enough to take care of themselves. We even tried the bus a couple of nights, again it was a pain. The bus was crowded, we had to wait for a bus with room on it, people rushing the door to get on and they ran in front of you. It didn't even seem like OC, having to drive to the boards. For us the boardwalk just makes sense.
I have a one year old we are planning a trip at the end of Aug. for Labor day. Where would you stay, would you stay on or off boards. We have done both as a couple and keep weighing the pros and cons of both.