we have brought our kids to certain fests since 3 months old. But I am very particular on which fests. Cate's first was Vibes 2005 at 3 mos. Jessica's first was Vibes 07 at 6 mos but vibes was smaller then. We stick to the worm fests & greyfox now- they are very family friendly.
We definitely go to bed earlier now. Many couples will switch off nights, allowing each partner to get a party night. I rarely last long so I usually go out & 'party' & then I go to bed so Frank goes out after that 
I think they key is that at least 1 of you is 'sober' enough to deal with things that could potentially go wrong in the middle of the night. Kids get sick, shit happens & someone has to be able to deal.
Congrats on your new lil one!! and bring earplugs of the baby