It depends. Two summers ago, we shortened our vacation by a day to make it home in time for a family wedding. The vacation was planned before we heard the dates of the wedding, but we could change it with little financial penalty. If it had cost us a lot (other than having one day less of vacation), I probably would have said "I'm sorry, it's already booked." But since we could change, we did... and I don't regret it.
It sounds like you want to go to the party -- and it wouldn't cost you a lot of money, and you wouldn't have to cancel your vacation to do it -- so I probably would. However, if you choose not to, I think that would be understandable, too.