It's not a huge hassle. I've done it with my daughter. Had some sandwiches packed in a cooler, and it is a nice break from park food.
Here's one problem though. In the morning during very busy times, they put cars on the top floor of the garage in the sun, and work down. If you ended up on the garage top, you'd have to come back at lunch, exit, then re-enter the garage with your parking stub so you end up between floors. Dropping/picking up the kids is not necessary, just keep walking to your van.
You're gonna get worn out no matter what you do. Going between IOA/USF and being on your feet will make anyone tired. I'd say to go ahead and pack the cooler.