While we weren't there in November, we were at HHI at the beginning of March, which can be similar weather-wise.
We had an unseasonably warm week, with temps reaching the mid-80's. We enjoyed swimming in th eheated pools and wading in the ocean (which was actually comparable to the water temps we are used to in NJ in the middle of the summer). There were quite a few families playing on the beach, and some even venturing into the water.
In addition to the beach activities, we enjoyed the following activities: cycling, kayaking, a dolphin cruise, a behind the ears tour, relaxing on hammocks, pushing one another on the swings, shopping, and enjoying the area restaurants.
We also don't golf, but found plenty to do in the off-season, and didn't want to leave at the end of our stay.