Poly - we stayed at Moorea on the end by the TTC and loved that location. We could just hop over to the TTC and grab the ferry to MK or Monorail to EP. It was a longer walk to the main building, but we didn't mind that as we weren't over there much. We also didn't have any issues with hearing the boat horns (we were on the back side of the building - no water view for us). If you are going to be using the pool a lot, you might want to get a location closer to the main building.
SSR - If you have a preferred room booked, I would request the Congress park section. If you have a standard room booked, I would recommend either Grand Stand (the boat to DS is somewhat close to that) or the Paddock and request a location that is as close to the end of that section (towards the Congress park section) as possible. We had a room that was in the Paddock for our last trip and it was our first time staying in that section. I was a bit nervous about it until I saw that it was actually really close to the bus stop for Congress Park and thus close to the Congress Park walkway that leads to DS. We ended up being really happy with the location. It was a bit of a hike to get to the Carriage House, but it was a lovely walk and it was fun to stop and watch the turtles in the pond as we were crossing the bridge.
If you don't want to deal with the slow elevators, you could request a lower floor. We usually end up with a ground floor at SSR (not by request, that's just how the chips landed). When we were at Poly, we were on the 3rd floor. I liked being higher up and since it's only two flights of stairs, we usually walked them instead of taking the elevators.
Have a great trip!!