It depends...what does your family like to do? If you enjoy museums, then there's LOTS of museums in Balboa Park - everything from Natural History to Aerospace to Fine Art/Modern Art. The Space Theater is there, too - there's an IMAX theater as well as a science center, which has interesting interactive activities for kids. And there's the botanical gardens, rose garden, native plant garden, sculpture garden...
If you like Navy stuff, the Midway is a carrier that's been turned into a museum on the waterfront. Really well done, from what I've heard - I still haven't visited. Nearby is the Star of India, one of the old "tall ships". You can take a harbor cruise from that area, too. Or take the ferry to Coronado Island. There's some shopping and restaurants at the ferry landing on the Coronado side, but you're not close enough to the Hotel del Coronado or the beaches to do much else.
If you like the beach, there's lots of great beaches - Coronado (which is next to the Hotel del Coronado), La Jolla Shores, Mission Beach (there's a small amusement park, Belmont Park, there, too). Or there's Mission Bay, which also has miles of beaches, but on the bay and not the ocean. There's lots of grassy areas for picnicking, though.
If you like to hike, Torrey Pines State Park is right on the coast, and has several miles of hiking trails through all the native vegetation. You used to be able to hike down to the beach, but I think they had to close that trail last winter.
Old Town State Park has a number of old homes/exhibits that tell about San Diego's early history. There's lots of shopping and restaurants in that area, too.
The Gaslamp District downtown has tons of restaurants and nightclubs.
Seaport Village is full of lots of picturesque shops right along the bay - there's a carousel there, and a number of restaurants, also. It's a nice place to stroll around.
There's Cabrillo National Monument, which (on a clear day) has a great view of downtown, Coronado Island, and the ocean. The visitor center there has exhibits on the discovery of San Diego Bay by Juan Cabrillo.
Hope that helps.
Laura