On a quiet back street in Newport, Rhode Island is the Pilgrim House Inn.  It is within easy walking distance of the harbor, shopping, restaurants, etc. Self-serve breakfast is served either indoors or weather permitting, on a balcony that gives a nice view of the water a few blocks away.