Sydney is the oldest, largest, and most beautiful of all Australian cities. While expensive, there are still plenty of things to do in Sydney on a budget.