Building Site

 
In the city you will rarely have much choice of building site, although on larger lots you can shuffle the house a few feet in any direction.  Uniform zoning setback often prevents you from making the best use of a lot, for example if the street is to the north, you might want to push the house closer to the street to increase the amount of sunny back yard (or in hot climates do the reverse), but zoning laws generally require a sizable front yard setback.

On the other hand, zoning allows houses to be as close as 10 feet apart on the side (ie 5 feet in each lot), which means that if one is north of the other, the southern house can almost completely block the sun to the other house, which is a tragedy.

For those who have a few acres, especially if there are quite a few,  there is an additional emotional component of deciding which locations are cherished, and possibly sacred and which are less so.  Contrary to what is often done, the best place for the house is typically not the nicest place on the lot, but a place that looks at it, or sometimes the least nice place..after all why plow the nicest spot under?  Sometimes even in view of the nicest spot is not the right place: consider your impact on your neighbor, or how you'd feel if the house with a view of your nicest spot was your neighbors.  There is often a compromise location, still nice, but one that is not an eyesore from everywhere else.¹

Assuming you do have some latitude, here are some things to consider:

Sun: locate the house so it gets enough sun, and if summer shading it needed, try to place the house so it gets summer afternoon shade.  Since shady outdoor spaces aren't used in most climates (hot being the exception), push the house as far north as reasonable, leaving the south side for porches, patios etc.  In hot climates, face the house east if possible, or otherwise hide the west face from the hot afternoon sun.

Trees: mature trees are a wonderful asset, but placed wrong they can block solar gain during the wrong part of the day, and lower the output on both PV and solar hot water systems.

Neighbors: an ideal location makes is so that you pass each other by, but you don't look into each others houses. This can be fixed somewhat with fences and hedges, but big barriers tend to create a sense of isolation.  Its probably obvious, but people generally don't like houses, trees, or fences shoved on them.

Connection to site: eventually you want to think about what each room will be looking out on.  The common areas want more connection to the street, while private areas want more to look out on green with a lesser connection to the street.  The building needs to be integrated into the site so that it becomes a part of it.  This is partly done by choice of site, and partly by building outdoor spaces on the edge of the building that act as transitions between the building and the site.

Connection to the street: you want to be connected, so that guests can find you, and so you can connect with neighbors, but far enough that privacy is maintained.  How will the walk from the street go?  Where will the driveway be (if any)?

Hilltops:  the best view might be on top, but it's also the place of the most wind.  Houses on hilltops are not only exposed to the weather more, but they look unnatural.  Dropping the house down 30 or even 50 feet, preferable tucked into a little alcove will not only make it feel much more protected, but it will keep everyone from having to look at it.

Critical areas:  Avoid areas that flood, or are subject to landslide.  In general avoid riparian corridors, as they are best left for wildlife. Good farmland is best used for food: avoid building on fertile soil.  Never build on the best soil: instead build on the area of worst soil.

Views: in rural areas where sweeping views are available, the best building site is usually not in the place of the best view, but rather at some compromise place that has a good view, but is more protected from the elements, and doesn't stick out like a sore thumb in your neighbors view.

Dead Spaces: in urban areas you often get "dead spaces": places that are too small, lack privacy or are too dark.  Think about landscaping and how outdoor spaces will be used and try to find a use for everything.

Notes

1: no matter how beautiful you think your house is, others are not likely to agree if it spoils their view.