When will we say "Enough!"?

This last weekend, I attended a family reunion for my wife's first cousins.  During one of the evenings, the discussion turned to building permits, to home owner associations, and other such nonsense.  Among the stories:

• Someone built a shed five or six square feet beyond the limit before requiring a building permit.  Because a building permit was too expensive, the person tore down the building, and then built a two-story shed exactly one square feet below the minimum.  Furthermore, this person spent most of his money building the first shed, and so this new shed was built a plank at a time, producing a significant eyesore for the neighborhood.
• Another person added a room to his house.  In order to afford the permit, he had to put in two little windows, and only four outlets per wall--but quietly made provisions to expand the window, and add more outlets, after he was finished.  While the inspector was angry afterward, this person made it clear to the inspector, and later, to the judge, that it was legal to alter his home without a building permit.
• A person managed to move into a new neighborhood without signing Home Owner Association papers; thus, he painted his house freely, without regard to HOA restrictions.  Somehow, the HOA managed to find a loophole anyway (I can't remember the details), and required this person to paint his home using Approved Colors(TM).  So he did:  he chose a drab brown, and then painted his entire house that color.  He then refused to water his grass, and when the HOA complained, he just painted it green.

We swapped several more stories like this, which I won't go into at this time--some of which I don't even remember--but throughout all this, I kept asking myself:  when will we say "Enough!"?  When will we stand up, and demand our Liberties, which is the birthright of every American?

In particular, why do we put up with Home Owner Associations?  I've seen these neighborhoods, and they are drab, boring, and soulless...but then, I suppose that's what you get when you care more about your property walues than you do about your neighbor's freedoms.

When will we say to our local governments, "I don't need your permission to build on my property!"?

And when will we say to local, state, and even federal, inspectors:  "I will build to whatever code I see fit to build to--and if I build a fire hazard, or a house that will collapse in an earthquake, or a house where the ceiling collapses--then I will take responsibility for it myself, up to and including manslaughter for the death of those I love!"?

At what point will we start to stand up to these subtle--and, in many cases, not so subtle--attacks on our liberties?