To realize how deeply rooted (ha!) the taboo of food gardening is, one must simply look to the first book of the Hebrew Bible, where poor Adam and Eve get the ultimate God-smack for essentially engaging in edible forest gardening.
Let's empathize with God's perspective for a moment here.
What were those naked hippies thinking, anyway? They think they can just go around picking fruit off of trees in some commonly owned garden-space without having to pay for it? From now on these freeloaders are going to have to TOIL to get their food. And not in forests, either. They are banned from forests! They gonna have to till the soil in them OPEN FIELDS. Yes. Oh, and I curse the soil forever. Love, God.
Before you become offended, please know that I too am offended by what I just said. Finishing my undergrad in the UVM Religion department (one of the first "comparative" religion departments at any state university), I am merely engaging in some well-earned intellectual hedonism. I beg your forgiveness.
Moving on, let us examine the economic power of populist food-gardening. To favor a dramatic assessment, the whole edifice of the food-economy (or the edible-resource economy) depends on one thing to keep doing the corrupt and exploitative things it does. I believe that thing is the fear factor. It is the uncertainty people feel when considering the prospect of obtaining food in ways other than what they are used to - and this includes the prospect of growing it. It seems, even, that the harmless little SEED is what is feared the most, since plenty of people seem to love the idea of "harvesting" meat from nature with guns. In psychological terms, hunting is more immediately rewarding than gardening; it provides greater returns sooner. Once you have a weapon and know how to use it, there is a high probability that when you point and shoot at an animal, that animal will become food for you. The risk and inputs are low.
When you garden, to whatever scale, you have to cultivate a system. And sometimes nature is on your side; you have the insurance of rather dependable things like the fact that the sun will shine every once in a while and green things dig that. But a lot of the time nature is kind of NOT on your side because you see, nature is a complex system where things are constantly competing for sunlight, water and food. When you create an artificial environment in nature, you are technically creating an agroecosystem - a simplified version of an ecosystem. The tricky part is this: the agroecosystem is encompassed by the more-complex ecosystem, and if we evoke the laws of thermodynamics, there's gonna be a WHOLE LOTTA ENTROPY that wants to come rain on your parade. Also, plants can't water themselves when they need it or move to areas of more or less sunlight... one thing that seems to make growing/harvesting meat more appealing to peeps.
When analyzing a function within any evolved system - be it an ecosystem or an economic system - many philosopher types like to invoke the question "cui bono?" or "who benefits?" So who in our economic ecology benefits from you NOT growing your own food (or buying it from your local farmer?) Well, I'll leave that as an open question for you to deploy in the field... but I can tell you with certainty -- not you!