Since taste by whichever definition and in whichever modality is essentially a matter of perception, a good place for further examination would be Oliver Sacks's recent book The Mind's Eye (New York: Alfred A. Knopf, 2010; ISBN 978-0-307-39809-3).

Among other pithy observations Sacks notes: "It [facial recognition] is similar with many other capacities, from stereo vision to linguistic power: some predisposition or potential is built in genetically but requires stimulation, practice, environmental richness, and nourishment if it is to develop fully. Natural selection may bring about the initial predisposition, but experience and experiential selection are needed to bring our cognitive and perceptual capacities to their full realization." (pp 101-102)

By the bye, Sacks himself 'suffers' considerably from prosopagnosia (inability to recognize faces) and is one of the case studies in this fascinating book.