(Language and Symbolic Systems, Yuen Ren Chao)
Phonetics may be compared to the lines of longitude and latitude drawn on the globe and phonemics to the mapping of actual continents and oceans and countries. The precise way in which the divisions are made is to some extent arbitrary. During the French Revolution, it was attempted, though without success, to change the quadrant of 90° into 100° decimal degrees. But certain features, such as the North and South Poles and the Equator, are a part the nature of things. Similarly, stops and continuants, voice and voicelessness are natural variables found in all human speech. In phonetics one tries to anticipate, after a broad survey of the accessible languages of the world, all the necessary distinctions and set up standard points (such as the cardinal vowels and the divisions along the roof of the mouth) and then assign the actual sounds of any language under study to the nearest standard points, with the appropriate IPA symbols, so as to have an accurate representation of the sound of that language.
(...) There is therefore really no conflict between the two points of view about a phoneme being a group and being a set of features. They are two sides of the same coin. In fact Bernard Russell long before the theory of phonemes had a theory of equivalence between the property of a class and class membership. To paraphrase his "principle of abstraction", we might say that humanity (in the abstract) is humanity (mankind). Applied to phonemics, we might say that the common property of a number of different sounds which makes them members of one phoneme consists in the fact that they belong to this class.
This evident circularity in characterizing the property of a phoneme by its members is unavoidable because if you stipulate that members of a phoneme must be phonetically similar, a condition often included in the definition of a phoneme, then you run into cases where what to foreigners seem very different sounds belong to the same phoneme and the differences are hardly noticeable to the native speaker. The solution to this problem, as to all solutions in science, is to make your circle of circularity as big as possible. One important step in carrying this out is to look for cases of what is known as 'complementary distributions'. If a dorsal stop occurs always with the palatal articulation when followed by a front vowel (as in key) and always by a velar articulation when followed by a back vowel (as in call) but never the other way round, there is a case of complementary distribution.
But complementary distribution alone is not sufficient to determine what sounds go together to be members of one phoneme. There must also be overall symmetry in the organization of sounds into phonemes. For example, besides the complementary distribution of the palatal consonant in key and the velar consonant in call, there is also a parallel difference in quality in he and fall. Likewise, we have parallel differences in the g of geese and gall. Thus, we arrive at a neat and symmetrical system of groupings. Similarly, not only is k aspirated when initial and stressed and unaspirated when following an s, but the same is true of t in term and steam and of p in peak and speak. On the other hand, no one would seriously make one phoneme out of two sounds [h] and [n] in English simply because [h] always occurs as a syllabic initial and [n] always as a syllabic ending. Not only are the two sounds extremely dissimilar phonetically, but there is no other parallel case of complementary distribution in the sounds of English.
To summarize, then, a phoneme can be defined as one of an exhaustive list of systematized classes of phonetically related sounds in a language, such that every form in the language can be given as a (usually serially ordered) set of one or more of these classes. As definitions go in matters concerning human behaviour, this definitions is no more than a summary of usage and procedure among linguists and the definition does not even guarantee that its applications will always result in one unique system for any give language.