We may categorize phonetic facts into four groups: random, individual, dialectal, and universal. Random (low-level) performance facts include effects of coughing, true speech errors, etc. Individual facts include vocal-tract size and shape (from which are derived the frequency scale of formant space for a given speaker), some voice quality features, etc. Language- or dialect- or style-specific facts include the degree of rounding on labial consonants (Sapir 1921:43), many details of phonetic vowel quality (Labov, Yaeger, & Steiner 1972), etc. Universal facts include, for example, the absence in language of voiced glottal stop phonemes, or the presence in all languages of voiced sounds,4.1 etc.
``Linguistic'' is a word with two contrary connotations: ``specific to language (in general),'' and ``specific to (particular) languages.'' While universalists prefer the first connotation, the second is more useful in the context of phonetic description, where on the one hand, universal facts are most likely to be physical facts, not linguistic ones; and on the other hand, those facts that are characteristic of particular dialects or languages and not of others must be considered linguistic facts.
The linguistically relevant difference between speakers of one dialect and speakers of other dialects is that they have learned to speak the dialect in question. Dialect-specific phonetic patterns must be learned by speakers of the dialect in question. If speakers had not learned these patterns in some way, they would apply to all speakers, including those who have not learned to speak that particular dialect.
In short, those phonetic details that are specific to particular languages or dialects must be accounted for as part of linguistic phonetic descriptions. More universal principles of general phonetics may account for other phonetic details, which need not be specified in the grammar of a particular language. Phonological (or linguistic-phonetic) theory and description can stop only where general phonetics begins -- that is, well within the purview of contemporary phonetics, which studies both general and language- or dialect-specific phenomena.
On the other hand, there may be intricate interactions between universal and dialect-specific principles in the phonetic pattern of a language, so that a full understanding of its surface-observable phonetic characteristics would require simultaneous study of both kinds of principles.