Yeah, sound is definitely not the whole story. I was just reading this paper on Combinatoriality and Compositionality, and they talk some about the importance of multimodal data when studying communication.

Multimodal communication in humans can take on the form of co-verbal gesturing, where spoken utterances are combined with movements of the arms and hands (Morgenstern, 2014). In apes, multimodal communication can include the co-occurrence of distinct facial expressions with manual gestures, such as variants of the reach gesture (Oña et al., 2019), the integration of visual and acoustic features in behaviors, such as lip-smacking (Micheletta et al., 2013), or the combination of social calls with different gestures (Genty et al., 2014). Bird song also can show variability in call combinations (Suzuki et al., 2019). For instance, bird songs often combine with coordinated visual displays whose performance can affect listener response (Girard-Buttoz et al., 2020; Williams, 2004). In all cases, the meaning of the units combined varies depending on how they are joined into larger aggregates, as well as how they are used in differential sociocultural settings.