Processing of Multimodal Advertising Text by Social Media Users: The Role of Playback Speed of the Audio and Video Sequence

The article examines the mechanisms of multimodal text processing as a tool of professional advertising communication and is based on the dual coding theory and principles for reducing cognitive load. The aim of the research is to examine whether the parallel and asynchronous acceleration of verbal and non-verbal components of the text affect the success of its processing and memorization. The stimulus material consists of four videos united by a common theme of “Beauty Care” previously aligned in terms of length and readability of the text. Each video was recorded in four variants: 1) original format – normal speed of video and audio; 2) parallel acceleration of video and audio; 3) acceleration of video while maintaining the original speed of audio; 4) acceleration of audio while maintaining the original speed of video. In the experiment, after watching each video, the participants (N=100, females aged 16 to 35, native Russian speakers, active social media users) were asked to suggest key words for the video and to rate its attractiveness, complexity, memorability, and lengthiness on a scale from -2 to +2. The results demonstrate a complex relationship between the speed of presentation of visual and auditory components and subjective ratings of the videos. Normal speed delivery provides the best results for key word extraction and ease of perception. Synchronous acceleration of audio and video leads to cognitive overload and worsens content memorization. Participants rated the format with accelerated video at the original audio speed as the most attractive and easiest to remember. Significant correlations were found between subjective ratings of attractiveness, memorability, and complexity of multimodal text perception. Regression analysis results confirmed that not only the presentation mode but also the topic of the video influences the perception of multimodal text. The findings allow for optimizing professional communication strategies in the digital environment, taking into account the audience’s cognitive characteristics.

1. Ahn, D., & Chan, J.C.K. (2025). What Drives Student Engagement and Learning in Video Lectures? An Investigation of Instructor Visibility, Playback Speed, and Student Preferences. Applied Cognitive Psychology, 39(2), e70026. https://doi.org/10.1002/acp.70026

2. Almaev, N.A., & Gradovskaya, N.I. (2002). Subjektivnoe shkalirovanie i kontent-analiz v otsenke emotsionalno-affektivnoj komponenty diskursa [Subjective scaling and content analysis in gauging the emotional and affective discourse component]. In N.D. Pavlova (Ed.), Psikhologicheskie issledovaniya diskursa [Psychological studies of discourse] (pp. 18-39). Moscow: Per Se (in Russian).

3. Bennion, K.A., Ford, J.H., Murray, B.D., & Kensinger, E.A. (2013). Oversimplification in the Study of Emotional Memory. Journal of the International Neuropsychological Society, 19(9), 953–961. doi:10.1017/S1355617713000945

4. Bratash, V.S., Riekhakaynen, E.I., & Petrova, T.E. (2020). Creating and processing sketch­ notes: A psycholinguistic study. Procedia Computer Science, 176, 2930-2939. https://doi.org/10.1016/j.procs.2020.09.210

5. Chernigovskaya, T.V., Petrova, T.E., Alexeeva, S.V., Dubasava, A.V., Prokopenya, V.K., & Chernova, D.A. (2018). Vzglyad kota Shriodingera: registraciya dvizhenij glaz v psikholingvistich­ eskikh issledovanijakh [The gaze of Schroedinger’s cat: eye-tracking in psycholinguistics]. St. Peters­ burg: St. Petersburg University Press (in Russian). https://doi.org/10.21638/11701/9785288059292

6. Dolcos, F., & Cabeza, R. (2002). Event-related potentials of emotional memory: Encoding pleasant, unpleasant, and neutral pictures. Cognitive, Affective, & Behavioral Neuroscience, 2, 252– 263. https://doi.org/10.3758/CABN.2.3.252

7. Ezhova, E.N. (Ed.). (2010). Reklama: tekhnologii vozdejstviya [Advertising: influence tech­ niques]. Stavropol: Stavropol State University (in Russian).

8. Fiorella, L., & Mayer, R.E. (2016). Effects of observing the instructor draw diagrams on learn­ ing from multimedia messages. Journal of Educational Psychology, 108(4), 528–546. https://doi.org/10.1037/edu0000065

9. He, Y., & Yu, J. (2025). Exploring the use of double-speed video playback in a fast-paced society through structural equation modeling. Scientific Reports, 15, 24481. https://doi.org/10.1038/s41598-025-09525-7

10. JASP - a fresh way to do statistics. (2025). JASP - Free and User-Friendly Statistical Software. https://jasp-stats.org/

11. Kibrik, A.A. (2010). Mul’timodal’naya lingvistika [Multimodal linguistics]. In Yu.I. Aleksan­ drov, V.D. Solovyev (Eds.), Kognitivnye issledovaniya: sbornik nauchnykh trudov. Vyp. 4 [Cognitive studies: a collection of scientific papers. Issue 4] (pp. 134-152). Moscow: Institute of Psychology of the Russian Academy of Sciences (in Russian).

12. Konovalova, A., & Petrova, T. (2023). Pun processing in advertising posters: evidence from eye tracking. Journal of Eye Movement Research, 16(3), 1-17. https://doi.org/10.16910/jemr.16.3.5

13. Konovalova, A.A., Petrova, T.E., Bespalova, K.V., & Mitina, A.V. (2021). Otsenka reklam­ ny`kh plakatov podrostkami i vzroslymi: psikholingvisticheskoe issledovanie na materiale russkogo yazyka [Subjective scoring of advertising posters by Russian teenagers and adults: a psycholinguistic study]. Kognitivnye issledovaniya yazyka [Cognitive Studies of Language], 3(46), 358–362 (in Rus­ sian).

14. Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contem­ porary communication. London: Arnold Publishers.

15. Kulikova, K.S. (2017). Vliyanie tempa rechi na vospriyatie radiosvodok studentami-anglofo­ nami [The influence of speech rate in radio news bulletins perceived by Anglophone students]. Vest­ nik Moskovskogo gosudarstvennogo lingvisticheskogo universiteta. Gumanitarny`e nauki [Vestnik of Moscow State Linguistic University. Humanities], 5(776), 46–56 (in Russian).

16. Luzina, L.G. (2011). Verbalnye i neverbalnye sredstva vozdejstviya v reklame [Verbal and non-verbal means of influence in advertising]. Reklamnaya kommunikaciya: lingvokul`turny`j aspect [Advertisement and communication: a linguocultural aspect] (pp. 41-48). Moscow: INION RAN (in Russian).

17. Massey, G.R., Wang, P.Z., Waller, D.S., & Lanasier, E.V. (2013). Best–worst scaling: A new method for advertisement evaluation. Journal of Marketing Communications, 21(6), 425–449. https://doi.org/10.1080/13527266.2013.828769

18. Mayer, R.E. (2009). Multimedia Learning, 2nd edition. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511811678

19. Mayer, R.E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learn­ ing. Educational Psychologist, 38(1), 43–52. https://doi.org/10.1207/S15326985EP3801_6

20. Murzin, L.N., & Shtern, A.S. (1991). Tekst i ego vospriyatiye [Text and its perception]. Sverd­ lovsk: Ural University.

21. Notkin, I.M. (2023). Vliyanie kreativa na effektivnost` reklamny`kh kampanij [The influence of creativity on the efficiency of advertising campaigns]. Prakticheskij marketing [Practical Market­ ing], 12(318), 46–51 (in Russian). doi:10.24412/2071-3762-2023-12318-46-51

22. Paivio, A., & Clark, J.M. (1991). Dual coding theory and education. Educational Psychology Review, 3, 149–210. https://doi.org/10.1007/BF01320076

23. Petrenko, V.F. (2010). Osnovy psikhosemantiki. 3-e izd., dop. [Basics of psychosemantics. 3d edition]. Moscow: Eksmo (in Russian).

24. Petrova, T.E. (2016). Eye movements in reading the texts of different functional styles: Evi­ dence from Russian. In I. Czarnowski, A.M. Caballero, R.J. Howlett, L.C. Jain (Eds.), Intelligent De­ cision Technologies 2016. Smart Innovation, Systems and Technologies, 57, 285-298. Springer, Cham. https://doi.org/10.1007/978-3-319-39627-9_25

25. Petrova, T.E. (2021). How does text format influence the reading process? Evidence from Russian texts. Perception, 50, S1, 105.

26. Petrova, T.E., & Puchkova, E.V. (2023). Vliyanie formata predyavleniya na uspeshnost` vospriyatiya poeticheskogo teksta [The influence of text format on the processing of a poetic text]. In E.V. Pechenkova, M.V. Falikman, & A.Ya. Koifman (Eds.), Kognitivnaya nauka v Moskve: novy`e issledovaniya: conference proceedings (pp. 381-385). Moscow: Buki Vedi, Moscow Institute of Psy­ choanalysis (in Russian).

27. R Core Team (2025). R: A Language and Environment for Statistical Computing. R Founda­ tion for Statistical Computing. https://www.r-project.org/

28. Serdobintseva, E.N. (2010). Struktura i yazyk reklamny`kh tekstov: uchebnoe posobie dlya vuzov [Structure and language of advertising texts: a coursebook for university students]. Moscow: Flinta Nauka (in Russian).

29. Shatin, Y.V. (2002). Postroenie reklamnogo teksta [Construction of advertising text]. Mos­cow: Berator-Press (in Russian).

30. Simakova, S.I. (2015). Vliyanie novykh tekhnologij na vizualnyj kontent zhurnalistskikh ma­ terialov [Impact of new technologies on visual journalism content]. Bulletin of Chelyabinsk State Uni­ versity, 5(360), 163–169 (in Russian).

31. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer.

32. Tharumalingam, T., & Risko, E.F. (2025). Altering the playback speed of recorded lectures as a learning technique: Examining student practices, motivations, and beliefs. Journal of Research on Technology in Education, 1–20. https://doi.org/10.1080/15391523.2025.2459130

33. Tharumalingam, T., Roberts, B.R.T., Fawcett, J.M., & Risko, E.F. (2025). Increasing Video Lecture Playback Speed Can Impair Test Performance – a Meta-Analysis. Educational Psychology Review, 37, 35. https://doi.org/10.1007/s10648-025-10003-9