Algorithmic interactive music generation in videogames

A modular design for adaptive automatic music scoring

  • Alvaro E. Lopez Duarte University of California, Riverside


In this article, I review the concept of algorithmic generative and interactive music and discuss the advantages and challenges of its implementation in videogames. Excessive repetition caused by low interactivity in music sequences through gameplay has been tackled primarily by using random or sequential containers, coupled with overlapping rules and adaptive mix parameters, as demonstrated in the Dynamic Music Units in Audiokinetic’s Wwise middleware. This approach provides a higher variety through re-combinatorial properties of music tracks and also a responsive and interactive music stream. However, it mainly uses prerecorded music sequences that reappear and are easy to recognize throughout gameplay. Generative principles such as single-seed design have been occasionally applied in game music scoring to generate material. Some of them are complemented with rules and are assigned to sections with low emotional requirements, but support for real-time interaction in gameplay situations, although desirable, is rarely found.
While algorithmic note-by-note generation can offer interactive flexibility and infinite diversity, it poses significant challenges such as achieving human-like performativity and producing a distinctive narrative style through measurable parameters or program arguments. Starting with music generation, I examine conceptual implementations and technical challenges of algorithmic composition studies that use Markov models, a-life/evolutionary music, generative grammars, agents, and artificial neural networks/deep learning. For each model, I evaluate rule-based strategies for interactive music transformation using parameters provided by contextual gameplay situations. Finally, I propose a compositional tool design based in modular instances of algorithmic music generation, featuring stylistic interactive control in connection with an audio engine rendering system.


Ames, C. (1987). Automated Composition in Retrospect: 1956-1986. Leonardo, 20(2), 169-185.

Ames, C. (1989). The Markov Process as a Compositional Model: A Survey and Tutorial. Leonardo, 22(2), 175-187.

Brent, R. (2007). Some long-period random number generators using shifts and xors. ANZIAM J. ANZIAM Journal, 48, C188–C202.

Brooks, F., Hopkins, A., Neumann, P., Wright, W. (1993). “An experiment in musical composition.” In S.M. Schwanauer, D.A. Levitt (Eds.) Machine Models of Music. MIT Press, Cambridge, Mass. ISBN 0-262-19319-1

Burraston, D., & Edmonds, E. (2005). Cellular automata in generative electronic music and sonic art: a historical and technical review. Digital Creativity, 16(3), 165-185.

Chomsky, N. (1956). Three models for the description of language. New York.

Chomsky, N. (1968). Syntactic structures. The Hague: Mouton.

Cope, D. (1996). Experiments in musical intelligence. Madison, WI: A-R Editions.

Cope, D. (2008). Hidden structure: music analysis using computers. Middleton, WI: A-R Editions.

Cope, D. (2000). The algorithmic composer. Madison, WI: A-R Editions.

Eigenfeldt, A. (2009). The Evolution of Evolutionary Software: Intelligent Rhythm Generation in Kinetic Engine. Proceedings of EvoMusart 09 – European Conference on Evolutionary Computing, Tubingen, Germany, pp. 498-507. Berlin: Springer.

Gardner, M. (1970). Mathematical games: The fantastic combinations of John Conway’s new solitaire game “live.” Scientific American, 223, October, 1970.

Hiller, L., & Isaacson L. Experimental Music (New York: McGraw-Hill, 1959; re-printed Greenwood Press, 1979)

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.

Lerdahl, F., & Jackendoff, R. (1983). A generative theory of tonal music. Cambridge, Mass.: MIT Press.

Miranda, E. (Ed.). (2011). A-life for music: music and computer models of living systems. Middleton, WI: A-R Editions.

Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foundations of machine learning. Cambridge, MA: MIT Press.

Niazi, M., & Hussain, A. (2011). Agent-based computing from multi-agent systems to agent-basedmodels: a visual survey. Scientometrics, 89(2), 479-499.

Nierhaus, G. (2009). Algorithmic composition: paradigms of automated music generation. Wien; New York: Springer. Retrieved from

Norris, J. (1998). Markov chains (1st public edition). Cambridge, UK; New York: Cambridge University Press.

Novák, V., Perfilieva, I., & Moko, J. (1999). Mathematical principles of fuzzy logic. Boston: Kluwer Academic.

Pearson, K. (1905). “The Problem of the Random Walk”. Nature. 72 (1865): 294. doi:10.1038/072294b0

Sweet, M. (2015). Writing interactive music for video games: A composer’s guide. Upper Saddle River, NJ; Boston; Indianapolis; San Francisco; New York; Toronto; Montreal; London; Munich; Paris; Madrid; Cape Town; Sydney; Tokyo; Singapore; Mexico City: Addison-Wesley.

von Neumann, J. Various techniques used in connection with random digits. A. Householder, G. Forsythe, and H. Germond (eds.), Monte Carlo Method, National Bureau of Standards Applied Mathematics Series, 12 (Washington, D.C.: U.S. Government Printing Office, 1951): 36-38.

Wolfram, S. (2002). A new kind of science. Champaign, IL: Wolfram Media.

Zicarelli, D. (1987). M and Jam Factory. Computer Music Journal, 11(4), 13-29.

How to Cite
Lopez Duarte, A. (2020). Algorithmic interactive music generation in videogames. SoundEffects - An Interdisciplinary Journal of Sound and Sound Experience, 9(1), 38-59.