Over the past 60 years, artificial intelligence has grown from an academic field of research to a ubiquitous array of tools used in everyday technology. Despite its many recent successes, certain meaningful facets of computational intelligence have yet to be thoroughly explored, such as a wide array of complex mental tasks that humans carry out easily, yet are difficult for computers to mimic. A prime example of a domain in which human intelligence thrives, but machine understanding is still fairly limited, is music. Over recent decades, many researchers have used computational tools to perform tasks like genre identification, music summarization, music database querying, and melodic segmentation. While these are all useful algorithmic solutions, we are still a long way from constructing complete music agents able to mimic (at least partially) the complexity with which humans approach music. One key aspect that hasn'tbeen sufficiently studied is that of sequential decision-making in musical intelligence. Addressing this gap, the book focuses on two aspects of musical intelligence: music recommendation and multi-agent interaction in the context of music. Though motivated primarily by music-related tasks, and focusing largely on people's musical preferences, the work presented in this book also establishes that insights from music-specific case studies can also be applicable in other concrete social domains, such as content recommendation. Showing the generality of insights from musical data in other contexts provides evidence for the utility of music domains as testbeds for the development of general artificial intelligence techniques. Ultimately, this thesis demonstrates the overall value of taking a sequential decision-making approach in settings previously unexplored from this perspective.