Notated Music in the Digital Sphere

Notated Music in the Digital Sphere Possibilities and Limitations

Edited by Margrethe Støkken Bue and Annika Rockenberger

15

STU D I ES FRO M TH E NATI O NAL LI B R ARY O F N O RWAY

NOTA B E N E

Notated Music in the Digital Sphere

Notated Music in the Digital Sphere Possibilities and Limitations

Edited by Margrethe Støkken Bue and Annika Rockenberger

N at i o n a l L i b r a r y o f N o r way, O s l o 2 0 2 1

Contents

Preface

9

Introduction

Margrethe Støkken Bue

11

1. RISM and Digital Research with Historical Musical Sources Jennifer A. Ward

20

2. Digital Encoding of Music Notation with MEI

35

Axel Teich Geertinger

3. The Editor’s Choice. From Sixteenth-Century Sources to Digital Editions Using MEI

57

Bjarke Moe

4. Strip and Tease: Digitally Undressing Tudor Scribes

76

Julia Craig-McFeely 5.

Preserving Digitally Inscribed Music

Jøran Rudi

6. Computational Musicological Analysis of Notated Music:

a Brief Overview

107

142

Olivier Lartillot

Contributors

162

164

Nota bene series

Preface

In the last couple of decades, digital humanities has evolved, giving dif­ ferent research fields new angles and new tools. Integrating technology and using digital resources in the humanities makes it possible to conduct research in new ways and give us new understanding of the humanities as a field.

For notated music, however, the development seems somewhat slower regarding digital research. Compared to text or sound, being more a kind of «vessel» for the composer’s intentions, notated music seems to fall between categories in the digital humanities.

To challenge this unsatisfactory state of affairs, The National Library of Norway arranged a full-day seminar in November 2018 aimed at investigating the digital possibilities and limitations for notated music in the sphere of digital humanities, looking further into areas and digital opportunities relevant for libraries, archives and music collections.

An international group of researchers with a diverse range of specialization in notated music in the digital setting presented their cutting-edge research and discussed creative solutions to the challenge of representing and scholarly investigating notated music with digital technology.

This collection of articles gives an overview of what was presen­t­ ­ed and discussed during this seminar.

9

As a leading institution within digital humanities, and with large collections of music, the National Library of Norway hopes to bring focus onto this area.

The editors would like to thank Hanne Brække Wulff and Hans-­ Hinrich Thedens at the National Library for valuable input and advice in the work with this book.

Oslo, 01. 12. 2020 Margrethe Støkken Bue Annika Rockenberger

10

Introduction

Margrethe Støkken Bue

Digital musicology is a diverse area of research. For many, the understanding of “music” within Digital Humanities concerns sound, recordings, or music notation software. The articles in this collection, however, discuss different aspects of digital musicology: primarily how pre-existing notated music can be represented digitally. The articles deal with subjects such as digital reconstruction, editing, encoding, databases, digital preservation and computational analysis of notated music.

Notated music is often, both physically and digitally, treated as a kind of text (i.e. written word). To some extent, this is useful, as both text and music have commonalities when notated for transmission. But as soon as we delve a bit further, the disparities begin to show. Anyone who has searched in digitised (historical) newspapers or books1 has come across Optical Character Recognition (OCR). OCR is a digital tool that converts images of typed, handwritten or printed text into

1 For instance National Libraries and Google Books https://www.nb.no/ https://gallica.bnf.fr/accueil/fr/content/accueil-fr?mode=desktop https://www.bl.uk/collection-guides/british-newspaper-archive https://www.google.com/googlebooks/about/ (all accessed November 9, 2020)

11

machine-encoded text and thus allows searching within the digitised text. The equivalent for music, Optical Music Recognition (OMR), has several possible definitions (Calvo-Zaragoza et al. 2020), while the OMR project at McGill University’s Digital Music Archives & Libraries Lab define OMR as “…the process of converting scanned images of pages of music into computer readable and manipulable symbols”.2 OMR is not as widely used as OCR, partly because it is only truly effective in modern typeset music, which often has a digital existence already. As we go back in time, printed texts remain accessible to OCR, but early printed music uses a different vocabulary of musical symbols and printing methods that defy accurate OMR. Besides, much of the repertory of early music is transmitted in manuscript form only and we still lack an equivalent to Handwritten Text Recognition (HTR) for notated music. To illustrate the issues with OMR, take for example the letter “A” in a text: it only has a limited number of representations.

The note “ ”, on the other hand, doesn’t tell us more than that it is a quarter note, and considerable additional information is needed to tell us what it represents to a musician. Therefore OMR demands more human intervention and enrichment and it is more expensive and at the same time less financially viable to develop. Since there is less notated music than there is text and the demand for electronic repre­ sentation is considerably lower, fewer resources are spent on increasing the quality of OMR. At this point in time, OMR is not of sufficient quality, and as a result, cannot be compared with OCR as a tool.

The best known area of digital musicology, and most commonly used, is digitised sheet music, both printed and in manuscript, presen­ ted as images. Many institutions and archives have digitised all or parts of their collections, making them available to an international readership. Even though there remain some challenges due to different metadata formats and models and interoperability of databases, the availability of digitised sheet music makes performing, research, dissemi­nation and curation much easier tasks. Dedicated databases for

2 https://ddmal.music.mcgill.ca/research/omr/ (accessed September 5, 2020)

12

notated music tend to be viewed as somewhat unnecessary (by other disciplines) since the digitised material is already available in the institutions’ databases and digital catalogues. The incentive for creating these databases is that the sheer number of works, and pages or images within the works, is so enormous that without the help of computational tools, the task of finding anything in huge collections is almost impossible.

Several institutions and projects have created digital work cata­­ logues, collating and registering all compositions by one composer, compositions of a composer family, compositions within a specific genre, a geographical area, or compositions connected to institutions.3 Some work catalogues are based on older, physical catalogues and collection descriptions, while others were digital from their inception.

These digital catalogues demonstrate numerous different approaches to structure, metadata, layout/design and whether or not they are connected to an edition, physical or digital. Also, the perhaps most important issue: digital work catalogues are dynamic. Where printed catalogues cannot be changed, digital catalogues are editable, making it possible to alter and add information as research uncovers new knowledge and reading of works and composers.

Some work catalogues, and databases for notated music, have created incipit search – the possibility to search using melody snippets instead of text. One example is the Répertoire International des Sources Musicales (RISM) catalogue which is using an input tool by Verovio, a music notation engraving library. Verovio has developed the Plaine & Easie Code (PAE), a “…library standard that enables entering

3 Examples of different work catalogues: Carl Nielsen Works http://www5.kb.dk/dcm/cnw/navigation.xq Bach digital https://www.bach-digital.de/content/index.xed?lang=en Swedish Musical Heritage https://www.swedishmusicalheritage.com/ Corpus Musicae Ottomanicae https://corpus-musicae-ottomanicae.de/content/index. xml;jsessionid=8AF019FDD743B26F13AEA59F25C1D819?lang=de Detmolder Hoftheater https://hoftheater-detmold.de/ (all accessed November 15, 2020)

13

music incipits in modern or mensural notation.”4 Because all the incipits in the catalogue(s) are encoded with the PAE the user is able to search in the notated music itself via a digital keyboard, and not only by the traditional methods of keywords and metadata. Other examples are the Catalogo Servizio Bibliotecario Nazionale (SBN) and the Gluck-Gesamtausgabe (GGA).5 This, however, is only useful if the user wishes to search the first few notes of a work. Full-text searching on musical sources using OMR – so the music does not have to be encoded, it can be read directly from the page – is still in its infancy, but the power of the tool can already be seen in the Liber Usualis project and its sister project Cantus Ultimus6 which has encoded a complete chant manuscript using OMR. However, OMR would need to be deployed across all digitized manuscripts and print sources to be useful enough to allow a researcher to identify, for example, a fragment that lacks its opening, or the reuse of part of one piece of music in another, something that still relies on feats of memory on the part of musicologists.

Encoding of notated music is possible with several formats, such as the Music Encoding Initiative (MEI), MusicXML and LilyPond.7 MEI “is a community-driven effort to define a system for encoding musical documents in a machine-readable structure.”, and is by far the most widely used in the medium of digital publishing connected to collected works, work catalogues and research. It is based on the principles of the Text Encoding Initiative (TEI),8 and provides the opportunity to encode notated music as text. The notated music is encoded

4

https://www.verovio.org/pae-examples.xhtml (accessed November 8, 2020)

5

https://incipitsearch.adwmainz.net/ (accessed November 15, 2020)

6 https://cantus.simssa.ca/manuscript/133/?folio=001r https://ddmal.music.mcgill.ca/research/omr/Search_the_Liber_Usualis/ (both accessed November 15, 2020) 7 https://music-encoding.org/ (accessed September 10, 2020) https://www.musicxml.com/ http://lilypond.org/ (all accessed November 16, 2020) 8

https://tei-c.org/ (accessed September 5, 2020)

14

in eXtensible Markup Language (XML), and the MEI-header. The header, which supplies descriptive and declarative metadata associated with a digital resource, gives several opportunities regarding what and how much information is needed for this particular file to be read.

Besides making it possible to encode and store the notated music in a machine-readable form required by databases.

Editorial work with encoding notated music presents new challenges and a need for new procedures. Interpretation of sources is still the same, as are the editorial decisions on what it should include and how. Added to this come several digital possibilities and challenges.

The roles of the editor and encoder can be conflicting, but the tasks of both roles must often be performed by the same person. Bjarke Moe (see ch. 3) puts it clearly: “A central point is that decision-making is to a large extent dependent on the purpose of the encoding. Consequently, the editor needs to take into consideration who is going to use the encoding and how. Looking at the relationship between the visual appearance of notation and the meaning of it, the editor must be able to address problems during the interpretation and presentation of the source.”

The system of music notation, (Common Music Notation or Common Western Music Notation),9 first became a subject of discussion in the 1900s. Originally the notation of so-called classical music raised a lot of issues, as this system is mainly constructed for notating Western classical music (Müller 2015, 5). The main issue with the classical system is that it does not allow for music that is not based on a strict pulse and pitch, which Western classical music to a large extent is, thus excluding non-Western musics and some folk or non-diatonic repertories. Electronic and digital developments in the notation of compositions and the development of new instruments for which the traditional notation of classical instrumentation was insufficient also exposed the shortcomings of that system. A notated musical work

9 https://music-encoding.org/guidelines/v3/content/cmn.html (accessed September 5, 2020)

15

including not only traditionally-notated music, but also images, graphics, sound, or microtonality, cannot be encompassed by the classical notational system (see ch. 5). Several non-Western forms of music, such as Chinese, Korean, Indian and Turkish, have developed their own notation system better suited to their instruments and form. An example of this is Ottoman Classical music. The project “Corpus Musicae Ottomanicae”.10 The project “…is a long-term project for the critical edition of Near Eastern music manuscripts…including parallel edition of song texts, and the online catalogue of Ottoman music sources. The project focusing on manuscripts of Ottoman music written in Hampartsum and staff notations during the 19th century…”. It includes work with both MEI (for encoding and metadata) and RISM (for cataloguing), and seeks to make it possible to encode the “Hampartsum” notation, as well as adapting metadata and cataloguing standards to fit the different demands of this music and notation.

A widely-known problem in accessing music from the past is damage to the physical object that preserves the music, usually a manuscript. Older material might not have stood the test of time, but more recent material can also suffer from various types of damage related to the use of lower-quality mass-produced materials. There are, however, many possibilities for digitally examining and “repairing” the manuscripts. The first step in this process is the digital capture of the manuscript at high resolution, and then employing advanced digital processing software (such as Adobe Photoshop™ or Gimp) to make it readable or even, when the damage is profound, digitally reconstruct images to create a version that might represent an undamaged state of the manuscript.

Computational musicological analysis combines musicological analysis with digital technology. By combining musicological parameters with computational analysis we are now able to examine metric, harmonic, intervallic and structural aspects of massive quantities of musical data in accurate ways that was not previously possible, and

10 https://www.uni-muenster.de/CMO-Edition/ (accessed November 16, 2020)

16

this has changed the landscape of musicology in ways that we have yet to understand fully. By teaching machines the processes of musicological data mining, patterns and connections can be found that the human brain cannot see, but also that it cannot necessarily conceive.

Not only because a machine can deal with much more massive quantities of information, but also because a machine does not prejudge and interpret the findings. For notated music, this is a research area very much in progress. There have been a number of attempts to use computational models to attempt to identify musical “fingerprints” in the works of a particular composer, notably that of Josquin Des Prez, in order to determine whether anonymous pieces were written by him, or whether all the works ascribed to him were indeed his own, but this involved manually inputting a musical corpus to a program.11 The Single Interface for Music Score Searching and Analysis (SIMSSA)12 project at McGill University in Montreal is focussed on the application of computational analysis directly on the original musical sources – and crucially on manuscripts with widely varying symbolic representations of music – so that any digitized page of music can be “read”, analysed and processed digitally.

Considering all of these varied aspects of (digital) musicology, the landscape of music within Digital Humanities is vast, but surprisingly lacking in visibility, given the wealth of research and development that is going on in the field. Notated music is, and will continue to be, an important part of both music research and performance, and should therefore not be neglected due to the complexity of musical languages or the challenges of standardisation and interoperability.

11 Michael Scott Cuthbert’s Music21 program encoded the complete works of Josquin in order to perform this structural analysis, and it has now been deployed across a number of associated repertories. http://web.mit.edu/music21/ (accessed November 24, 2020) 12 https://simssa.ca/ (accessed November 24, 2020)

17

Literature

Calvo-Zaragoza, Jorge,Jan Hajič and Alexander Pacha. 2020: “Understanding Optical Music Recognition”. ACM Computing Surveys 53 (4).

Cantus Ultimus. https://cantus.simssa.ca/manuscript/133/?folio=001r (accessed November 15, 2020) Common Music Notation.https://music-encoding.org/guidelines/v3/content/ cmn.html (accessed September 10, 2020) Corpus Musicae Ottomanicae. https://www.uni-muenster.de/CMO-Edition/ (accessed November 16, 2020) https://www.researchgate.net/publication/342762210_Understanding_Optical_ Music_Recognition (accessed November 9, 2020) Incipit search. https://incipitsearch.adwmainz.net/ (accessed November 15, 2020) Liber Ususalis. https://ddmal.music.mcgill.ca/research/omr/Search_the_Liber_ Usualis/ (accessed November 24, 2020) LilyPond. http://lilypond.org/ (accessed November 16, 2020) Music 21. http://web.mit.edu/music21/ (accessed November 24, 2020) Music XML. https://www.musicxml.com/ (accessed November 16, 2020) Müller, Meinard. 2015. Fundamentals of Music Processing. Audio, Analysis, Algorithms. Switzerland: Springer International Publishing AG. Optical Character Recognition. https://en.wikipedia.org/wiki/Optical_charac� ter_recognition (accessed September 5, 2020) Plaine & Easie Code. https://www.iaml.info/plaine-easie-code (accessed November 8, 2020) Plaine & Easie Input.https://www.verovio.org/pae-examples.xhtml (accessed November 8, 2020) RISM Catalog. https://opac.rism.info/index.php?id=4 (accessed November 8, 2020)

18

Literature

Single Interface for Music Score Searching and Analysis. https://simssa.ca/ (accessed November 24, 2020) The Digital Image Archive of Medieval Music. https://www.diamm.ac.uk/ (accessed November 15, 2020) The Lost Voices Project. http://digitalduchemin.org/ (accessed November 15, 2020) The Music Encoding Initiative. https://music-encoding.org/ (accessed September 10, 2020) The Text Encoding Initiative. https://tei-c.org/ (accessed September 5, 2020)

19

RISM and Digital Research with Historical Musical Sources1

Jennifer A.Ward

The Répertoire International des Sources Musicales (RISM) aims to describe and document the current locations of musical sources worldwide. Interest in and the relevance of RISM’s work has intensified following two major events in the past decade: the release of the free, online catalogue in 2010 and the wide availability of the cataloguing programme Muscat in 2016. RISM strives to increase its utility for a wide variety of users who wish to approach the data from a variety of angles, including from the fields of musicology, music performance, digital humanities, librarianship and music information retrieval.

Through the research on primary sources in music that RISM has been involved in for more than 65 years and the technology that makes discovery, use and fruits of this labour possible, RISM is looking towards new possibilities of conducting research and providing better, specialised tools for understanding our field.

1 I would like to thank an anonymous reviewer for helpful comments used in revising this paper.

20

The Répertoire International des Sources Musicales (International Inventory of Musical Sources, known by its acronym RISM) was founded in 1952 with the immediate aim of determining what historical musical sources were left and where they were located following the destruction caused by World War II (Lesure 1960, 25–26). More than 65 years later, RISM still leads the scholarly community in documenting musical sources worldwide. Interest in and the relevance of RISM’s work has intensified following two major events in the past decade: the release of the free, online catalogue in 2010 and the wide availability of its specialised cataloguing programme Muscat in 2016.

RISM has implemented digital developments in order to increase its utility for a variety of users who wish to approach the information in RISM from diverse angles, including from the fields of musicology, music performance, digital humanities, librarianship and music information retrieval. Recent RISM activities in South Korea, Chinese-speaking regions of Asia, and Latin America have enriched the pool of sources, enabling the field to be even more international. By using feedback from users, observing trends in scholarship and technology and reacting to the requirements of the various ongoing RISM projects worldwide, the project has been able to offer tools that are relevant to libraries, archives and researchers while continuing development to allow the organisation and presentation of the data in new ways.

The present article focuses on how RISM provides a foundation for investigating the cultural history of music and how RISM can be used as a base for further studies. First, a presentation will be offered of what the RISM project is and what current initiatives are underway.

An outline of the flow of data in and out of RISM will be described, followed by a discussion of the data structure used in Muscat. At the end, a few directions for collaboration between RISM and other projects will be explored.

RISM today The goal of RISM has not fundamentally changed since the post-war years: RISM is an international, non-profit organisation with the goal 21

of documenting the current locations of musical sources worldwide.

Musical sources are music manuscripts, printed music editions, libretti and treatises (Figures 1 and 2). They can be found in libraries, archives, churches, museums and private collections. The RISM project is led by the Zentralredaktion (Central Office) in Frankfurt, Germany, and work is carried out by independent working groups around the world, usually based at national libraries or major universities whose members catalogue the musical sources preserved in their countries in a central database (RISM Zentralredaktion). The Central Office manages the database and ensures that the data are consistent and published in the RISM online catalogue. This catalogue is available to the general public free of charge (RISM Catalog). Contributors to RISM can access their data for free, and they benefit from having their musical sources integrated into an international context of transmission. In short, RISM documents what exists and where it can be found.

Fig. 1. Johan Daniel Berlin (1714–1787), Menuet, autograph manuscript. Oslo, Nasjonalbiblioteket (N-Onm), Mus ms 91:15 (RISM ID no. 170000267). https://www.nb.no/items/URN:NBN:no-nb_ digimanus_267112?page=1 (CC-BY-NC-ND)

22

Fig. 2. Johan Daniel Berlin, Musicalisches Divertissement (Augsburg, [1751]). Munich, Bayerische Staatsbibliothek (D-Mbs) 2 Mus.pr. 3029#Beibd.1 (RISM ID no. 990004823). http://daten.digitale-sammlungen.de/bsb00104352/image_1 (CC BY-NC-SA 4.0)

Currently, the RISM online catalogue describes more than 1.2 million sources from over 40 countries. The RISM catalogue is a collaborative product of the RISM Central Office, the Bavarian State Library and the State Library of Berlin. Since its launch in 2010, the RISM catalogue has had three major versions, the most recent of which received an upgrade in September 2019.

Geographic coverage As a global project with the “I” of “international” written into its name, RISM takes seriously the goal to document musical sources from around the world. While the bulk of the sources documented by RISM is found in Europe and the United States, the work of our contributors worldwide has resulted in more dots on the map outside these areas: in Latin America, including Argentina, Brazil, Chile, Colombia, Guatemala, Mexico, Uruguay and Venezuela; in Asia, including mainland China and Hong Kong, Japan, South Korea and Taiwan; and Israel in the Middle East (see Figure 3). Many sources new to RISM have come from the areas just mentioned, and even within the traditional RISM sphere there are new contributors.

23

Fig. 3. Locations of RISM working groups and sources in the online catalogue.

Chronological coverage The original scope of RISM focused on polyphonic music from between 1500 and 1800. RISM working groups that were project-based or supported by grants found these chronological limits to be a convenient time frame in terms of securing financial support and setting achievable goals. Indeed, in some countries there is still much work to be done even within these three centuries. As smaller countries completed cataloguing projects that fell within this time frame, work was extended to 1850. But as interest in RISM arose in countries that were not involved with the project in its earliest years, it was evident that a chronological framework that is very well suited for European polyphony may not fit in other cultures and contexts. Therefore, RISM encourages national groups to determine their own chronological scopes, even if it extends beyond 1800.

Since all manuscripts are unique, handwritten sources from any time period may be included in RISM. Users will find manuscripts ranging from medieval antiphons to modern-day madrigals. Printed materials, on the other hand, are more widely disseminated and collected by libraries (especially modern editions), so the focus in RISM is on printed items of historical value, generally up to around 1900.

Currently, the RISM catalogue includes Scandinavian sources from Denmark (approx. 13,300 records), Finland (approx. 1,100), Norway (approx. 1,200), and Sweden (approx. 44,000).

24

Current documentation initiatives in RISM While RISM’s geographic and chronological boundaries have expanded, the amount of data in the RISM database has increased. A significant addition came when records for printed music were added to the online catalogue in 2014. The records originated as entries in the blue printed volumes known as series A/I, Einzeldrucke, or music by a single composer (RISM 1971–2003), and the years 1500–1550 from B/I, which describes music in anthologies from the sixteenth and seventeenth centuries (Lesure 1960). The revision and publication of records for the remaining years of B/I, 1551–1700, is underway.

RISM has also been able to import data from external library catalogues and cataloguing projects. Data imports have been possible for the RISM United Kingdom working group, the Moravian Music Foundation in the United States and the Moravian Library in Brno, Czech Republic (Moravská zemská knihovna). Planned imports include the musical holdings of the Italian union catalogue Servizio Bibliotecario Nazionale (SBN Musica), the Musicat project of the Seminario de Música en la Nueva España y el México Independiente at the Universidad Nacional Autónoma de México in Mexico City (Seminario), the national libraries in Austria and Spain (Österreichische Nationalbibliothek; Biblioteca Nacional de España), the music from the Catalogue collectif de France (Catalogue collectif), and additional smaller projects. Other opportunities to import batches of data from libraries or other institutions and projects would be welcome.

The outward view: the RISM online catalogue In addition to the records for 1.2 million musical sources named above, the RISM catalogue contains some 238,000 records for authority files for personal names, institutions and secondary literature (see Figures 4 and 5). The catalogue is mobile-friendly, and the interface is available in English, French, German, Italian and Spanish. The advanced search allows a search of the following indexed fields: A/I or B/I number, catalogue of works number, composer, genre, institutions, key, language, library siglum, liturgical festival, music incipit (including transpositions), other names, plate number, publisher, provenance, 25

RISM ID number, scoring, shelf mark, source type, title, watermarks and year. A speciality of the RISM catalogue is its musical incipit search, which searches the opening measures of a piece. An on-screen keyboard allows users to enter exact pitches in five octaves and search either those notes or their transposition (see Figure 6). The incipits are rendered using the Verovio engraving library (Verovio).

Fig. 4. The RISM catalogue record for the minuet by Berlin (RISM ID no. 170000267). https://opac.rism.info/search?id=170000267&View=rism Fig. 5. RISM’s authority record for Johan Daniel Berlin.

Fig. 6. The incipit search in the RISM catalogue for the minuet by Berlin.

26

Using RISM data The RISM records are rich in practical applications, such as reuse in local library catalogues, as well as research possibilities that go beyond descriptive bibliographic data. To make it easier for the records to be reused, all of the RISM data are freely available as linked data and linked open data directly from the RISM catalogue’s website. They are licensed under a Creative Commons Attribution 3.0 Unported licence (CC BY 3.0). This licence allows usage including downloading, redistribution, adaption and change. The full dataset in open data format includes the records for both the musical sources and the authority files.

Specific batches of data, such as from a certain library or for a particular composer, can be queried through a SPARQL endpoint and an SRU (Search/Retrieve via URL) interface. Each entry in the RISM catalogue includes a link to the record in MARCXML and RDF/ XML format.

Libraries can take advantage of RISM’s open data to import records for their sources into local catalogues. In the United States, the Moravian Music Foundation (MMF) harvested the RISM records that describe the music manuscripts in its collection (Blum 2017). The records had previously only been available through the RISM catalogue. Since the online catalogue of the Moravian Music Foundation is implemented by OCLC, the RISM records are simultaneously available in the online WorldCat catalogue, increasing the visibility of the MMF’s manuscripts (WorldCat). Links in the MMF’s catalogue point users to the original RISM records. In the course of this project, some corrections to the RISM records were provided through a batch transfer that allowed fields updated by the MMF to be integrated into the RISM records. For librarians interested in using RISM records in local catalogues, Blum explains the steps needed to adjust exported records to local requirements, including adding fixed fields, mentioning which fields need to be deleted or adjusted, and implementing fields for RDA compliance.

Digital collections and other online presentations of digital objects are additional opportunities for RISM data to be utilised.

27

Metadata is an essential part of digital collections, and existing RISM data can be used for search and display. An example is the Music Library Digital Scores Collection at the University of Washington, which features the university’s manuscript holdings of vocal music from the seventeenth and eighteenth centuries (Graham and Pierce 2012). The manuscripts had already been described in RISM, and these records were extracted so the metadata could serve as a basis in the university’s CONTENTdm database. Once the digital library was established, links to its digital objects were added to the corresponding RISM records.

The RISM data have also been used in research projects. Some projects have examined melodic similarity utilising RISM’s over 1.6 million music incipits. Jelmer van Nuss and others at Utrecht University used the RISM incipits to explore the potential for identifying anonymous sources, which has a very practical application for the thousands of anonymous sources in the RISM catalogue (Van Nuss, Giezeman and Wiering 2017). A further project involving Utrecht researchers plus members of RISM Switzerland implemented incipit similarity technology developed in Utrecht in the Muscat search interface (Zitellini et al. 2018).

Beyond incipits, RISM data were employed in the Big Data History of Music project (Rose, Tuppen and Drosopoulou 2015). The project traced the development of musical culture in Europe, looking at aspects such as the geographical distribution of publications, quanti­ tative highs and lows in music printing over time, and case studies using cities such as Venice. Bibliographic records from RISM’s A/I, A/ II (music manuscripts) and B/I series, being one part of a larger data set, could be mined for information on the locations of music publishers, publication years and quantity of publications printed. This datadriven approach confirmed conclusions that paralleled some already known in the literature, but the project could produce numbers to support these arguments. Moreover, individual musical sources could be tied into to wider social, economic and political trends, demonstrating the promising potential of big data methodologies in music research.

28

The inward view: data structure in RISM The data used in the initiatives described above are managed in the cataloguing programme Muscat, which is provided to members of RISM working groups. Muscat is an open source, web-based and platform-independent system. It is the result of a collaboration between the RISM Central Office and RISM Switzerland (Muscat project). It was originally developed by RISM Switzerland and RISM United Kingdom and had been in use by them for more than ten years before an implementation was made available to all RISM project participants in 2016 (Güggi and Pugin 2017).

Muscat facilitates the management of sources and the organisation of essential elements, which is crucial when numerous contributors are involved with a project. Muscat has a multilingual interface (English, French, German, Italian, Portuguese and Spanish, with Polish in preparation), a commenting system, folders and a versioning system for records. The versioning system keeps track of changes, shows the modifications and allows a record to be restored to a previous stage. Authority files and standardised vocabulary are built in so they can be accessed, referenced and added to while cataloguing.

Different cataloguing templates are offered depending on whether the source concerns handwritten or printed music, a libretto or a treatise. More than 50 fields are available for cataloguers, including many that are unique to cataloguing music (composer, catalogue of works number, genres, key, opus number, scoring, plate number for printed music, music incipit). Recently the capability to catalogue printed music was improved with the addition of the fields book format, colophon and fingerprint identifier along with the extended ability of contributors to input more detailed descriptions (see Figure 7). A field for watermarks supports specialists in the field of paper studies. Muscat aims to be a cataloguing programme that meets the needs of music specialists. In providing specialised fields, detailed information can be entered and retrieved. Data are structured around the MARC21 format, which eases data exchanges between RISM and libraries.

29

Fig. 7. A revised physical description for a printed music edition in Muscat: Carl Philipp Emanuel Bach’s Geistliche Oden und Lieder mit Melodien (Berlin, 1764), with holdings including the Nasjonalbiblioteket in Oslo (N-Onm; RISM ID no.

990003092).

From a cataloguer’s point of view, Muscat is user-friendly in that an autocomplete feature links to authority files and controlled vocabulary.

Music incipits, rendered by Verovio, are displayed as they are input.

Cataloguers can add external links, such as links to digitised music, a project website or additional bibliographic information. Images of elements such as bindings, watermarks or handwriting samples can be uploaded to enhance catalogue records and authority files.

Personal name authority records are linked to the Virtual International Authority File (VIAF), and information from VIAF can be imported to Muscat’s authority records. This connection between Muscat and VIAF enables RISM records to link to other resources that use VIAF numbers. RISM has been a VIAF contributor for personal names since 2018.

30

Opportunities for musicologists For RISM, Muscat’s ease of use affords individuals and institutions the flexibility to contribute at varying degrees of involvement according to what staff or project resources allow. This is a new and different model for contributing to RISM, and it reveals opportunities for collaborating with projects both as part of a traditional RISM national group but also outside the scope of one. Muscat can facilitate collaboration between musicologists and RISM, providing a structured database, tools to manage the data and even data to use.

With more than 1.2 million records for musical sources available in RISM, the combination of the Muscat application plus RISM’s data offers an effective and efficient tool for research projects. As a database, Muscat has music-specific fields not found in other widely available data management systems. The programme can serve as a project database for catalogues of works, critical editions, source studies or the study of music in a particular region or institution. Detailed source descriptions can therefore be entered into RISM with an eye towards future use as the basis for a catalogue of works or other specialised study of repertoire.

Records created for RISM are integrated into the international pool of musical sources, all of which have a consistent structure in common, thereby contributing to data stability and longevity. As an open source programme, Muscat can of course be used independently of RISM. With some technical adjustments, Muscat can be tailored to the needs of a project and even use RISM data as a starting point.

Data created using Muscat are consistent with the rest of RISM data, keeping open the possibility of sharing data with the international RISM project if desired. RISM welcomes the opportunity to collaborate with musicologists in this way.

One example of a potential use of Muscat in a research project can be demonstrated by an inventory of manuscripts described by Metoda Kokole (Kokole 2016). The 102 manuscripts of Italian vocal music once part of the Attems collection in Slovenia are listed in an appendix to Kokole’s article. Since there are RISM records for each of the manuscripts, a reference to this article was inserted in each record 31

through its short title, KokoleM 2016. Now all records can be linked and retrieved through the short title, and it is possible to reconstruct the collection in this virtual way.

A musicologist focusing on this Attems repertoire could use these records in Muscat’s integrated faceted search to explore aspects such as composers, instrumentation, genre and dating. Such a study is of course limited to the data in Muscat, but additional sources could be added if they were not yet in RISM: for example, a different collection to facilitate comparison. The data could also be downloaded and manipulated through external digital tools for additional analysis of the data. The preceding illustration was based on a small group of sources, but one might imagine what this would be like with an institutional collection or with the works of a single composer. A research project could take the RISM data as a basis for a more in-depth study of this music, refining and adding to the data as the project progresses.

Looking ahead Like all other public scholarly projects, RISM is called on to meet the demands of its users and contributors in a constantly changing scholarly and technological landscape. Since Muscat is open source, RISM can consider collaborative projects that would promote the further development of Muscat. This way, a project could allocate resources already at the funding stage to develop a certain aspect of Muscat that could be integrated back into the international Muscat project. The project would benefit from the tailored feature, and RISM would be able to offer this feature to the wider Muscat community. With such partnerships, both RISM and researchers would benefit from each other’s specialised knowledge, passing this on to end users through enhanced catalogue records.

Through the research on primary sources in music that RISM has been involved in for over 65 years, and the technology that makes discovery, use, and reuse of the fruits of this labour possible, RISM is looking towards new possibilities of conducting research and providing better, specialised tools for understanding our field.

32

Literature

Biblioteca Nacional de España. http://www.bne.es/ (accessed February 28, 2020).

A Big Data History of Music, Royal Holloway, University of London. https://www.royalholloway.ac.uk/research-and-teaching/departments-andschools/music/research/research-projects-and-centres/big-data-history-­ofmusic/ (accessed February 28, 2020).

Blum, David. 2017. “The Moravian Music Foundation Experience Using Bibliographic Records Downloaded from RISM.” Fontes Artis Musicae 64, no. 4: 355–366. https://www.jstor.org/stable/26769863 (accessed February 28, 2020).

Catalogue collectif de France. https://ccfr.bnf.fr/portailccfr/jsp/public/index.jsp (accessed February 28, 2020).

Güggi, Cédric and Laurent Pugin. 2017. “Zehn Jahre Entwicklungs- und Katalogisierungserfahrung mit Muscat.” Forum Musikbibliothek 38, no. 1: 20–27. https://journals.qucosa.de/ejournals/fmb/article/view/458 (accessed February 28, 2020).

Graham, Anne and Deborah Pierce. 2012. RISM Data as Metadata for Digital Collections. Paper presented at the conference Music Documentation in Libraries, Scholarship, and Practice, June 4-6, in Mainz, Germany. http://www.rism. info/en/publications/conference-2012.html (accessed February 28, 2020).

Kokole, Metoda. 2016. Migrations of Musical Repertoire: The Attems Music Collection from Around 1744. In Musicians’ Mobilities and Music Migrations in Early Modern Europe: Biographical Patterns and Cultural Exchanges, ed. Gesa zur Nieden and Berthold Over, 341-378. Bielefeld: Transcript Verlag. https://www. jstor.org/stable/j.ctv1wxt49 (accessed February 28, 2020).

Lesure, François. 1960. Recueils imprimés, XVIe-XVIIe siècles. Munich: G. Henle.

Moravská zemská knihovna (Moravian Library). https://www.mzk.cz/en (accessed February 28, 2020).

Muscat project. http://muscat-project.org/ (accessed February 28, 2020). Österreichische Nationalbibliothek. https://www.onb.ac.at/ (accessed February 28, 2020).

33

Litteratur Literature

RISM, 1971–2003. Einzeldrucke vor 1800. Kassel: Bärenreiter.

RISM Catalog. https://opac.rism.info/ (accessed February 28, 2020).

RISM Zentralredaktion. http://www.rism.info/ (accessed February 28, 2020).

Rose, Stephen, Sandra Tuppen and Loukia Drosopoulou. 2015.

“Writing a Big Data history of music.” Early Music 43, no. 4: 649–660. http://em.oxfordjournals.org/content/early/2015/09/02/em.cav071 (accessed February 28, 2020).

Seminario de Música en la Nueva España y el México Independiente. http://musicat.unam.mx/ (accessed February 28, 2020).

Servizio bibliotecario nazionale. https://opac.sbn.it/opacsbn/opac/iccu/avanzata. jsp (accessed February 28, 2020).

Van Nuss, Jelmer, Geert-Jan Giezeman and Frans Wiering. 2017.

“Melody Retrieval and Composer Attribution Using Sequence Alignment on RISM Incipits.” http://dspace.library.uu.nl/handle/1874/359995 (accessed February 28, 2020).

Verovio. http://www.verovio.org/index.xhtml (accessed February 28, 2020).

WorldCat. https://www.worldcat.org/ (accessed February 28, 2020).

Zitellini, Rodolfo, Geert-Jan Giezeman, Frans Wiering and Laurent Pugin. 2018. Incipit Melodic Similarity Matching in Muscat. Paper presented at the 19th International Society for Music Information Retrieval Conference, September 23-27, in Paris, France. http://ismir2018.ircam.fr/pages/events-lbd.html (accessed February 28, 2020).

34

Digital Encoding of Music Notation with MEI

Axel Teich Geertinger

There are profound differences between the notation of text and music in terms of their purpose, use of symbols and how they translate into a machine-readable form. This paper identifies the difficulties of encoding music notation as compared to text and how these difficulties may be handled. Despite the challenges, encoding formats for notated music which are as sophisticated as comparable textual formats are now available, such as the XML schema defined by the Music Encoding Initiative (MEI). But are there any limits to what is possible to encode? The author argues that today the primary limiting factor is not the available encoding systems but rather the ambiguity and complexity of music notation itself.

Introduction Compared to text or sound, digital research involving notated music seems to develop somewhat more slowly; relevant innovations and trends in the field of text encoding are usually only adapted by music encoding systems after a certain delay. To some degree, I will argue, this delay is due to the challenges involved in music encoding. There are definitely profound differences between text and music notation in terms of purpose, sets of symbols and how these translate into a format processable by a computer. My first question is therefore: why do these differences make encoding music notation difficult? The other ques35

tion to be discussed is: what are the limits to digital encoding of music notation? Generally, the difficulties occur at one of the two steps involved: 1) the reader’s interpretation of the notation or 2) the translation of the interpretation(s) into a machine-readable encoding. Any subsequent computerised processing of the encoding would be the third step but is not the primary concern here.

Music notation itself is an open, ever-expanding field, constantly developing in order to capture more or new aspects of sounding and sound-related artefacts. To focus the discussion, the music notation discussed in this article is confined to “Common Western Notation” or “Common Music Notation” (CMN); even within the repertoire of CMN the challenges are more than enough to illustrate some of the conceptual relations between music, notation, and encoding; other notation systems like neumes, 20 th-century graphic scores or Labanotation for dance are obviously facing similar challenges but are not considered here for the sake of clarity. In the following, the general term “music notation” is therefore to be understood as “Common Music Notation”. Also, the focus here is entirely on notation, that is, symbolic representations of music, and their translation into machine-readable encodings. Non-symbolic formats such as audio recordings pose different challenges and are therefore not considered.

Approaches to the encoding of music notation Data describing notated music in a digital environment range from digital images of manuscript or printed material and digitally typeset PDF files to semantic representations capturing the logical (musical) contents of the notation. The limitations of the former two are obvious in terms of their ability to be further processed for purposes other than visual presentation. It is very difficult, for instance, to perform any analytical tasks, searches or customisations such as transpositions on a purely graphical description of a score without converting it into a logical representation first. Therefore, this paper focuses on the much more flexible logical representations only.

Over the past six decades a considerable number of codes to symbolically describe music in a digital context have been suggested: a 36

selection of about 20 early musical codes of some importance was presented in Eleanor Selfridge-Field’s famous book Beyond MIDI (Selfridge-Field, 1997) more than 20 years ago. Wikipedia lists 50 different file formats1 used by current notation software (Wikipedia: Comparison of scorewriters); the list does not include older software such as SCORE or codes not supported by the notation software listed; for instance, EsAC, Plaine and Easie, **Kern and MEI are not mentioned.

A useful and commonly used model to distinguish classes of information in music encoding was proposed with the Standard Music Description Language, identifying four distinct domains of information: the visual domain (score information), the gestural domain (performance information), the analytical domain (theoretical information), and the logical domain described as “the underlying musical information, which abstractly represents compositional intent’ (Newcomb 1991, 76)”.

To be considered genuine music notation codes – not just graphical descriptions of the notation – encoding formats must include the logical domain to some extent in order to describe core musical elements such as pitch and duration. This is the main reason for not regarding PDF or other visually oriented formats as musical encodings.

Apart from the logical contents, each code has certain strengths and weaknesses depending on the purpose for which it was invented. Some codes are especially suitable for describing music as represented in the visual domain (Lilypond, SCORE), others focus on the analytical domain (**Kern) and others still on the gestural domain, i.e. performed music (MIDI). In contrast to the symbolic representations mentioned, MIDI may actually be labelled a semi-symbolic approach to music encoding as it contains recorded performance information such as timing and dynamics as well as basic symbolic information such as note pitch.

The primary focus of MEI (Music Encoding Initiative) is the encoding of music documents for archival or academic purposes; like a

1 Strictly speaking not the same as encoding formats, however. In principle, the same encoding may be stored in various file formats.

37

few other codes, it is comprehensive enough to accommodate information not only concerning the logical domain but also the visual and, to some extent, the other domains. This paper concentrates on MEI, which has become a de facto standard for the encoding of music notation in academia.

Some characteristics of music notation as compared to text Although some of the first computerised encodings of music date back to the 1960s,2 it is probably true that encoding systems for notated music have had a slower start than those for text. A comparison of the history of the Text Encoding Initiative (TEI) and the Music Encoding Initiative (MEI) may serve as a quick illustration of the typical delay of advances in the field of music as compared to text. As is apparent from their names, the two are closely related in terms of both purpose and technical implementation. Both are aimed at encoding documents for archival and academic purposes. TEI defines its chief deliverable as a set of guidelines that “specify encoding methods for machine-readable texts, chiefly in the humanities, social sciences and linguistics” (Text Encoding Initiative). Correspondingly, MEI strives to “define a system for encoding musical documents in a machine-readable structure.

MEI brings together specialists from various music research communities, including technologists, librarians, historians, and theorists […].” (Music Encoding Initiative). Technically, both TEI and MEI are expressed in XML, and many of the XML elements available in MEI are explicitly modelled on their TEI counterparts.

The effort to establish what was to become the Text Encoding Initiative officially began in 1987, and the first guidelines for TEI mark-up were released in 1994. The Music Encoding Initiative

2 An early encoding format of importance was the Digital Alternate Representation of Musical Scores (DARMS), invented in the mid-1960s by Stefan Bauer-Mengelberg.

38

started as a one-man project in the mid-1990s. Perry Roland from the University of Virginia, Charlottesville, presented a first draft of the standard at the International Society for Music Information Retrieval (ISMIR) conference in 2000. It was not until around 2009/2010, however, that MEI had its international breakthrough, not least due to substantial funding from the Deutsche Forschungsgemeinschaft (DFG) and the US National Endowment for the Humanities (NEH).

One of the most obvious reasons for any delayed advances in music encoding as compared to text encoding is the much smaller target group. Text is ubiquitous and used in virtually all professions.

More interesting in this context, however, are the inherent characteristics of music notation, making it much more challenging to define an encoding system for music than for text. These include the – in principle – unlimited number of symbols usable in music notation, the graphical/spatial nature of music notation, and its contextuality. Some of these aspects of musical codes were discussed in Beyond MIDI (Selfridge-Field 1997, 15–20). More recently, Johannes Kepper also discussed some of the challenges in encoding music notation (Kepper 2011, 223–236). The following is intended to go beyond these by including aspects such as the changing meaning of symbols over time and distance and the interpretation and translation of the symbols necessary to make them machine-readable.

Extensibility of the notational system Alphabetic script uses a very limited set of symbols, mainly letters, numbers, spaces and punctuation. Innovation in language and literature is not achieved by inventing new letters for the alphabet but by combining existing letters and words in new ways. Music notation, on the other hand, does not even have an alphabet. The symbols involved operate at very different levels and, in an attempt to capture the essentials of a sounding artefact, music notation tends to extend its set of symbols constantly to reflect changing performance practice, new instrumental techniques and the refinement of the notation system for still more precise instructions to the performers.

39

Fig. 1 shows an example from Siegfried og Brunhilde, an opera fragment by Niels W. Gade. Here, Gade introduces shields to signify that the knights are to hit their shields with their swords in the rhythm indicated by the symbols.

Because the composer is free to use any symbol or notation she or he considers useful for communicating musical intentions, a notation encoding system needs to be extensible just as music notation itself, even within the repertoire of Common Music Notation.

Code ex. 1 shows a simple way of defining a custom symbol in MEI, pointing to an image of the symbol. <symbolDef xml:id=”shield”> <symName>Knight’s shield</symName> <graphic target=”shield.png”/> <annot> A stylised shield with a cross on it as used by Gade in ‘Siegfried og Brunhilde’</annot>

</symbolDef> … <dir tstamp=”1” place=”above” > <symbol altsym=”#shield”/> </dir> <dir tstamp=”3” place=”above” > <symbol altsym=”#shield”/> </dir>

Code ex. 1. Defining a custom symbol by pointing to an image file in MEI. The optional <annot> (annotation) element provides a human-readable explanation. Once declared using <symbolDef>, the user-defined symbol can be referred to in a performance directive (<dir>), for instance. The tstamp (time stamp) attributes denote the horizontal positions expressed as the current bar’s beats.

Simply reproducing the shield graphically does not make its meaning processable, of course; a computer would not know whether the symbol is intended to have any influence on, for instance, a computer-generated performance of the piece. That would probably not be a problem in this case where the directive is an instruction primarily aimed at a human reader (a scenographer, for example). Alternative approaches will be presented below.

40

Fig. 1. Niels W. Gade: Siegfried og Brunhilde (1857). Fragment.

The graphical/spatial nature of music notation Music notation can be considered a multi-dimensional system rather than a one-dimensional string of symbols. Time is one dimension; the simultaneous events at any point in time (corresponding to the resulting timbre) are another. Finally, music notation also allows a number of independent, optional layers to refine or modify the basic notation: expression, articulation, technique, tempo, etc.

Leaving aside the peculiarity of the non-existent time between a bar line and the first beat of the bar, the temporal dimension in music notation, in contrast to text, is a continuum in which the exact placement and the dimensions of a symbol on the page may affect interpretation. The dimensions and the placement of a diminuendo hairpin, for instance, not only influences timing but may also change its meaning when reaching certain limits. Fig. 2 shows an example from Gade’s cantata Elverskud (or Erlkönigs Tochter). The dynamic markings (wedges) in woodwinds and brass are gradually decreasing in length from the uppermost staff downwards. The flutes definitely have a diminuendo while the trumpets appear to have an accent. The problem is to determine whether the parts between them – especially the horns – have diminuendos or accents. The timpani having both fz and a diminuendo does not make interpretation any easier.

41

Fig. 2. Niels W. Gade: Elverskud. Prologue.

Auto­graph score, p. 7 (DK-Kk C II, 139 Folio, 1928-29.178).

Clearly, the first problem here is the reader/encoder’s interpretation of the notation. If the encoder is not in doubt or does not consider it important to express alternative readings, the encoding is straightforward, simply giving the encoder’s interpretation only. If, on the other hand, the encoder wants to convey the information that there may be more than one possible reading at this point, it also raises the question of how to encode multiple interpretations and how to deal with them when processing the data afterwards. <choice> <unclear reason=”ambiguity”> <artic artic=”acc” place=”above” staff=”1” tstamp=”1”/> </unclear> <unclear reason=”ambiguity”> <hairpin form=”dim” place=”above” staff=”1” tstamp=”1” tstamp2=”1.5”/> </unclear> </choice>

Code ex. 2a. Handling ambiguity in MEI. In this example, the starting point of the diminuendo is set to beat 1; the length is assessed to 0.5 beats.

Even though doubt and ambiguity are quite possible to encode (code ex. 2a) and may provide useful information in a critical edition, open choices and ambiguities are problematic for computational processes.

No matter if the encoding is to be processed for visual presentation, for 42

analysis or for computer-generated performance, someone has to make a choice eventually. One way to make a decision without losing information about ambiguity or doubt is to use editorial mark-up indicating a lemma (preferred reading) and one or more alternative readings within an <app> (critical apparatus) element: <app> <lem> <unclear reason=”ambiguity”> <artic artic=”acc” place=”above” staff=”1” tstamp=”1”/> </unclear> </lem> <rdg> <unclear reason=”ambiguity”> <hairpin form=”dim” place=”above” staff=”1” stamp=”1” tstamp2=”1.5”/> </unclear> </rdg> </app>

Code ex. 2b. Resolving ambiguity by providing a recommended reading.

The <lem> element may also provide information about the person responsible for the recommendation and about which sources support the one or the other reading.

This would indicate to the software handling the encoding that the lemma option should be the default choice. Optionally, the encoder may include an annotation (<annot>) to further explain the situation in a text aimed at the human reader.

Ambiguous notation like the one in fig. 2 is quite common in Gade’s autographs, and indeed it raises the question of whether Gade made a clear distinction between diminuendos and accents at all; perhaps it is not really a question of either/or but of revising our understanding of Gade’s musical thinking and notation. On the other hand, contemporary scores and the printed sets of instrumental parts based on the autograph show how copyists and engravers also tried to interpret the notation to mean either diminuendos or accents – and, by the way, came to slightly different conclusions.

43

Contextual dependence In a text, the identification of individual letters is generally indepen­ dent of context. Provided that the overall alphabet in use is identified correctly and the letters are legible to the reader, the symbol shaped “A” may be identified unequivocally in a textual context as the letter “A”, regardless of any symbols preceding or following it.3 The meaning of a symbol in music notation, on the other hand, is highly contextual.

Often the meaning cannot be established by looking at the individual symbol only. For instance, we cannot determine the pitch designated by a note without looking back to read the clef, the key signature, any accidentals and other information that may affect the pitch, such as ottava brackets or instrument transpositions. There is no single symbol for the pitch A (in whatever octave); to write the pitch A, a combination of at least three independent elements is needed: a note, a staff and a clef.

As a consequence, computerised processing of musical notation is correspondingly more complex than text processing. At its core – that is, at CPU level – a computer can only handle one instruction at a time and only one atomic piece (or if performing calculations: two pieces) of data. This makes handling of plain text a quite straight­ forward process as seen from a computing point of view, while music notation usually needs more complex data structures and accordingly complex computations.4

3 The letter’s phonetic realisation and its function in the words and language of the text do not alter the identification of the symbol. Whether a reader decides that the correctly identified letter is actually erroneous and should be replaced is also irrelevant to the identification process. The point here is the local, atomic nature of the identification of the symbol itself.

4 Many high-level programming languages and concepts such as object-oriented programming have been developed to help programmers and users operate the computer and data at much higher levels of abstraction. Eventually, however, the computer needs to break down every process to a low-level programming language (machine code), that is, a linear sequence of very basic instructions processable by the central processor (CPU). Multiple processors working in parallel may be combined to speed up the workflow, but the process basically still is a linear one, comparable in structure to a string of text.

44

Historical changes and geographical variations in music notation In music notation, symbols may have had a very different meaning in the past than they do today. For instance, in the 17th century a flat sign on a note may not signify lowering the pitch of the note one half step below its unaltered pitch, but may instead just mean the cancellation of a sharp.

Fig. 3. Bernardo Pasquini: Il Lisimaco (1681). Sinfonia (D-Hs ND VI 2602). The flats in the last bar clearly do not indicate C flat but C natural.

Unless the encoder chooses to simply modernise the notation without a comment, a logical encoding of this piece would need to record the information that the notated flat in this case is not to be processed as a flat in the modern understanding but as a natural. <note pname=”c” oct=”3” accid=”f” accid.ges=”n”/>

Code ex. 3a. MEI offers distinct attributes for written and performed (gestural) accidentals. The attribute accid=”f” refers to the written flat, accid.ges=”n” to the performed natural.

Code ex. 3a shows a simple and compact way to make explicit the difference – as perceived from a modern point of view – between the visual appearance (flat) and the expected performance (natural) in MEI. This kind of encoding is fairly easy to handle computationally.

An alternative encoding laying more emphasis on the historical nature of the apparent contradiction could be as follows: 45

<app> <lem> <reg> <note pname=”c” oct=”3” accid=”n”/> </reg> </lem> <rdg> <orig> <note pname=”c” oct=”3” accid=”f” accid.ges=”n”/> </orig> </rdg> </app>

Code ex. 3b.

Example 3b uses a “critical apparatus” approach similar to the one in example 2b above, explaining that the preferred reading as c natural is a regularised version (as indicated by the tag <reg>), while the notated c flat represents the original notation. The original version retains the accid.ges attribute demanding a natural in performance in order to ensure that any software processing the encoding would perform the original notation as intended (as a natural) and not according to modern reading (flat).

Verbalisation/translation In order to make music notation machine-readable, it must be translated not only into a code consisting of characters and numbers. To be able to process the information on a logical/semantic level, we need to identify explicitly what the symbols are or what they mean and not just what they look like. In some cases, however, the intended meaning of a notational feature may simply be unknown.

Fig. 4. Peter Heise: String Quartet No. 1 (1857). Larghetto. Autograph part, violin 2 (DK-Kk C II, 7k, Heises samling 134).

46

An example is the notation of accents(?) in Peter Heise’s String Quartet No. 1. Whether his mirrored accents in this particular manuscript do have a distinct meaning – very short crescendos, for instance – or whether they are just a slip of the pen remains unresolved.5 In this case, the encoder would either have to decide on a distinct interpretation (an accent) with or without offering alternative interpretations (similar to the preceding examples) or resort to a purely graphical description of the symbol, potentially excluding it from being semantically processed.

On the other hand, decisions of this kind are no different from the ones to be made by any reader of the original notation: the reader will either have to choose an interpretation or ignore the symbol entirely. So, also in this case, the limiting factor is not the encoding but the fact that the original notation itself is difficult to interpret. <symbolDef xml:id=”wedge_pointing_left”> <symName>Wedge pointing left</symName> <line form=”solid” x=”0” y=”0.5” x2=”1” y2=”0”/> <line form=”solid” x=”0” y=”0.5” x2=”1” y2=”1”/> </symbolDef>

…

<note pname=”c” oct=”5” accid.ges=”s”> <artic artic.ges=”acc” altsym=”#wedge_pointing_left”/> </note>

Code ex. 4b. MEI encoding of the graphical features of Heise’s mirrored accent and a note referencing it. In this example, the attribute artic. ges=”acc” informs the reader or computer that the realisation of the arti­ culation in performance should be a regular accent.

5 Peter Heise’s autograph score (DK-Kk, CII, k7, Heises samling 081a) does not have any dynamic markings at the corresponding location. Markings in the other parts and in analogous passages suggest, however, that Heise did indeed intend the markings to be regular accents.

47

A possible limit to even the most sophisticated music encoding system is music notation that defies its own logic. The strongly suggestive graphical nature of music notation makes the example in fig. 5 instantly comprehensible to the human reader; nonetheless, it is quite difficult to describe in terms of a semantic, machine-readable encoding.

Gade is one of the composers with the good fortune of being able to spell their names with music notation. Although not as musically distinct as B-A-C-H, the letters G-A-D-E are easily spelled musically. Furthermore, the symmetrical structure of the intervals involved (second, fourth, second) makes it possible to read the name from both sides by turning the page upside down and using an alto clef instead of the G clef. While it is perfectly possible to encode Gade’s musical signature for visual rendering simply by adding a mirrored C clef symbol at the end of the staff, this visually-oriented encoding does not account for the fact that each note has two possible pitches, depending on the orientation of the page. <score> <scoreDef> <staffGrp> <staffDef c lef.line=”2” clef.shape=”G” key.sig=”0” lines=”5” n=”1”/> </staffGrp> </scoreDef> <section> <measure> <staff n=”1”> <layer> <note dur=”1” oct=”4” pname=”g”/> <note dur=”1” oct=”4” pname=”a”/> <note dur=”1” oct=”5” pname=”d”/> <note dur=”1” oct=”5” pname=”e”/> <clef line=”3” glyphname=”cClefReversed” glyphnum=”U+E075”/> </layer> </staff> </measure> </section> </score>

Code ex. 5. Visually-oriented MEI encoding of Gade’s musical signature.

48

Fig. 5. Niels W. Gade’s musical signature.

While making Gade’s signature machine-readable may not be a realworld concern, there are of course genuine compositions based on this very type of ambiguity, such as some of the canons in J. S. Bach’s Musikalisches Opfer. This kind of notation – like other “Augenmusik” – operates at a level of abstraction that is difficult to capture in a truly machine-readable way. Either we encode the graphical appearance of the notation (similar to the Gade example), or we encode the resolution by writing out the implied parts; alternatively, we invent a customised encoding system and the software necessary to handle this special notation.

Fig. 6. Two examples from J. S. Bach: Musikalisches Opfer (Neue Bach-Ausgabe VIII, 2).

49

Standardisation and complexity Depending on the purpose of the encoding, there are ways to describe even situations that do not comply with the conventions of music notation, but it may be necessary to customise the format (i.e. modify the MEI schema in this case) or to provide multiple encodings in order to include multiple possible interpretations.

It seems fair to say that encoding systems are available that allow us to describe music notation at virtually any level of detail. The description possible is not really limited by the technology available; if something can be identified, it is probably also encodable in one way or another.

The project Beethovens Werkstatt may serve to illustrate the level of detail at which scholars are able to encode music notation today.

The project aims to document Beethoven’s compositional processes by identifying, visualising and transcribing the multiple layers of writing found in the sketches for selected works, such as his Sonata Op. 111.

The layers are made visible not only by tracing the symbols and colouring them according to the order of writing, but each little doodle is also meticulously identified, encoded and annotated if needed, provi­d­ ing the basis for a commented transcript of the sketch, layer by layer.

Fig. 7. A detail of the sketches for Beethoven’s Sonata Op. 111 (Beethovens Werkstatt).

50

Another example is the encoding of music metadata. The Danish Centre for Music Editing created the metadata editor MerMEId to enable scholars to produce thematic catalogues of works or detailed source descriptions for their editions using MEI metadata containable in the MEI <meiHead> element (the MEI file’s “header”). The level of detail and the complexity of descriptions made possible with this tool far exceed that of traditional library systems. The thematic catalogues of the works of Carl Nielsen, Johan Peter Emilius Hartmann and Johann Adolph Scheibe,6 all edited entirely with MerMEId, do not exploit all the possibilities offered by MerMEId, and to keep the software manageable both technically and from a user’s perspective, MerMEId in turn does not support all encoding options available in MEI. The limitations we are facing today with advanced encoding formats such as MEI, therefore, do not really concern our ability to capture the information we want; rather, the limitations relate to project resources, software usability, preservation of data, and data interchange or interoperability.

In the early days of music encoding, a multitude of more or less project-specific encoding systems and infrastructures created isolated data silos, eventually leading to the loss of valuable information because data were not easily interchangeable between systems and therefore died along with the containing infrastructure. Over the past couple of decades, efforts to promote open standards for the encoding of music notation have addressed these issues. Projects no longer need to invent their own encoding schemes. However, the effort to support a wide range of applications requires a standard to be very “loose” in terms of restrictions on the encoding. In other words, to enable projects to speak the same “language”, that language needs to be diverse and detailed. MEI is such a “loose” encoding format; it often allows the user to encode the same information in a number of ways, depending on the project’s purpose. The following is an example of the encoding of basic performance metadata.

6 http://www.kb.dk/dcm/cnw.html, http://www.kb.dk/dcm/hartw.html, and http://www.kb.dk/dcm/schw.html, respectively (accessed December 6, 2018)

51

<event> <desc>First performance</desc> <date isodate=”1832-10-29”>1832-10-29</date> <geogName role=”city”>Copenhagen</geogName> <corpName role=”venue”>The Royal Theatre</corpName> <persName role=”conductor”>Claus Schall</persName> </event>

Code ex. 6a. A structured encoding of basic performance metadata.

The encoding in ex. 6a is highly structured and therefore well-suited for machine processing. This database-like organisation of information is especially useful when the same kind of information is to be processed repeatedly. The MerMEId editing software, for instance, uses the above model which allows the use of a standardised input form and allows the software to generate consistently formatted lists of performances.

If, on the other hand, the performance metadata were to occur in a descriptive text such as the preface of a critical edition, a semantic­ ally marked-up paragraph of text as shown in ex. 6b is likely to be a more convenient approach. In a text-like rather than list-like context, the encoder would probably even want to omit the outer <event> element and just consider it a paragraph of text happening to contain a date, a place, a name etc. The mark-up would still be equally useful for generating indexes or cross-references but not table-like lists of performances. Consequently, a project’s encoding approach will already anticipate the intended output and the primary purpose of the data to some extent. <event> <p>The first performance took place at <corpName role=”venue”>The Royal Theatre</corpName> in <geogName role=”city”>Copenhagen</geogName> on the <date isodate=”1832-10-29”>10th of October 1832</date>, conducted by <persName role=”conductor”>Claus Schall</persName>. </p> </event>

Code ex. 6b. A marked-up text intended for human readers.

52

In practice, therefore, interoperability – that is, various systems being able to process data from various sources – is often not realistic as encodings become increasingly detailed and specialised; the interchange of MEI data across projects and systems will require transformations based on knowledge about the encoding practice of the proj­ ects involved. The flexibility and comprehensiveness of MEI allows a wide range of projects to use the same data format, but paradoxically it also tends to create new data silos of the kind that standardisation was intended to antagonise.

The paradox, however, is only apparent. The diversity of encoding practices, even within the same format, basically reflects the fact that reality is complex. A simple data format reducing the encoding options to only some basic aspects of notation is more likely to support interoperability, but the inherent restrictions also severely limit the scope of use cases. The development of comprehensive encoding schemas such as MEI has enabled digital musicology to move from operations on the most basic aspects of music (such as analytical tasks involving only pitch and duration) to detailed studies of music notation and musical sources. Even though projects like Beethovens Werkstatt and the thematic catalogues produced by DCM utilise completely different parts of the MEI schema, they still speak the same “language”.

This enables users and developers to share tools and concepts. It also allows encoded music to be included directly in catalogues or vice versa without mixing encoding schemas or languages. To minimise problems of interchange, the MEI community is discussing recommendations or “best practice” examples to encourage a certain degree of uniformity whenever possible. If necessary, varying or project-­ specific encoding habits may be regularised by transforming data to a standardised “dialect” of the format to facilitate interchange.

Finally, linking technologies have gained great importance.

Linking is a way of overcoming the problem of data silos – not by breaking them down into a single pool of homogeneous data, but by providing access to relevant data from, or in, a variety of contexts.

Linked Data technology connects highly specialised or detailed resources to complement other online resources. For instance, thematic 53

catalogues based on MEI may link to library systems, providing a work-centred perspective on the library’s collection to supplement the generally item-centred perspective of library catalogues (Geertinger 2019).

Fig. 8. The edition of Carl Maria von Weber’s works and writings exemplifies the use of Linked Data.

Images and text from external sources are integrated directly into the edition’s pages along with references to authority files for further linking (Carl-Maria-von-Weber-Gesamtausgabe).

To summarise, the situation today seems to be that the major obstacle to translating Common Music Notation into a digital, machine-processable format is no longer the absence of adequate encoding formats.

Instead, one of the main challenges is the complex, historically evolving and sometimes ambiguous nature of music notation itself: it makes encoding an endeavour that is far from trivial, not least because the encoding process forces the encoder to be very explicit about the interpretation of the notational elements – more so than is necessary in the traditional processes of copying or engraving music by hand.

Once encoded, ambiguities and variant readings may also pose a challenge to the subsequent processing of the data. When encoding choices and variant readings, a preferred or “default” reading should be indicated to guide the processing software.

Multiple ways of encoding the same notation make it possible to optimise the encoding for certain uses or to emphasise certain aspects 54

of the notation (for instance, the historicity of the meaning of accidentals as discussed in section 3.4.) but may at the same time be an impediment to standardisation and data interchange. Such obstacles seem to be the price for comprehensiveness and flexibility of the encoding format, but probably a price worth paying.

55

Literature

Beethovens Werkstatt. https://beethovens-werkstatt.de/ (accessed October 18, 2018).

Carl Maria von Weber Gesamtausgabe. Digitale Edition. https://weber-gesamtausgabe.de/A002068 (accessed October 20, 2018).

Geertinger, Axel Teich. 2019. “Samlingsformidling gennem tematiske værkfor­ tegnelser.” In Fund og forskning 58: 213-234.

Kepper, Johannes. 2011. Musikedition im Zeichen neuer Medien. Historische Entwicklung und gegenwärtige Perspektiven musikalischer Gesamtausgaben.

Norderstedt: BoD. MerMEId – Metadata Editor and Repository for MEI Data. Royal Danish Library. http://www.kb.dk/en/nb/dcm/projekter/mermeid.html (accessed December 6, 2018).

Music Encoding Initiative. http://music-encoding.org (accessed October 20, 2018).

Newcomb, Steven R. 1991. “Standard Music Description Language complies with hypermedia standard.” Computer 24, no. 7 (July): 76–79.

Selfridge-Field, Eleanor. 1997. Beyond MIDI.The Handbook of Musical Codes.

Cambridge (Mass.): The MIT Press.

Text Encoding Initiative. http://tei-c.org (accessed October 20, 2018).

Wikipedia: Comparison of scorewriters. https://en.wikipedia.org/wiki/Compari�son_of_scorewriters#File_formats_and_extensions (accessed September 20, 2019).

56

The Editor’s Choice.

From Sixteenth-Century Sources to Digital Editions Using MEI Bjarke Moe

The article is concerned with discussing the role of the encoder of musical notation and with exploring how to make choices when encoding. A central point is that decision-making is to a large extent depen­ dent on the purpose of the encoding. Consequently, the editor needs to take into consideration who is going to use the encoding and how. Taking its point of departure from an ongoing project on sixteenth-century monophonic songs, the article discusses the relationship between the visual appearance of notation and the meaning of it. Furthermore, the article explores how an editor is able to address problems during the interpretation and presentation of the source by employing Music Encoding Initiative (MEI).

Encoding as interpretation The claim has been put forward that despite the new digital possibilities, the fundamental model of scholarly editing has not changed at all (Tanselle 2006, 6; Bordalejo 2013, 75). This might be true to some extent, but during the process of preparing a digital edition, a new procedure has emerged: the encoding, the process during which the contents of a work is converted into a format suitable for data processing.

Encoding texts or musical notation is not a straightforward process resulting in an objective data format that encompasses the entire contents of the source. In its early days, in fact, encoding was 57

described as “a series of acts of translation from one semiotic system (that of the primary source) to another semiotic system (that of the computer). Like all acts of translation, it must be seen as fundamentally incomplete and fundamentally interpretative” (Robinson and Solopova 1993, 21). The nature of an encoding, thus, is fragmentary and reflects the original source only to a certain extent. Because the encoding process is a transformation from one system to another, text scholars have compared it to “transcription”, a term traditionally used to describe the seemingly simple conversion from handwriting to typewriting, for instance. As a reader of a transcription, one needs to know the limitations of it. The transcription might or might not give an impression of the visual appearance of the text; it might or might not show the exact wording of the text. Similarly, as an encoder “you need to believe in transcription. It is impossible for a transcription to reproduce the original object; it is always a selection of features from that object” (Lavagnino 2006, 338). Taking this statement further, the present article argues that encoding is an act of editing, requiring the editor to choose which entities to encode and how to encode them.

Based on sixteenth-century musical sources, the article will discuss some fundamental problems around how to read the sources and how to encode the interpretation of them. A crucial point will be that the editor needs to address the user of a digital edition when editorial problems arise. To help editors identify problems, the article will discuss how to employ Music Encoding Initiative (MEI) to describe music notation, and the article argues that the editorial procedure of encoding music as an act of balancing visual appearance and musical meaning.

Digital editions of music During the preparation of a digital edition of music, an editor must decide how to interpret the sources, how to encode the interpretation and how to communicate the interpretation to the user. Reading and interpreting the musical sources obviously requires the editor to consider philological questions and concerns related to the musical notation and its meaning. Further reflection is needed when encoding the 58

notation, for instance regarding the methods for encoding of visual appearance and musical meaning. An interplay between reading and encoding the source seems to influence the editor’s decision-making.

Interestingly, considerations on how to encode the notation might influence the interpretation of the source. The editor also has to consider the purpose of the edition. It might be a mere presentation of the source, leaving the user to interpret the meaning of its visual appearance, or it might be an edition customised for the user, allowing them to interact with the contents (Wiering 2009). In order to make decisions about this, the editor needs not only to consider what kind of interface the edition should have. The editor has to decide how to encode the notation and how to help the user to understand the source.

Besides choosing what to do when encoding, the editor has to explain the choices available to the user, at least on two levels: on a global level discussing the principles for the source reading and the encoding, and on a local level, explaining the readings and emendations undertaken on a specific source. Furthermore, thorough pre­ sentations of the sources, their contexts and reading guides are needed in order for the reader to engage fully with the digital resources.

Fig. 1 illustrates how a digital edition might be prepared. The interdependence of the processes of editing and encoding makes the role of the editor vital.

59

Reading and transcribing sources Working with notated music from the sixteenth century is troublesome, since the notational conventions often differ significantly from modern ones. Reading and understanding the sources calls for specialised knowledge, and transcribing the music into modern notation for the sake of present-day users is problematic. Notational conventions are not stable, nor are they homogeneous. The procedure of transcribing old notation therefore presents the editor with a quandary that modernising the notation will affect the “horizon” within which the reader interprets the edition. In the nineteenth century hymnologists who presented readings of sources in printed editions invented new notational conventions in order to describe what they thought were the most essential parts of the musical sources. Johannes Zahn created a collection, Die Melodien der deutschen evangelischen Kirchenlieder, showing the variety of melodic forms in the surviving sources of hymns in the German Protestant church. His goal was to trace the original melody and to show what changes a melody underwent up until its use in Zahn’s own day. While explaining some of his considerations on which melodies he had collected, Zahn did not describe his method in transcribing them. Fig. 2 shows an example of what seems to be a simple notation of a hymn from the sixteenth century that Zahn transcribed in his collection (fig. 3). When comparing his edition with the original notation, it is striking how many details he changed. First of all, note values in mensural notation were reduced to fit with modern standards, making crotchets the basis of the beat. This procedure may seem uncontroversial, but as Zahn insisted on keeping the mensuration sign (the symbol similar to the time signature cut-c), his readers might understand it as alla breve. Rather than describe the number of beats within a bar, it shows the hierarchy of the notes. When reducing the note values four times as in this case, the mensuration sign is disconnected from the notes (the semi-breves) to which it refers. Zahn also changed the clef to reflect the fact that readers were more familiar with the G clef. Furthermore, he modernised the orthography of the vocal text and left out the fermata on the final note. None of these changes was mentioned by Zahn.

60

Fig. 2: The melody “Dys synd die heylgen zehn gebot” as it appears in its first edition: Eyn Enchiridion, Erfurt 1524, fol. A2b. (Wikipedia Commons) Fig. 3: Transcription by Johannes Zahn (Zahn 1889, 524).

Fig. 4: Transcription by Henrik Glahn (Glahn 1954, II:22).

Fig. 5: Transcription in Stalmann 1996, 97. The original clef, time signature and note values are indicated at the beginning of the first staff. To the left of the vocal text, ** indicates that the text is placed separately below the music in the original source.

On the second staff, || denotes a line break in the source. (© by Bärenreiter-­Verlag Karl Vötterle GmbH & Co. KG, Kassel)

61

Transcriptions of the same melody by later scholars show how priorities changed. Henrik Glahn’s collection of Lutheran hymns continued the tradition introduced by Zahn (see fig. 4). However, Glahn explained how he transcribed the sources and altered the notation of them for the purpose of comparing different versions of the sources. To serve his investigation, Glahn omitted the vocal text and added lines between the verses. To a modern reader, a vertical line across the staff might be interpreted as a bar line between two bars. In the context of sixteenth-­ century notation, however, it is mostly used as a division between sections. Bar lines used in their modern meaning as a separation of two metrically corresponding units only came into regular use in the seventeenth century.

The ambitious project Das deutsche Kirchenlied was initiated in the 1960s with the purpose of collecting biographical data on and publishing critical editions of German hymns. Volumes that contain music reflect openly on the editorial guidelines of the edition. The transcriptions aim to give as much information from the sources up front, providing the reader with an impression of the layout and notational characteristics of the source (see fig. 5). As a result, the transcriptions are quasi-diplomatic, and they invite the reader to dig into the material and to search for answers to individual questions. Still, we need to keep in mind that a transcription serves a purpose and that it never gives an objective reading of a source. Hans-Otto Korth, editor of Das deutsche Kirchenlied, has in fact stressed that the act of editing melodies is arbitrary (Korth 2004).

A new project In 2018 the project “Music and Language in Danish Reformation Hymns” was started by the Society for Danish Language and Literature as an interdisciplinary project concerned with investigating aspects of music, literature, language, theology and liturgy of the early Protestant church in Denmark (website: Musik og sprog i reformationstidens danske salmesang). The objective is to publish digital editions of nine sixteenth-century publications from the period 1529–1573. Six of the publications contain musical notation intermingled with the text. Each 62

of the nine publications will be edited and published digitally on the internet (website: Danske reformationssalmer. Melodier og tekster 1529–1573), showing texts in html and music notation in SVG format (scalable vector graphics), based on encodings in TEI respectively MEI. Facsimiles of the sources will be available side by side with the edition.

The contents of the music prints vary greatly, from woodcuts (some of which are small and very difficult to read) to beautifully decorated type prints easy legible even to a modern reader with only little experience of mensural notation. Some of the prints were prepared in workshops by publishers with limited or even no experience of musical notation. Other prints were made at the best workshops in the kingdom, supported by financial contributions from the state. Such large differences in appearance should make an impact on a user who wants to engage with the web portal, being a kind of virtual collection of surviving prints. Therefore, when making digital editions of a source collection like this one, information on the historical context of the sources is needed.

Fig. 6: Excerpt from Frans Vormordsen’s mass book, which was printed using primitive woodcuts for the music (Vormordsen 1539, fol. B1r).

(Reproduced by courtesy of the Royal Danish Library, Copenhagen)

Fig. 7: The square notation in the gradual by Niels Jesperssøn employs notes looking like semiminims to indicate a subdivision of a main note (Jesperssøn 1573, 81–2). (Reproduced by courtesy of the Royal Danish Library.

Copenhagen)

The sources employ two main notational forms: mensural notation and square notation. Variations occur within these two forms, including a rather unconventional mensural notation with only four lines, no 63

mensuration sign and no repeat sign (see fig. 6). A mixture of the two main forms are seen as well, being a kind of semi-mensural square notation employing three different symbols for noteheads to signify the length of a note (see fig. 7). The same melody is present in other sources, suggesting how the rhythms could have been interpreted.

When publishing digital editions in modern transcription, the great variety in notation will disappear. Providing facsimiles on the website solves this problem to a certain degree. However, the editor should consider providing accompanying texts that explain technical details about how the sources were produced and why they look as they do.

Some melodies are present in more than one source, and often they are notated in different ways. Therefore, every source is encoded separately. The strength of this procedure is that the user is provided with an opportunity to collate the different melody versions side by side.

Instead of seeking an original melody form or prototype that might not even have existed or been used, each of the sources is treated as a record of how the melody was used or was intended to be used. Furthermore, it is possible to process the data for the purpose of easily collating forms of the melody. The collection, eventually containing more than 500 melodies, will be searchable, providing a more thorough repository for musical analysis, for instance.

The purpose of the website is to make texts and melodies avail­ able to anyone interested in hymn singing, layman as well as researcher.

On the basis of the encodings, three separate research projects, pre­ defined by the involved researchers, will be carried out. A core criterion is that it should be possible to read the music without knowledge of notational forms no longer in use. Therefore, the editions should be interactive, enabling the user to transpose melodies, change clefs and shorten note values. Short introductory texts will describe the sources and their notational nature to enable the user to explore the material on their own. Suggestions for further reading will help the user acquire additional information on particular topics.

64

Music Encoding Initiative One particularly helpful tool in digital editing of music is MEI, Music Encoding Initiative, an extensible mark-up language schema developed to describe musical notation. Rather than reproduce the visual appearance of the source, the encoding structures the contents of the source using elements interrelated in a predefined hierarchy governed by the MEI schema. The schema is continuously being developed, and the current version 4.0.1 was released in 2019 (website: Guidelines for Music Encoding Initiative, version 4.0.1). The schema describes how tags (for instance <beam> and <note>) are related and how the attributions of a tag (for instance @dur specifying the written duration of a <note>) should be used.

From the beginning, the MEI schema was designed on a model from the so-called “Standard Music Description Language”. In this language, musical notation was described within four domains: 1) the logical domain describes the meaning of a notational symbol in relation to other symbols; 2) the gestural domain captures information on the musical performance interpreted through a symbol; 3) the visual domain describes the physical appearance of a symbol; and 4) the analytical domain provides a symbol with a quantitative label suitable for data processing (website: Standard Music Description Language). These four domains enable a notational element to be encoded in different ways. A note can be encoded as a logical structure of its musical content separate from its visual appearance on the page, or it could be encoded as a realisation of how it should be performed without reproducing how it was written in the notation (website: Introduction to Music Encoding Initiative). It is possible even to encode within both domains simultaneously, describing different qualities of the same note; it is not necessarily possible, however, to encode everything in four different ways. A fundamental condition is that the encoder decides what to do. Fig. 8 below shows one possible encoding of a short music example. The attribute @pname (pitch name) describes the note within the logical structure of the traditional letter notation. The exact position of the note is recognisable with @oct, which is an analytical value designating the specific octave according to the American 65

standard pitch notation. The attribute @pnum combines these two values into one digit according to, for instance, MIDI note numbers (C4=“60”, C4 sharp=“61”, D4=“62”). The disadvantage of this attribute is that it is very difficult to read the melody from the code.

The duration of the note is encoded based on its visual appearance, a crotchet (dur=”4”) with a dot (dots=”1”). The total length of the note is not described but can be calculated from these two attributes. Since the note has a staccato dot, its gestural duration in a performance would probably differ from the written length. The attribute @dur.ges hence describes a possible duration measured according to the logical scale of note values (1=semibreve, 2=minim, 4=crotchet, 8=quaver etc.).

Fig. 8. Left: music example; right: possible MEI encoding (clef and time signature are given for reference and are not present in the encoding).

Appearance and meaning Before encoding the notation, one needs to consider the purpose of the edition and its representation of the sources. The encoding could either describe the notation as it looks, or it could describe what it means.

Advantages and disadvantages should be discussed, for instance, whether the user would benefit from seeing the original symbols. The dilemma goes back to the discussion on notation then and now, and we therefore have to acknowledge that an edition (or an encoding) will be interpreted from the view of a modern reader. In what follows I will describe some specific problems related to sixteenth-century monophonic songs.

Musical notation containing monophonic songs often employs five or four lines in a staff. Encoding the number of lines is easy using MEI. The editor should explain to the inexperienced user that reading the music is not dependent on the number of lines even though only four lines might seem unfamiliar. Changing the number of lines from four to five in order to meet the expectation of the user might be an obvious way of transcribing the notation. Unfortunately, it has con­ 66

sequences for understanding other parts of the notation. Traditionally, four lines were used for melodies having a range small enough to fit to the staff without the need for ledger lines. In the example shown in fig.

9, the melody does not lend itself to such a definition, and consequently the clef changes from staff to staff, even in the middle of a staff in order for the melody to fit within the four lines. These clef changes are irrelevant if the melody is notated with five lines, which is why changing the number of lines as an editorial principle needs to be explained globally and even locally as in this case. Otherwise, the user of the edition will be puzzled by the seemingly redundant clef changes.

Fig. 9: A Gloria melody notated on four lines.

The clef changes seven times in this excerpt in order for the melody to stay within the staff (Vormordsen 1539, B3r). (Reproduced by courtesy of the Royal Danish Library, Copenhagen)

The visual appearance of clefs has changed dramatically over the centuries. The use of the clef seen in fig. 9, for instance, has ceased; the meaning of it, though, has been taken over by another symbol – the modern C clef. To encode a C clef, the attribute clef.shape=”C” could be added to the element <staffDef>. This might pose a problem, though, since the meaning of the clef in monophonic songs is not 67

denoting a certain pitch as the reader might be used to. In the polyphonic music of the Renaissance, the clefs need to point at a certain octave in order for the vocal parts to have the right harmonic relations.

In cases of monophonic melodies meant for a chorus or a congregation to sing, the clef seems to denote the placing of a C in a non-specific octave. Boys singing a plainchant would obviously sing an octave above the men in the choir, just like the female members of the congregation participating in singing a strophic hymn. Even an F clef could be used to denote a non-specific octave, for instance in an Alleluia melody meant for “two boys” (“Duo Pueri”) to sing (see fig. 10). If the voices of these boys had not mutated yet, they would obviously sing the melody an octave above. Keeping the original clef and modernising its appearance seems reasonable, but the user might think that the clef points at a specific octave. Therefore, a comment on the historical use of clefs is appropriate.

The attribute @clef.shape is used in MEI to describe “the clef symbol” (website: Guidelines for Music Encoding Initiative, version 4.0.1). In this particular case of monophonic songs it might seem unfortunate to employ it, as it does not only denote the actual shape of the clef but also its specific meaning, namely that of pointing at the exact pitch (for instance C4 if clef.shape=”C”). Therefore, the encoder might consider using the attribute @altsym to encode the visual appearance of the clef or referring to the facsimile using @facs.

Even if it is possible to encode a clef and leave a comment stating that it does not point at a specific octave, there is still a problem with the encoding of the notes. The element <note> makes use of the attri­ butes @pname and @oct in order to describe the pitch and octave of a certain note. One might think that one way of interpreting the non-specific octave clef would be to leave out the @oct attribute. A consequence of that, however, is that it would be impossible to describe the relationships between the individual notes in the melody, for instance whether the interval G-C goes up a fourth or down a fifth.

Consequently, we are forced to use the @oct attribute and encode one version – so to speak – of the melody according to how it is written in the source with an octave-specific clef. This decision requires a global 68

comment on the use of the attribute @oct . This and other similar problems are caused by the disparity of encoding notation and interpreting notation. The encoding schema aims to describe the notation as accurately as possible, whereas an editor might allow the notation to be as open to interpretation as possible.

Fig. 10: Two boys intonate the beginning of the Alleluia melody, and the choir continues. An F clef is used to notate both phrases. (Jesperssøn 1573, 138).

(Reproduced by courtesy of the Royal Danish Library, Copenhagen)

The editor should consider how much the digital edition should reproduce the visual appearance of the notes. If the edition renders the notes as diamond-shaped in order to replicate the look of the source as closely as possible, some users might feel alienated because of the notation, whereas others might take it as an opportunity to engage with the historical documents. If the basis for an edition is to take the perspective of a modern-day reader, the notation should also be modernised in terms of noteheads. Users who want to study the physical characterist­ ics of the sources can look at the facsimiles, which are easily available side by side on the website. Some notational elements, though, cannot simply be transformed into modern notation, for instance, ligatures.

Critical editions mostly offer interpretations of ligatures, transcribing 69

the symbols into modern notation. To indicate the presence of a ligature, a bracket is added around the notes involved in the ligature.

Encoding a ligature in the MEI mensural schema could be done by using the element <ligature>, which contains the notes that it represents. According to the present version of MEI (4.0.1), the element is interpretive, and the visual appearance of the original ligature can only be partially encoded. It is possible to encode the form (recta or obliqua) but not details on the direction of the ligature, the presence of tails, dots or coloration. Furthermore, it seems difficult to describe complex ligatures consisting of three or more notes. Even if an encoder wants to describe the visual appearance of this antiquated sign in order for the user to see the basis of the interpretation, it is not possible using the present version of MEI. Editorial interference Editorial methods in digital environments have changed how the editor works with the source during the preparation of the edition. The Lachmannian establishment of the “Urtext” is pushed aside in favour of a dynamic reading of the source. According to Patrick Sahle, the editor within a digital paradigm “does not write the edited text. Rather, it is developed gradually from the material documents, from visual evidence through the transcription and through the application of critical, historical, stylistic and philological knowledge” (Sahle 2016:31). If the “text” is developed gradually, the question is how the level of editorial interference can be determined. In this section I intend to discuss how an editor should respond to the changing of appearance and meaning of musical notation from historical source to modern editions.

The point of departure will be the perspective of the user and his or her “notational horizon”. Fig. 11 shows four categories that are combinations of whether the meaning of a notational element has changed over time or not, and whether the appearance has changed or not. The four categories are not consistent as it might be difficult to assess the exact meaning of the notation. Furthermore, a broad variation in visual appearance is often seen in the sources. The definition of the four categories even depends on the user’s knowledge and experience 70

of notation. Therefore, the following discussion is rather an attempt to draw attention to fundamental problems that an editor should consider during the process of editing and encoding.

Fig. 11: From the perspective of the user of an edition, the table shows four combinations of how the interpretation of musical notation is affected by the changing of visual appearance and meaning from historical sources to modern notation.

The row “new appearance” covers old signs that are no longer in use.

From the perspective of a present-day user with only limited experience of notation, these signs are new ones. To communicate the meaning of those signs, the editor could invent new ones assigned with a distinct and unambiguous meaning. A ligature bracket in a critical edition is a good example of this. Small accidentals above a note, on the other hand, could be misunderstood. At least six different ways of employing editorial accidentals have been used (Caldwell 1995, 57–60), making it impossible for a reader to figure out the editor’s method without an explanation. The procedure of encoding the notation proves to be a significant advantage as it enables the editor to describe the nature of the sign within the four domains mentioned earlier. An accidental therefore can be defined either as a component of the original source, as an implied meaning according to the notational conventions or as a purely editorial sign. As long as the encoding explains the relationship between the source and the edition, the digital interface could 71

replicate the sign in its original shape or by using a modernised form.

The encoding will then help the user interpret the notation if an explanation is given up front on the website.

A musical sign that has the same appearance and the same meaning then and now might not be a problem. Even in a situation like this, paradoxically enough, the editor should consider giving an explanation. Once the user has noticed that some signs have changed, he or she will begin wondering what the meaning of other signs is. This would be a positive situation, of which many editors can only dream.

Instead of risking that the user loses his or her interest in such cases, the editor should make an approach and share thoughts on the topic, for instance regarding the number of lines, explaining that it was typically related to a certain repertoire and to melodies with a small range.

The same situation occurs when a sign has a new visual appearance but the same meaning as an equivalent modern sign. The diamond-shaped notes or the odd-looking C clef of sixteenth-century notation are often modernised to comply with present-day notational standards. A short comment on the change of appearance is appropriate.

The real problem that an editor faces is how to deal with signs that have the same appearance but a new meaning. Vertical lines across the staff might look like a bar line to a modern reader. Its meaning in a historical context might be quite different. In addition, mensuration signs looking like modern time signature are risky to use, as they might be misinterpreted by the user. In cases like these, explanations up front are necessary in order for the user not to be left alone to figure out the meaning of the notation. Modernising the symbols might actually be a disservice, since it prevents the user from reflecting on their meaning.

Therefore, the editor might want to use antiquated signs or to arrange the interface in such a way that it immediately draws the user’s attention to the problem.

The role of the editor?

What I have argued in this paper is that the editor has to actively choose what to do when encoding. The decision-making is an ongoing process based on considerations from source reading, encoding and 72

from the virtual communication with the user. During this process, documentation and user interaction is obviously very important.

As editors of digital editions, we are often occupied with discussing our own role during the act of encoding or editing. We should not forget the perspective of the users of our editions, even though they might be difficult to profile. We probably have a fair idea of who would be interested in using our digital resources, or at least we know who ought to use them. We should strive to engage them; to offer them the opportunity to explore the editions; to give them a reason to invest their time and efforts in the material; and we should share our experience with them to allow them to make an informed choice that may inspire them to continue discovering new things on their own.

Scholarship can be defined as standing on the shoulders of others, pushing the boundaries of what earlier generations have accomplished. To paraphrase a point made by Peter Robinson, a scholar of textual criticism, we should turn the metaphor around: instead of thinking of ourselves as heirs of the work of earlier generations, we should think of ourselves as those giving future generations something to work with (Robinson 2016, 201). This is not to be pretentious but to acknowledge the potential of those scholars succeeding us and to engage with the users of the digital resources. We should therefore continue to rethink the form and content of the scholarly products that we are targeting. Instead of aiming for presentations of completed studies, we should design our products as the beginning of something – a collection that can be enhanced and developed by future scholars.

Digital environments make this possible.

73

Literature

Bordalejo, Barbara. 2013. “The Texts We See and the Works We Imagine: The Shift of Focus of Textual Scholarship in the Digital Age.” Ecdotica 10: 64–76.

Caldwell, John. 1995. Editing Early Music. Oxford: Clarendon Press. Second edition.

Danske reformationssalmer. Melodier og tekster 1529–1573. https://salmer.dsl.dk (accessed September 21, 2020).

Glahn, Henrik. 1954. Melodistudier til den lutherske salmesangs historie fra 1524 til ca. 1600, 2 vols. Copenhagen: Rosenkilde og Bagger.

Guidelines for Music Encoding Initiative, version 4.0.1. https://music-encoding. org/guidelines/v4/content/ (accessed November 22, 2019).

Introduction to Music Encoding Initiative. http://music-encoding.org/ resources/introduction.html (accessed January 15, 2019).

Jesperssøn, Niels. 1573. Gradval. En Almindelig Sangbog. Copenhagen: Laurentz Benedicht.

Korth, Hans-Otto. 2004. “Einstimmige wissenschaftliche Edition: Aufgaben und Grenzen der Darstellung.” In Jahrbuch für Liturgik und Hymnologie 43:212–34.

Lavagnino, John. 2006. “When Not to Use TEI.” In Electronic Textual Editing, ed. Lou Burnard, Katherine O’Brien O’Keeffe and John Unsworth, 334–8. New York: Modern Language Association of America.

Musik og sprog i reformationstidens danske salmesang. Project description. https://dsl.dk/projekter/musik-og-sprog-i-reformationstidens-danske-salmesang (accessed January 15, 2019).

Robinson, Peter and Elizabeth Solopova. 1993.“Guidelines for the Transcription of Manuscripts of The Wife of Bath’s Prologue.” In The Canterbury Tales Project Occasional Papers, vol. 1, ed. Norman Blake and Peter Robinson, 19–51.

Oxford: Office for Humanities Communication.

Robinson, Peter. 2016. “The Digital Revolution in Scholarly Editing.” In Ars Edendi Lecture Series, vol. 4, ed. B. Crostini, G. Iversen and B.M. Jensen, 181–207.

Stockholm: Stockholm University Press.

Sahle, Patrick. 2016. “What is a Scholarly Digital Edition?” In Digital Scholarly Editing.Theories and Practices, ed. Matthew James Driscoll and Elena Pierazzo, 19–39. Cambridge: Open Book Publishers.

74

Literature Litteratur

Stalmann, Joachim. 1996. Das deutsche Kirchenlied. Abteilung III. Die Melodien bis 1570. Melodien aus mehrstemmigen Sammelwerke, Agenden und Gesangbüchern, vol. 1/2. Kassel: Bärenreiter.

Standard Music Description Language. Chapter 5, Basic concepts. www.lim.di. unimi.it/IEEE/SMDL/C5.HTM (accessed January 15, 2019).

Tanselle, Thomas. 2006. Foreword. In Electronic Textual Editing, ed. Lou Burnard, Katharine O’Brien O’Keefe and John Unsworth, 1–6. New York: Modern Language Association.

Vormordsen, Frands. 1539. Handbog Om den rette Euangeliske Messe. Malmø: Oluf Ulricksøn.

Wiering, Frans. 2009. “Digital Critical Editions of Music: A Multidimensional Model.” In Modern Methods for Musicology. Prospects, Proposals, and Realities, ed.

Tim Crawford and Lorna Gibson. Farnham: Ashgate, 23–45.

Zahn, Johannes. 1889. Die Melodien der deutschen evangelischen Kirchenlieder, vol.

1. Gütersloh: C. Bertelsmann.

75

Strip and Tease: Digitally Undressing Tudor Scribes

Julia Craig-McFeely

High-resolution digital images of our collections are reshaping our understanding of how we can approach and use damaged manuscripts.

The most damaged sources can usually not be held or examined closely, but a very high quality digital image can serve as a literal surrogate, allowing us to access and even repair a damaged manuscript in a way that would otherwise be impossible. This way of interacting with manu­scripts has radically changed the landscape of document research as well as the activities and functions of archives, libraries and scholars, and it has opened the door to some exciting advances in research. This paper discusses two very different approaches to manuscripts with extreme problems of legibility and access, examining the results and processes needed to tease out enough information by stripping back damage to achieve a usable result.

In 2016 LIM Editrice produced a colour reproduction of a manuscript in Florence that was probably more influential and ground-breaking than they realised at the time.1 The publication reproduced an entire manuscript of music that had been lost when it was scraped and re-used for monastic records, leaving the music on

1 Janke, Andreas, and John Nádas. 2016. The San Lorenzo Palimpsest. Florence, Archivio del Capitolo di San Lorenzo Ms. 2211. Ars Nova, n.s., 4. 2 vols. Lucca: Libreria Musicale Italiana.

76

the original leaves as a very faint palimpsest which was only rediscovered recently as the ink began to darken enough to make some of it visible. But the reproduction does not show the manuscript as it looks in the analogue world. The book reproduces what one of the editors, Andreas Janke, describes as “false-colour” or “pseudo-colour” images.

They were created by taking multispectral images of the manuscript, then recombining the limited-bandwidth images into a single object to reveal the palimpsest music.

I believe this is the first publication to reproduce an entirely digi­tally altered version of a complete manuscript.2 Janke’s innovative work is going to be influential as much in the readings it offers as in breaking through the barrier of what we might consider acceptable as a representation of a manuscript. The publication shows that in some cases manipulated images can be a far more useful representation of a damaged manuscript than unedited RGB (Red, Green, Blue) images or even UV images,3 but it also raises new questions about what academia and the wider community want to achieve with a publication of this kind and where creators should, or might, stop in the editing process.

2 The methodology is examined in: Janke, Andreas and Claire MacDonald. 2014. Multispectral Imaging of the San Lorenzo Palimpsest (Florence, Archivio del Capitolo di San Lorenzo, Ms. 2211). Manuscript Cultures 7. The methodology was applied to another fragment and written up in Janke, Andreas, Sebastian Bosch, Claudia Colini, Oliver Hahn, and Ivan Shevchuck. 2018. The Atri Fragment Revisited I: Multispectral Imaging and Ink Identification. Manuscript Cultures 11. Pseudo-colour recovery using multispectral imaging (on non-music sources) is also being used by the Lazarus Project based at the University of Rochester, see: https://lazarus.website/.

3 Print images use CMYK (Cyan, Magenta, Yellow, blacK) to accommodate printing technology, but all image capture devices have RGB sensors.

77

Fig. 1. Italy, Florence, Archivio del Capitolo di San Lorenzo MS 2211, RGB images of fols 22v and 49v taken by DIAMM in 2001.

Fig. 1 shows how the manuscript looked when it was photographed by the author in 2001 with a high resolution scanning back. Close examination of the right-hand margin of the image on the left reveals the ghost of the lost text, and these marks can train the eye to pick out more notes in the body of the page. The other folio is far more heavily damaged, and these two images give some idea of the varying levels of destruction to the music throughout the book. There is no parity between pages in the extent of the damage such that a single process could be applied to every page and achieve the same level of results, so any attempt to recover the lost text would require a “bespoke” approach.

Fig. 2 shows ultraviolet images taken in the same session as the RGBs. These were taken under full-spectrum low-frequency ultraviolet light and captured with a colour sensor, but without the UV interference bandpass filter that is built into most cameras since that would limit the UV response. The process results in blue images, but these are rendered in grayscale (as they are effectively monochromatic in the blue spectrum) with some level adjustment to improve contrast and enhance readability for the purposes of reproduction here.

78

Fig. 2. Italy, Florence, Archivio del Capitolo di San Lorenzo MS 2211, Ultraviolet images of fols 22v and 49v taken by DIAMM in 2001.

Fig. 3. Italy, Florence, Archivio del Capitolo di San Lorenzo MS 2211, “false colour” images of fols 22v and 49v from Janke/Nadas 2016 vol. II. © Centre for Study of Manuscript Cultures, Universität Hamburg, reproduced by kind permission of the editors.

79

The major disadvantage of UV in the case of most palimpsest sources is that it darkens the overwriting as much as the palimpsest text, so although writing that has been scraped off is more visible, there is still significant interference from the overwriting. The UV has darkened the palimpsest enough to improve the legibility of what is in the background on folio 22v, but the extent of the scraping on folio 49v means that the UV is little better than the RGB image: virtually all of the palimpsest on this page (and many others in like condition) is therefore still lost.

Janke’s images of the same pages shown in fig. 3 reveal considerably more than could be seen in the previous images, even on the UVs, but his process is most noticeably successful on folio 49v, the more badly damaged of the two leaves, for which the UV image was almost completely unrevealing. Retaining the overwriting – which is most visible in the yellow layer on folio 22v – interferes significantly with any reading, but Janke’s argument for retaining it is that without it the reader would no longer be able to tell the difference between notes and rests that can be seen despite the overwriting, and those that might be there but are hidden (or partially obscured) by overlaid pen strokes (he has a couple of well-chosen examples of missing rests).

Similarly a stemmed note with the notehead hidden might be misread as a rest.4 Until this publication, those involved – and those with a research interest – in digital recovery of data from damaged manuscripts have been reluctant to publish work in print that has involved extensive digital editing or that does not obviously represent the document as it appears to the naked eye. We have been extremely cautious not to spark criticism of what digital intervention can do and have assumed,

4 For more on this see Janke/Nadas 2016, Vol. I, 11, and especially in Janke/MacDonald 2014, 117. Also, Janke, Andreas. Forthcoming. Challenges in Working with Music Palimpsests. In New Light on Old Manuscripts: Recent Advances in Palimpsest Studies, Austrian Academy of Sciences, eds. Claudia Rapp, Jana Gruskova, Grigory Kessel and Giulia Rossetto. I am grateful to the author for generously providing me with images and information about his techniques.

80

perhaps unfairly, that other researchers would be suspicious of the results.

Two decades ago I suggested that the ideal way to manage the dissemination of digitally manipulated images in the analogue world was to publish them alongside accurate unedited pictures, so that the extent and type of the editing is manifest to the end user. I still consider this to be the ideal approach to exemplifying digital manipulation in the analogue world, but in a situation where large numbers of images are involved this is impractical, so images that could be published may remain unseen if this is treated as an unbreakable rule.

The digital medium has greater flexibility in this respect, but even digital journals are reluctant to include multiple images even though that would allow editors to reproduce edited images alongside the original. The San Lorenzo publication opens the door to publishing edited or manipulated images without the expensive necessity of reproducing pictures of the manuscript in its visible state in the same medium. This is helped to a great extent by steadily-improving support for the longevity of digital delivery websites such as diamm.ac.uk where unedited versions of the images can be made available for comparison, but we cannot assume this resource will always be there. The digitally edited version of the San Lorenzo palimpsest provides a far more readable, and therefore accessible, object than true-to-life representations, thus superseding them, but the appearance in this case ensures there is no suggestion that this represents the original as it appears to the naked eye.

The question for me is whether it would have been more useful to take the multispectral light work as a foundation and create an edited digital version of the manuscript that would provide legible reconstructed readings of the lost text based on the different types of evidence available: standard high-resolution RGB images; UV images; false colour images; textual concordances with other sources.

A quick (and rather unsubtle) attempt at combining various processes to try to obtain a better idea of how folio 22v might have looked if it had not been re-used might look something like fig. 4: the music visible on all three images has been combined and the overwriting 81

removed. The top four lines have been manually “filled” to compensate for the damage done to underlying notes by removal of the overwriting; the lower three lines and text underlay retain the gaps caused by removal of the overwriting.

Fig. 4. Italy, Florence, Archivio di San Lorenzo MS 2211, fol. 22v after a combi­ nation process to retrieve the lost musical text. Lines 1–4 are more heavily edited than the latter lines.

The problem with “going further” with the San Lorenzo palimpsest is that even with digital retrieval using the RGB images combined with standard UV and Janke’s multispectral images, there is still not enough data to reconstruct the pages in their entirety. To create a performable version of the original there would always have to be editorial 82

intervention (for example to deal with lost notes/rests or misreadings occasioned by removal of the overwriting), thus the reconstruction must go beyond the information provided by the original document if a workable/playable solution is to be achieved. There are of course many ways of showing which parts of a leaf are reconstructed in this way (highlighting information that has been editorially inserted without reference to information extracted from the original document), but the act of committing something to print tends to limit our ability to consider alternative solutions. Lacunae could simply be left blank, or highlighted, but the issue of even knowing where the lacunae are (as in Janke’s problem above) remains, particularly when they constitute a tiny rather than a large area of a leaf and when there is no concordance to support a reading. For the San Lorenzo palimpsests therefore, Janke’s solution may be the only one possible in a representation of this document, leaving modern editions to reconstruct the musical text.

The time required for digital reconstruction on a manuscript as extensive as San Lorenzo (111 folios) may not justify the end result, mainly because so much of the reconstruction would be entirely editorial and therefore, to at least some extent, speculative.

A reconstruction of considerably greater extent was undertaken by the Tudor Partbooks project,5 in which more than 700 images of a set of five 16th-century partbooks were digitally edited to repair damage caused to the original documents by acid burn-through, lacing and offset.6 The results reconstruct the appearance of the leaves of the books based almost entirely on content revealed by different types of

5 2014–17, funded by the Arts and Humanities Research Council (UK); a collaboration between the universities of Oxford and Newcastle, www.tudorpartbooks.ac.uk.

6 The first printed reproductions of repairs to damage of this type were in Bent, Margaret.

2009. Bologna Q15: The Making and Remaking of a Musical Manuscript, Introductory Study and Facsimile Edition 2 vols. Lucca: Lim Editrice. The images were reproduced in grayscale in an appendix to the introductory study in support of the editions made by the author. The study accompanied a full-colour real-size facsimile, providing access to the original unedited images. The work on these images was primarily to make it possible to read enough from the leaves to create an edition of the music, so little effort was expended on niceties of appearance.

83

images of the original manuscript. Any notes that could not be reconstructed from the various surrogates available were supplied by concordances, so there was no need for editorial intervention in the content. The appearance of the images is dramatically different to that of San Lorenzo, as one might expect given the radical difference in photographic sources and output editing, but also because the condition of the original document could be revealed far more easily in the case of burn-through than in the case of a palimpsest source. The edited/ reconstructed images are finished in a manner that allows them to be used in place of the original object as performing or research sources.

On reading from a print-out of these images, one experienced performer commented that he could not tell they had been edited. While flattering to the quality of the work, this raised ethical and practical implications which are discussed elsewhere and touched on below.7 Before the early 1500s music was mainly copied in “choirbook” format with each voice part copied in a block, often signalled with a decorated initial or inset first stave line. Choirbooks were often large and unwieldy volumes, designed to be placed on a lectern allowing a whole choir of singers to see the music and perform from a single book.

Around 1500 a more intimate form of notation began to supersede the choirbook, reflecting the more intimate setting in which the music was performed. Individual voice parts were each copied into a book of their own, creating a set of individual partbooks usually with many works in the set. The loss of one book in the set can thus render all the works in the book unperformable unless the missing part for each work can be retrieved from another set.

The Tudor Partbooks project undertook two reconstructive tasks: a team in Newcastle reconstructed the lost Tenor voice book from a set of six partbooks copied by John Baldwin8 using a combi­

7 Craig-McFeely, Julia. 2020 forthcoming. Restoration, reconstruction, and revisionism: altering our virtual perception of damaged manuscripts in Disiecta Membra Musicae.

Studies in Musical Fragmentology, ed. G. Varelli. Berlin: De Gruyter.

8

Oxford, Christ Church College Library, manuscripts Mus. 979–983.

84

nation of stylistically appropriate re-composition and reference to concordances where they existed, while an Oxford team based at DIAMM worked on an intact set of books in the Bodleian Library known as the Sadler partbooks that were damaged by acidity and withdrawn from public access because of their fragility.9 Only a few early manuscript partbook sets survive from Tudor England, and even fewer are complete (only 11 of a total of 20 sets;10 there are also 16 “orphan” books11 that may or may not originally have belonged to a set), so the loss to attrition of the Sadler set, in which all the parts are extant, was particularly frustrating. The books are valu­ able witnesses to a number of aspects of music and manuscript construction that makes them worthy of closer study: they are perhaps the most heavily decorated set of English partbooks to survive from this period – and indeed any other – and contain an important reference to Thomas Morley’s birthdate which is reliable as the books were compiled in Norwich where the composer worked for part of his life. They were owned by a merchant, setting them apart from most other contemporary sets which usually have close links to the court, royal chapels or the colleges of Oxford and Cambridge and providing a rare

9

Oxford, Bodleian Library, MSS Mus. e. 1–5.

10 The following nine partbooks sets are missing one or more books: Oxford, Christ Church Mus. 979–83, known as the “Baldwin” partbooks (1580; 5 out of 6 surviving); Cambridge, St John’s College K.31 and Cambridge University Library Dd.13.27, “UJ” partbooks (1530; 2 of 5 surviving); London, British Library Royal Appendix 74–76, “Lumley” partbooks (1550; 3 of 4 surviving); Oxford, Bodleian Library Tenbury 1486 and Berkeley private collection motets “Willmott & Braikenridge” partbooks (1591, 2 of 5 surviving); Oxford, Bodleian Library Mus. Sch. e. 420–22, “Wanley” partbooks (1549; 3 of 4 surviving); Oxford, Bodleian Library Tenbury 389 and McGhie private collection partbook (1590; 2 of 5 [estimated] surviving); Cambridge, Peterhouse 471–74, “Henrician” partbooks (1539–41; 4 of 5 surviving). Later sets include the two Peterhouse Caroline sets that were considerably more incomplete until a recent discovery of a pile of manuscripts behind some sealed panelling: “Former Caroline Set” MS 47 (1625–40; 7 of 10 surviving); “Latter Caroline Set” MS 42 (1625–40; 7 of 8 surviving).

11 In fact 23 orphan books survive from the Tudor period, but many are represented only by a few fragmentary leaves from bindings which may be witnesses of a larger corpus of lost complete partbook sets or of copying and transmission practices that are poorly understood for lack of evidence or analysis; some of these “orphans” were evidently never part of a larger set.

85

insight into music-making in the merchant classes outside London during the reign of Elizabeth I. The partbooks represent music of singular importance, partly because of their origin, but also because most of the repertory they preserve is of high quality, including works by Aston, Byrd, Clemens non Papa, Fayrfax, Ferrabosco, Robert Johnson, Meerbecke, Morley, Parsley (whose works are only found in Norwich sources), Parsons, Sheppard, Tallis, Taverner, Tye, van Wilder, White and a single (nondescript) work by the owner of the books, John Sadler, who identified himself copiously throughout.

The books are decorated in a variety of styles ranging from floral motifs and Animalia rich in Tudor emblems to heavy coloured capitals.12 There are numerous concordances with other major sources, and several crucial readings of works that appear in other incomplete partbook sets (so are lacking a voice part) are complete in the Sadler set; there are a few unica from significant composers, so on the whole the books are a repository of major musical and historical interest. They are also one of the largest sets in dimensions, measuring 205 × 280 mm (around A4 size, considerably larger than usual for quarto books in landscape format). No provenance for the books has yet emerged after the death of their first (known) owner. He died intestate, but his brother (who inherited the estate) does not mention them in his own will, so they disappear from sight until they are found in a 19th century sale catalogue. After passing through the hands of various musicians and collectors, surviving a warehouse fire and other misadventures, the books were bought by the Bodleian Library.13 Despite careful husbandry, nothing could compensate for the poor materials originally used, and by the early 20th century the acidity of ink and paper had rendered the books largely unreadable and dangerously delicate, so

12 Matthias Range and Julia Craig-McFeely, “Forty years in the wilderness: John Sadler of the Sadler partbooks” Music & Letters (forthcoming), available online at www.ml.oxfordjournals.org.

13 The afterlife of the books is examined in detail in Burke, James. 2016 “John Sadler and the “Sadler” Partbooks (GB-Ob MSS Mus. e. 1–5)”. DPhil Dissertation, University of Oxford.

86

they were withdrawn from access some time in the 1960s. Disbinding and de-acidifying conservation only took place in the late 1970s, at which time the most damaged pages were stabilised with Chinese paper, unfortunately making these already-difficult passages even harder to read (see fig. 8 below).

Fig. 5. Microfilm, Mus. e. 2 fol. 16r, c. 1975

Fig. 6. Mus. e. 2 fol. 16r Modern RGB digital image taken by Bodleian Library Reprographic Services, 2012. Stabilised with Chinese paper. Image reproduced by kind permission of the Bodleian Libraries.

87

The manuscripts were microfilmed around the same time as the conservation took place, but the microfilm, being in monochrome, is almost completely useless for reading the music (see fig. 5), although details of the decoration and some notation is nevertheless discernible in a minority of the images, and some of the least damaged pages may be read, albeit with difficulty. To all intents and purposes, though, these books have been lost to musicological examination and scholarship for more than half a century.

Fig. 6, the newly captured high-resolution colour version of the same page shown in fig. 5, demonstrates the gulf between the quality of information provided by the two reproduction methods, although the legibility is still seriously compromised. (A further surrogate of this page is shown in fig. 19 below.) On un-tissued pages it was usually possible to tease out the difference between surface writing and showthrough by enlarging and looking for crystalline deposits on the surface ink, as shown in fig. 7. Tissue overlay, however, hides crystals and almost completely cancels out the difference in colour and sharpness/ fuzziness between show-through and surface writing (see fig. 8 for a detail with partial tissue overlay), though with care it was possible to discern the difference between surface and show-through for the bulk of the affected areas.

Fig. 7. Clarification of pen strokes defined by crystalline deposits. The “cleaned” version is shown on the right.

Fig. 8. Mus. e. 5 fol. 17r detail.

Partial tissue repair showing the variation in differentiation of surface ink between supported (on the right) and unsupported (on the left) paper. Image reproduced by kind permission of the Bodleian Libraries.

Of the 774 page-images, at least 700 required work to reach a state us­­ able for direct performance, ranging from 3 to 30 hours per image, with most of the remainder requiring some editing to create a parity of 88

appearance across the corpus.14 Before attempting any editing work the team had to define the appearance and intention (both in terms of audience and end use) of the final output: should a recreation of a state the manuscripts were in when they were written be attempted (was that even possible?), or should an appearance that would approximate the way these manuscripts would look today if they had been prepared using better materials be the object? This is by no means a new question, but this is the first time it has been applied to the digital re­­­ covery of a set of (music) manuscripts. The books were copied over an undefinable period leading up to 1585, and there is some evidence that by the time the last pieces were added there was some showthrough beginning to appear from earlier pieces, so attempting a “birth” snapshot was pointless. The course that was eventually decided on was to bring the books to a state that they might have been in today had the copyists used less acidic materials.

Pages with Chinese paper overlay raised a separate issue: it might be argued that since the tissue overlay had been occasioned by acid damage, then its effect on the underlying text should be editorially removed (e.g. by darkening the colour of the ink). However, reconstructing the shapes of the pen strokes on these pages was significantly more difficult than on other pages since the tissue obliterated much of the fine detail that allowed the editor to redefine the edges of the original penwork on other pages. The decision was taken therefore that the “cloudy” effect caused by tissue overlay should be retained as a visual signal to the reader that the repair in these areas should be viewed with greater caution than on pages that had not been supported with tissue overlay.

14 File sizes range from ~176 MB The images were edited using Adobe Photoshop, which allowed all the adjustments to be saved in layers that could be turned on/off and edited without touching the background layer of the master image. This meant that the unedited image could be continuously referenced during editing. The resulting files created their own issues of storage, with some complex images reaching file sizes in excess of 2 GB. Layered recovery work can be used as ground-truth training data for machine-reading of manuscripts: see the SIMSSA project, www.simssa.ca.

89

Fig. 9. Oxford, Christ Church College Library Mus. 982 p. 97. One of the Baldwin partbooks copied on printed paper issued by Tallis and Byrd. (The books are smaller in format than the Sadler set.).

Image reproduced by kind permission of Christ Church Library.

Fig. 10. Mus. e. 2 fol. 44v. Baseline image from Sadler partbooks (NB relative size relationships between this image and that shown in fig. 9 above are not preserved). Image reproduced by kind permission of the Bodleian Libraries.

90

In order to create an end result that is not only convincing but also justifiable we needed evidence of how the books should look. Fortunately there are numerous surviving contemporary sets of books that are in reasonable condition and can be used as models: the Dow partbooks15 are particularly well preserved, and the Baldwin set (the other set in the Tudor Partbooks project), although incomplete, is otherwise in excellent condition (see fig. 9). It was not necessary, however, to rely solely on external sources for baseline images: there are a few pages from the Sadler books that have not suffered from show-through, and these appear in the same section of each book where one scribe had used a different ink blend from the others (see fig. 10).

The next question was the extent to which the images should be edited: should unsightly patches be left if they did not interfere with the reading of the musical text? This question was at least partly answered by a rule of thumb that editing should remove all damage caused by acidity; thus not simply making the pages legible, but bringing them as close as possible to the way they might have looked today if they had been prepared with better materials, using the baseline images as a guide. This meant that all damage should be repaired, even if it was not interfering with the reading, bringing the edited images into line with the baselines. Quite apart from this consideration, the aesthetic appearance of a partially edited image was unsatisfying for the editor and less attractive to the end user, but there were practical benefits to the policy too: making everything look as good as possible prevents the editor from having to make potentially complex decisions about whether to leave something or repair it and how much to intervene. Variations in parity of appearance between the work of a number of editors are also obviated by a “fix everything” policy. The rule requiring removal of all acid damage applies to all the pages, not just some, so even damage on relatively readable pages was eliminated.

However, that very rule also meant that we should not correct or interfere with scribal copying errors, either ones that had been corrected by

15

Oxford, Christ Church College Library, manuscripts Mus. 984–988.

91

the scribes or ones that had not. As far as possible the books should be presented as the scribes had left them, even if that meant leaving mistakes in copying, scratchings and overwriting, since these elements are part of their genesis.

Fig. 11. Mus. e. 3 fol. 43r. Detail showing a scribal correction.

Reproduced by kind permission of the Bodleian Libraries.

The detail shown in fig. 11 is of part of a page that requires no editing: the confusion on the middle line was caused not by show-through from the reverse but by the scribe copying the wrong notes and words, blotting them out (and perhaps also scratching some of the ink off), and recopying the correct music over the top. This type of damage provides evidence of copying practices and shows that text and music were written concurrently by the same hand. Nor should the bump in the top line be straightened out, where the copyist apparently left his finger protruding above the ruler when he drew the line, or the red ink blot at the top just beneath the stave line removed.

The Sadler books have been handled and used more than the Dow and Baldwin sets, so the grubby edges of the pages were not cleaned, nor was other evidence of use removed. Non-acid-related “damage” that was retained includes water damage (which in one instance provides evidence of disturbance to the original page order), overwriting (one scribe practising the formation of note shapes by writing over work of another scribe), library stamps, fingerprints (which may or may not be those of a copyist), pasteovers and ink-spatter (see figs. 12–17).

92

Figs. 12a and 12b. Water damage; the latter illustration also shows a second scribe adding to the text underlay of the first.

Fig. 13. Overwriting (certainly not occasioned by ink fade)

Fig. 14. Library stamp

Fig. 15. Fingerprints, the first probably contemporary, the second more likely a later accretion]

Fig. 16. Blots and smudges

Fig. 17. Pasteover (there is only one)

93

Although it seemed unnecessary to repair holes in the painted/inked initials and filled finals since this did not affect readability, in the end these decorative elements were repaired for the sake of consistency of overall appearance (see fig. 18).

Fig. 18. Reconstructed decorations.

The editing process involved drawing with extreme care around every pen stroke where bleed or interference from the reverse of the leaf had obscured the original shape of the stroke (notes, letters, decorations) with a paper “pattern” created from a clean part of the page, stripping the bleed and show-through back to the original pen strokes, leaving them exactly as they should appear, in their original shape and form.16 This process has given us a extremely intimate understanding of the scribal hands, quite different from that acquired by the usual palaeographical examination of a hand. The first question that is usually

16 The technical process is discussed in more detail with accompanying images in Craig-­ McFeely, Julia. 2018. “Recovering lost texts, rebuilding lost manuscripts.” Archive Journal Digital Medieval Manuscript Cultures special edition. Eds. Michael Hanrahan, Bridget Whearty https://www.archivejournal.net/essays/recovering-lost-texts-rebuilding-lost-­ manuscripts/ (accessed June 14, 2019).

94

asked about the edited images regards the palaeographical value they have in relation to the source manuscript. Are they reliable witnesses to the work of the scribes, or are they in fact no better than a diplomatic transcription? It is easy to repair a lost note simply by cloning an undamaged one of the same value from elsewhere on the page, or even from another page. This, however, would make the final result paleographically useless and would be ethically difficult to defend, since the result is no more than a modern edition using old note shapes. Despite the considerably greater time investment required for the process employed, the end results justify that choice. Very few notes on any page were completely lost, and when they were lost the replacements that had to be cloned are clearly marked so that they can be excluded from a palaeographical appraisal of the copying (see fig. 22 below).

These images, therefore, have considerable reliability for palaeographical study and have clarified the fact that not one but many scribes were responsible for the music and decoration in the books.

The workload was far higher than had been estimated before receipt of the images and before end result criteria for the appearance of the editing had been established. This presented the team with a serious problem: would the lack of time and editors compromise the quality of the work? Without any prior contact with the Sadler books, estimates had been based largely on guesswork based on the microfilm, but seeing the colour images it became apparent that the manuscripts were in considerably worse condition than had been expected and far more of the pages than had been estimated would need work. A conservative re-estimate of the time required was closer to 5,000 hours of work than the 850 that had been budgeted for.

Thanks to an opportunity to present our work to an early music society, an untapped resource of enthusiastic amateur musicians came forward who were fascinated by the project and wanted to help. Members of the team presented the project and its problems to early music groups across the UK and posted to message boards reaching those in other countries interested in early music. More than 50 volunteers came forward, from all walks of life. Many had never used image processing software, and some of the project hours were allocated to 95

creating a series of training videos for those who could not come to Oxford for training.17 It was not possible to provide the volunteers with full copies of the software because of cost, so staged processes were developed that could be undertaken on old computers running the “Lite” version of the software; leaving the internal team, and volunteers with their own software, to manage the fine tuning.

The ways in which these diverse users interacted with the project team and the problems of digital editing were extremely informative in terms of how this work affects, and can be affected by, the non-academic community. Volunteers came from all walks of life, from marine engineers to professional cooks, students or housewives to a retired graphic designer. It reminded us of the extraordinary reach of music from this period, the attraction it holds for non-professional musicians and the vitality of original sources in the hands of those who would not normally have such intimate contact with its transmission.

For many, the technical challenges of the work were as fascinating as the music they were repairing. Engaging with sources as an expert editor can lead to a blinkered view of the manuscript, but the volunteers reminded us of many things such as the aesthetic view of the original scribes. They reminded us that people who do not interact with manuscripts on a daily basis have no concept of the fragility of the materials behind the music that they perform and listen to, and how easily these things can be lost. The volunteers learned new skills both in using modern software and in reading 16th-century music notation and secretary handwriting. They produced “manuals” to help each other to read the texts and find shortcuts around the software. In the case of

17 Tuition videos were distributed through the project website (www.tudorpartbooks. ac.uk) and the Tudor Partbooks YouTube Channel (these are updated from time to time with revised versions of the original videos, and films showing new or more advanced techniques for experienced Photoshop users. https://www.youtube.com/playlist?list=PLVIQO75esTNYJTEW180Kj7LtmEBN8hMEt) These are currently being used, along with images from the DIAMM collection, by the multimedia laboratory of the Department of Cultural Heritage, University of Bologna (www.framelab.unibo.it) for their course in Image Processing and Virtual Restoration, part of the Library and Archive Science Degree. A time-lapse reconstruction (2 hours 20 minutes reduced to 8 mins) is included, showing the entire process from original image through to finished reconstruction.

96

one volunteer, the simple physical requirement of manipulating a mouse with care and accuracy coupled with the worthwhile goal of the end result led to regaining motor skills lost after a stroke.

Some of the highest quality editing work has been done by people who do not read music, since they do not interpret what they see and therefore do not inadvertently correct the notation. The fine mouse control (we were unable to supply graphics tablets) and meticulous care required meant that many volunteers dropped out, but nine contributed very significantly, and the project could not have been completed without them. The use of such a diverse team coupled with the large quantity of images affected all the decisions that were made about the editing and methodology, since there were constraints not only of time but also of the varied expertise of the team, their access to software, and their ability to learn new skills. The input of the volunteers required the evolution of more efficient techniques and the partial automation of at least some of the process, but most of all they progressed the work to such an extent that the quality of the output did not need to be compromised.

Forensic Reconstruction The work on the books began with the term “restoration” in mind, but it is not possible to restore a page without an accurate target picture of what it originally looked like; despite the baseline images this information was not available for each page. Although much of the writing was discernible and could be recovered with confidence, areas of severe lacing, large holes or solid dark patches could only be conjectured based on the shape, size and position of the patches, albeit with a fair degree of certainty thanks to the relatively predictable musical vocabulary of the time and because it was possible to transcribe and align all the parts. It became clear that the extent of the necessary editorial intervention and the level of damage repair meant this process could not be described as “restoration” but was rather “reconstruction”. The shift in terminology was liberating, since it allowed the editors to take a more aggressive approach: removing all the show-through (not just that which was obscuring music), which made it possible to implement 97

more or less automated pattern fills for the early stages of the work.

The resulting increase in turnover was instrumental in making the project feasible. Although the original pitches and values could be recovered, it was not possible to “restore” the true appearance of the lost areas of every page, so it was necessary to look further than the images immediately to hand for information.

The master high-resolution RGB images had sufficient colour depth for software to differentiate between colours that the naked eye could not; the high resolution meant that pages could be enlarged such that very fine detail could be seen, allowing editors to disentangle the majority of questioned readings. These images were the primary source of information, and approximately 90% of the text could be recovered from these pages. In many cases readings could be supported by applying processes such as thresholding (extreme brightness and contrast adjustment), level adjustment (darkening and lightening) on selective colours, colour balance or contrast adjustment to the master, creating a supplementary surrogate.

Tudor Church Music Photostats In the early 1920s the Carnegie Trust made a grant to the editors of the Tudor Church Music series to make negative Photostats of some pages of the Sadler books required for their editions. These black and white prints have survived in Senate House Library in London.18 They were scanned at high resolution, and the resulting images were superimposed on the master RGB images. Fig. 6 shows a particularly damaged leaf with some sections that were extremely difficult to read. These

18 London, Senate House Library, Tudor Church Music Box 48. The exact date of the imaging is unknown; the approximate date comes from identification of the handwriting of a librarian on labelling in the images who retired in the late 1920s. We are immensely grateful to the department of Special Collections in Senate House Library and its administrator, Charles Harrowell, who have preserved these Photostats and kindly allowed the author to visit them with a high-resolution scanner to make scans for the reconstruction project. Information about the TCM collection may be seen here: http://www.senatehouselibrary.ac.uk/our-collections/special-collections/printed-special-collections/tudor-church-music-collection (accessed November 1, 2017).

98

were much clearer in the Photostat, fig. 19, which shows that many of the current holes in the leaf were not present a century ago; the Photostats also gave us an indication of shrinkage and movement of the paper in the last 100 years, although the shrinkage and cockling of the Photostats themselves created challenges in superimposing them on the original.

Fig. 19. Photostat image of Mus. e. 2 fol. 16r (cf figs 5 and 6 above showing microfilm and RGB images of the same leaf). Reproduced by kind permission of Senate House Library, London.

Even when show-through is heavy the Photostats provided a less damaged reading, and inverting the image to positive sometimes assisted in reading. On the page shown in figs. 6 and 19 the Photostats enabled us to read everything except the music lost under the capital at the end of the first line. A few areas are more difficult to read as they are obscured by gauze from an early attempt to stabilise the damage; the gauze was lifted during the 1970s conservation project, but its impression still remains on some pages (see fig. 22).

However, the Photostats introduced a problem to what had hitherto been a relatively straightforward reconstruction evidence chain: the Photostat shows what appears to be a paler ink adding four breve 99

shapes to one of the decorative finals (fig. 20a). Inverting the image to positive (fig. 20b) confirms that the latter string of four breve shapes are indeed paler than the main symmetrical block of four breves comprising the original final, and the darkness of the original group cannot be attributed to the show-through from the reverse since the reverse notes do not coincide with the notes of the final (fig. 20c). Yet the modern RGB image (fig. 20d) does not show this discrepancy because it has been overlaid with Chinese paper. Simply cleaning around the notes means that the discrepancy that is showing something about the behaviour of this group of scribes is lost (fig. 20e). Should the colour discrepancy hidden by the tissue a) overlay be reinstated in some way? This would falsify the information conveyed by the RGB images in a different way than by simply cleaning around b) the shapes of the pen strokes, and it is not possible to be certain that what can apparently be seen on the Photostats is what it appears to be since it cannot be c) verified in the modern image. All that can be done, therefore, is to point it out the discrepancy and its source in the commentary to the print output. d)

e) Fig. 20, Detail of Mus. e. 2 fol. 16r, a–e from top down. a) Negative Photostat; b) Photostat inverted to positive; c) negative Photostat with different ink layers highlighted: blue original final, green added notes, red show-through; d) new RGB image; e) reconstructed image.

100

Infrared images Because the manuscripts are still withdrawn from access, the project was only allowed to make infrared images to confirm readings that could not be deciphered from the other surrogates. For the most part all that remained uncertain were areas where there were holes, in which case infrared imaging would not have helped; therefore only 30 leaves were re-photographed under infrared conditions, and in each case the infrared was needed for only a tiny portion of the page. These images were extremely revealing, not only in elucidating questioned readings, but in confirming what we already suspected about varying ink types, since each ink type responded at different levels of intensity on the infrared spectrum. They also provided a view of the ink strokes as they had originally been made on the paper, before ink bleed, that showed a more elegant and refined script than is now apparent. Unfortunately the leaf shown in fig. 20 was not one that was photographed under infrared light.

Fig. 21. Mus. e. 4 fol. 12v detail, From L to R: RGB, Photostat and IR images

The example shown in fig. 21, the opening of Hugh Aston’s Te Deum (this is the only source for the Tenor voice part for this work and in other partbooks the piece appears with a different text), is of a leaf where the Photostat was unhelpful partly due to gauzing on line 4, the most questioned part of the leaf. The infrared image clarified almost all of the page, confirming what had already been retrieved from the 101

RGB and Photostat images but elucidating the one remaining questioned area at the start of line 4 (see detail in fig. 22). The IR images were fascinating, and it is clear that a complete set of infrared images of the manuscript would be highly desirable in supporting palaeographical study of the books.

Extrapolation Where a note could not be read, it was often possible to deduce both value and pitch from the visible context. In a string of minims, for example, a single missing notehead could be deduced in value from positioning, and in pitch from the length of the tail, by eliminating pitches that were visibly clear of notes, and by applying knowledge of the musical vocabulary. The issue at the start of line 4 shown in fig. 21 was particularly problematic as these are notes without tails, and there are no other sources that can be used for comparison. The area is shown in fig. 22. An earlier musicological reconstruction had suggested a solution, but it was clearly very different from what must have been written here.

Fig. 22. Detail of Mus. e. 4 fol. 12v. from L to R: RGB, Photostat, infrared, reconstruction

The boxed note must be on the second stave line up and is not in either of the vertically adjacent spaces; it has no tail, although there are clearly tails erased above the notes. The note must be a void breve or semibreve; the context does not allow for a colored (i.e. filled black) note.

The Photostat is obscured by gauze. The infrared image confirms the two diamond notes following the boxed note, and it also confirms the absence of vertical sidestrokes necessary if the missing note were a breve; it also confirms the region (i.e. pitch) in which the note was written. The missing note can therefore only be a semibreve on the second line up.

102

The value of the note is confirmed by transcription of all parts, since any other value pushes the polyphony out of alignment, and in this example the erasures, due to obvious dittography, are also reconstructed. The only note whose shape could not be seen on any of the surrogates is boxed in green to show that its shape has been cloned, even though its pitch and value are not in question. The solution is not as grammatically suitable as the earlier musicological attempt to reconstruct the work but perhaps confirms Aston’s less-than-first-rank reputation.

Contemporary concordances Contemporary concordances form a layer of information external to the Sadler books themselves. Variants between concordant copies of the same work are common between contemporary sources, but the Baldwin partbooks provided readings (except for the missing Tenor part) that were close enough to the rest of the music that they could be used for checking or support in deciphering difficult areas. Indeed, there is an unusually high number of concordances between the Baldwin and Sadler books, and now that both books are equally readable a new evaluation of their relationship is possible.

All decisions were founded on loyalty to the original, but the simple time-vs-result equation had a major bearing on decisions about methodology, as did the logistics and market for the paper output, and wear and tear on the eyes of the editors. Every pen stroke had to be painstakingly checked and rechecked against the unedited image, concordant sources and modern editions to ensure that pale marks, sharps, dots etc. had not been inadvertently eliminated or notes had been filled in that should be void. Finally, facsimile copies were sung or played from to pick up anything that might have been missed in the previous checking stages. Copying errors made by the scribes were retained, but are footnoted in the publication so that performers are not unnecessarily derailed by musical errors.

From their first known modern appearance in a sale catalogue in 1849 these books have been described as being “in the hand of John Sadler”, but it is now possible to establish that the books were not 103

copied by him. They may have been copied under his direction, but this seems less likely than that he acquired them when the main body was complete and added various identifying markings, including his merchant mark which identifies him as a Norwich Grocer. Sadler added gatherings at the front and rear with new music (the main body of the book has a continuous numbering sequence that does not include the outer fascicles) and decorated the books generously with his name.

He may have been responsible for the first binding of limp vellum, made from dismembered pieces of an earlier parchment choirbook.19 The decorators are another group – possibly drawn from the first group, but changes between decorators do not on the whole correspond with changes between the music scribes, so the copying and decorating appear to have a complex layered genesis. Now that the handwriting can be examined, it is possible to see the differences in the ways pen strokes were executed and other idiosyncrasies in the hands.

There is abundant evidence of the use of these books as working copies: they were not “good copies” saved for posterity20 but performing books. Frequent sections of overwriting indicate their use as script models, in this case probably to allow a group of scribes to emulate each other’s hands in order to create a uniform finish.

There are sections where the counting is written in to help an inexperienced singer or player or to support passages of long held notes (much as in the way large sections of rests in modern orchestral parts are given counting numbers); slurs and alignment marks are used to

19 The original bindings are now stored separately: Oxford, Bodleian Library Mus. e 21.

20 Scholarly argument is somewhat divided on whether manuscripts containing copying errors could therefore never have been used for performance. Common sense would suggest this is the case but perhaps denies the possibility that performers were more intelligent about their reading than we are, and were able to retain memory of errors they had discovered without making corrections to manuscripts that might also have a calligraphic display aspect to them. Consensus regards the set copied by Robert Dow as one that was never played from, since the state of preservation suggests it was not extensively handled and there are copying errors that render some works unperformable. See Milsom, John.

2010. The Dow Partbooks, Facsimile with introductory study. Oxford: DIAMM Publications; Mateer, David G. 1986–7. Oxford, Christ Church Music MSS 984–8: An Index and Commentary. Royal Musical Association Research Chronicle 20: 1–18.

104

clarify the underlay; there are signs supporting rehearsal and performance such as “section” numbers, and there are both signae congruen­ tiae and marks similar to lute graces that may have been inserted by players using the books as viol consort repertory (see fig. 23).

Fig. 23. A selection of the contemporary marks and signs employed to support performance of the works.

This is the first time a project of this kind and on this scale has been attempted, and it is certainly the first time that reconstruction at this level will appear in print. It might have been feasible to edit these images so that it was possible to use the result as if it were the original while maintaining something in the presentation that ensured a reader understood it was a heavily edited version, but this would have compromised the appearance with little obvious benefit. With an edited corpus of this size it is simply impractical to reproduce the whole thing in both edited and unedited form, although the unedited images (along with the original bindings) can be examined in zoomable high-resolution colour at https://www.diamm.ac.uk/sets/127/. For those without easy internet access (and considering the possibility that DIAMM may not be online indefinitely) each of the published partbooks includes a selection of unedited images in an appendix, and the reconstruction process is described in detail in the introductory study, although there is never any guarantee that a user of the books will read this. One can hope that the title of “collaborative reconstruction” (since many editors worked on each image) rather than “facsimile” might obviate any 105

misconception about the reproduction, though the quality of the finish is such that the extent of the intervention can only be fully grasped by viewing both versions of each image side by side.

Reconstruction at this level is immensely time-consuming, and therefore expensive. It requires editors with an obsessive-compulsive mindset and a highly ethical critical approach to their work, but the ground truth data provided by the layered editing will be fed forward towards future computationally-generated processes that will increasingly alleviate the level of individual intervention necessary in an application of a similar process to other sources in the future. This project and the innovative combinative imaging processes of Andreas Janke have shown that badly damaged manuscripts are no longer as lost as they used to be.

Fig. 24. Mus. e. 3 fol. 22v, one of the most damaged leaves, before and after reconstruction work. Shapes that could not be recovered from any of the surrogates are boxed in green. Original image courtesy of the Bodleian Libraries, reconstructed image by the author, Laurie Clifford-Frith and Ken Lewis.

106

Preserving Digitally Inscribed Music

Jøran Rudi

Musical notation as a prescriptive tool has existed for at least 4,000 years, aiming to preserve both musical ideas and performance practices. The notation system for instruments and voice that we use today is hundreds of years old and has over time been developed into the complex sets of instruction that one can find in contemporary scores, where new musical expressions are pursued through unusual excitation methods and increasingly detailed performance control.

This notation system has also found its way into the digital domain, and much research effort has gone into developing methods for migrating paper-based scores to machine-readable documents. However, digitally inscribed music often relies on more than pitch, duration, pulse and rhythm, and much of the modern electronic expressions cannot be captured or reproduced using conventional notation.

In order to simplify discussion in this article, the terms techno­ logy-based and technology-dependent music are used as common denominators for all music that originates in electronic music technology. The terms overlap with mixed music, where acoustic instrument parts are mixed with electronic sound and, in the use of technology, also overlap with music for acoustic instruments that has been composed with notation programmes on a computer. However, acoustic music composed in this manner does not rely on electronics for its sound, technology has only been a practical tool.

1 07

Electronic technology – computers, instruments, machines, software and code – has developed rapidly since the digital paradigm started to make headway into music during the 1970s and 1980s. Technology becomes obsolete, script or code can no longer be compiled, and as a result much of the music has become unavailable and can no longer be performed. Harddisks and floppies can no longer be read, and the software needed to read old files will not run on new machines.

This is especially crucial for music where sounds and compositions are generated or processed in real time; where the only notation is the computer code or the instrument makers’ proprietary formats. Much music repertoire from the last 25–30 years is rapidly disappearing, and the understanding of the developments that brought us to where we are today is becoming incomplete.

This article will discuss the urgency of preserving the heritage of digital music, present early projects and practices in this area, and provide recommendations for future efforts in securing more up-todate and complete records of musical history in formats that also facilitate performability, so that the music can remain an active part of our cultural heritage and be heard and understood by future generations.

Preamble My work in music technology started in the late 1970s, and I have been organising, promoting and composing digital music in Norway since my return from computer music studies in New York in 1990. This background has given me intimate knowledge of technological and musical development internationally and in Norway, and in 2019 I published the first book on electronic technology and music in Norway: Elektrisk lyd i Norge – fra 1930 til 2005. (Rudi 2019b) In December 2016 my then colleague Notto Thelle and I gave a presentation at the conference Interfaces: Tradition and technology in musical heritage work, organised as part of the project Norwegian Musi­ cal Heritage, in which The National Library was a partner. The main aims of the musical heritage project were philological research, critical editing and publication, and preservation of musical scores through digitisation. The project has now ended after having published an 108

impressive collection of preserved manuscripts that would otherwise have been lost and remained uninterpreted.1 The project fits well within the musicological tradition of developing critical editions and preserving notated scores for posterity, with descriptions of relevant performance practices.

Our presentation was called Digitally based music: Archiving, migra­ tion and performance, and our aim was to start a discussion about digital music and the performance practices with software and hardware that are rapidly becoming obsolete and also unavailable in all forms.

We also touched on issues regarding digitising and preserving recorded sound and pointed to the need for adequate media migration and formatting.

Technology is now a significant part of music in nearly all its forms, in notation, performance, production and recording, and there is a rich heritage of music that has been made with technology and could not have existed without it. In what we can call technology-based or technology-dependent music, the integration of technology goes even deeper – in combinations of compositional concepts and tools, in the detailed construction of sounds and timbres, and in the often intricate connections between data, micro (timbral) levels and macro (structural) levels. Structural relationships are played out on several levels and linked so that changes in one spot might ripple through the entire composition. Traditional notation is not useful in describing such works, and questions emerge: how do we sustain these new types of music and musical practices so that the few works that survive the merciless selection processes of history can be performed in the future?

How should these compositions be performed, and how can we deal with the obsolete technology? Which methods of reconstruction, substitution and simulation can be used to best preserve the composers’ intentions?

1 More information about the project can be found at this address: http://www.musikkarven. no/english/about/ (accessed June 8, 2018).

109

Notation and tradition Since the 1950s, much research has been done on Cuneiform tablets, and it has been established that musical notation on these kinds of tablets from Mesopotamia is at least 4,000 years old, perhaps even older.

Cuneiform writing was developed in the Babylonian city of Uruk, now in Iraq. Kilmer & Civil (1986) has published an analysis of a particular tablet that was found in Syria in 1950, and according to them the tablet contains a hymn to the moon god Nikal as well as instructions on how to perform it.2 The instructions on performing the hymn are fragmentary, but it seems clear that the music was composed in harmonies of thirds and written using a diatonic scale.

Fig. 1: The Cunei-tablet with a section of A.D.Kilmer’s transcription. Images of this tablet can be found on several websites, and this one is taken from http://www.openculture.com/2014/07/theoldest-song-in-the-world.html (accessed December 17, 2018).

2 Kilmer, Anne Draffkorn, and Miguel Civil (1986). “Old Babylonian Musical Instructions Relating to Hymnody”. Journal of Cuneiform Studies, 38, no. 1:94–98.

110

The intention 4,000 years ago was much the same as is it is now: to preserve music through a prescriptive and normative system for how the performance should be done in order to make future performances align favourably with the original. The reasons why this music was written down can be many, but there is reason to believe that the music preserved in this manner was linked to the social hierarchies of the time and to the cultural canons that described why things needed to be as they were. An investigation of social and political aspects of musical notation is certainly an interesting direction for research. However, it falls outside the scope of this article.

Western notation emerged in the mediaeval catholic church, and the beginnings of the system used today are often attributed to the monk Guido of Arezzo (995–1050) and his invention of staff notation.

Since this early plainchant notation, musical script has grown in complexity and continues to develop even today, with detailed notation of interpretation and excitation techniques. The function remains the same as with the Cunei-tablet, however: notation preserves and makes music available for performance in a prescriptive manner that allows performances to be as close as possible to the composer’s intention.

However, any of us who has written scores or performed from them knows how coarse these instructions really are and how much they depend on the interpretation and skill of conductors and musicians.

For successful realisation, notation depends on interpretation, and this interpretation exists within a performance tradition or (as a minimum) in a similar cultural sphere. Consequently, music from outside a given tradition will easily lack authenticity in performance and not sound the same as the original. In Norway the perhaps clearest historical examples of this are from folk music, and how the skewed, uneven rhythms and non-chromatic and approximately just-intonated intervals are poorly captured by notation. When folk music is appropriated and used as inspiration and raw material for compositions notated in equal temperament, as in many of the works of celebrated composer Edvard Grieg, for example, many of the essential characteristics of the music disappear. One of the aims of the national romantic tradition that Grieg was part of was to construct a new canon of 111

“Norwegian-ness” as a cultural underpinning and firming up of the idea of the nation state. This was done by combining authentic elements and expressing them through an internationally recognised musical paradigm.

This type of appropriation was criticised by the folk music community, and among others, Eivind Groven was highly critical. From his point of view as a fiddler, composer and musicologist, he made it clear that folk music lost key aspects of its character when being squeezed into equal temperament and even rhythms, and to him this reduction of timbral and rhythmical detail could be likened with reducing the colour in paintings, for example. The way the music sounded was changed by the way it was notated.

Fig. 2: These images of Monet’s haystacks show how a reduction of colours results in increased graininess and starker colour contrasts, completely changing the nuances in Monet’s original.

(Author’s illustration.)

The link between tool and result forms the basis for the cultural canon, and Western musical notation clearly faces challenges when it encounters music that is different from the music it was shaped by. Music from other cultures with other scales and rhythms has fared even worse than Norwegian folk music when being written down in Western notation. However, it should come as no surprise that a tool developed for a specific purpose is not well-suited for purposes other than that it was designed for. We also see this in the Western contemporary tradition, where composers often augment and extend the notational system to be better able to represent their ideas for expanding the music.

These days notation is most frequently done electronically, and scores are normally printed for musicians to perform from. Notation 112

on paper is a robust representation, and preservation efforts in scanning and digitising old and newer manuscripts are many, as several other articles in this collection describe. A further step in these preservation efforts is to develop machine-readability of scanned scores and mapping into MIDI-data, which is essential for automatic performance and perhaps even more important for musicological analysis.

With advances in machine learning, these digital transcriptions can be material for automatic search and musical information retrieval, which opens up avenues for interesting and comprehensive research in what is often labelled systematic musicology. The processes of making scans machine-readable and developing reliable automation are by no means trivial, but they are well defined and easy to comprehend, and progress is steadily being made. With pattern recognition added to the equation, advances in interpretation and performance have come rapidly, as exemplified by the software Wekinator3, which can recognise and recreate articulation from performances or unrelated gestures, and Wave­ net4, which can generate the sound of piano performance, for example.

Understanding how machine learning can be used to support creativity and become an interesting “partner” is a new challenge, although it is perhaps even more tempting to see what machines can create on their own or with human interrupts only rather than the strongly defined learning goals often implemented today. The current state of the art is mostly focused on training the machine-learning software on existing music and on coding it for maximum similarity between the learning material and the machine learning results.5 With several of the other texts in this collection focusing on digitisation and migration of printed and handwritten scores, there is not much point for me to describe this in more detail. Instead I direct the

3

http://www.wekinator.org (accessed January 24, 2019).

4 https://deepmind.com/blog/wavenet-generative-model-raw-audio/ (accessed January 24, 2019).

5 A Beatles copy from Sony CSL studios can serve as a fun example. However, it is worth noting that it is only the harmonic grid that has been automatically composed – the arrangement, text and song are by humans. https://www.youtube.com/watch?v=LSHZ_ b05W7o (accessed January 24, 2019).

113

gaze towards how music technology has changed composition and performance and how technological inscription has radically expanded the challenges for musical heritage initiatives. It is only little more than twenty years ago that digital technology started to change the way contemporary music generally was composed and performed, although the pioneering genre computer music is close to 60 years old. This new and soon-to-be all-encompassing digital platform resulted in new methods for composition and performance, and preservation and archival efforts must follow musical practices in order to create future possibilities for understanding this part of our musical heritage, its preconditions, how it was made and how it has evolved over time.

However, there is currently a lack of good systems for documenting and preserving an adequate amount of digitally dependent music for future analysis and possible performance, and our scope for understanding this part of our musical heritage will be significantly weakened if the necessary steps are not taken.

Inscription in technology – a quick historical overview Today staff notation is often created on computers, and scores can be printed out wherever there is a printer with the appropriate paper size.

Preservation of these digitally born manuscripts is not difficult as long as standard file formats such as PDF are used and the fonts in the document exist on the machine’s harddisk. However, if a composer stores the notation in a proprietary file format for a specific software and the software stops working, or where new versions of the software are not sufficiently backwards compatible to allow old files to be opened, chances are that even manuscripts with standard Western notation are lost.

In digital notation, the core elements of pitch, duration and rhythm are the same as in previous centuries, although often augmented by the new practices of contemporary composers. Inscription of processes with sound in technology is considerably more complex than mere pitch notation, and the new digital methods facilitate music creation with new affordances in terms of accuracy, complexity and control structures that go far beyond the paper-based or analogue 114

electronic technologies of the past. For a more detailed investigation of these issues, please refer to Rudi (2019a).

Electronic instruments, electronic and concrete music With the invention of electronic and concrete music, sound events replaced the notated pitches as the basic musical unit, while acoustic music remained interval-focused. Since then, several new terms have been coined in order to better describe the variation and development of this expanding concept, and electroacoustic, sound-based, acousmatic, and computer music all describe direct descendants of early concrete and electronic music. With the growth of technology skills amongst wider user groups, electronic dance music and electronica have come into existence together with more countercultural glitch and noise music and more visually oriented sound art and acoustic art based on environmental sound environments. A fine cultural grid is needed to outline the differences and descriptors of numerous sub-­ genres. Social aspects play an important role as well, as much of this development stems from the enablement of broader user groups.

It is the emergent qualities of sound spectra that carry the narratives in these new music genres; they are not represented though written scores. It should be mentioned, however, that for some genres, particularly those that bridge over into the visual arts, the interaction between abstraction and recognisability remains important, bringing aspects of representation into the works and consequently reducing the aspects of absolute music derived from the spectral changes alone. One example of which there are several examples can be listening to the action of writing with pen on paper, where temperament and intensity cannot always be identified in the written text itself but can be easily heard. Another example can be very soft sounds from our environment that are amplified so that we can hear them and gain a deeper understanding of how various biotopes function.

With this absolute focus on sound, preservation of sound quality is essential, since any reduction of sound quality means degradation of the music itself and consequently that any reduction will reduce our 115

ability to perceive the qualities of the sounds as they develop. Future scholarship on technology-based music will depend on perceiving these qualities and on having the music available in as authentic condition as possible.

Reproducibility is the holy grail of preservation, and most types of electronic music face the same challenges as notated music: inscribing music in a prescriptive manner so that future performances can be realised without too much deviation from the composer’s intentions.

The necessary technological tools must be available together with the necessary supporting materials and instructions for performance.

With all this in place, the question of how to interpret the music in a different social and acoustic context still remains, much the same as with acoustic music. Several approaches have been documented: Laura Zattra (2004) has designed an analytical method for electroacoustic music and provided several examples of the usefulness of her method, Michael Clarke (2006) has developed interactive methods that rely on rich source material, exemplified by an elaborate work on Jonathan Harvey’s Mortuous Plango, Vivos Voco. An issue of Computer Music Journal (31:3) was largely dedicated to analysis of John Chowning’s pioneering composition Stria and features five articles on different strategies and aspects for analysing this piece. (Chowning 2007, Zattra 2007, Dahan 2007, Badouin 2007, Meneghini 2007).

Inscription of music into electronic technology is older than both concrete and electronic music, and Jaime Oliver (2018) traces this development to the early electronic instruments, more specifically to how the musical possibilities were defined by the circuit boards of the theremin. The theremin is a particularly interesting example because the instrument did not have a keyboard for performance and relied on abstracted gestural input from the performer. The Theremin did not have one specific timbre, but by building affordances into the electronics, it was authored for several timbres. Oliver describes this new level of abstraction, where the sound was removed from the acoustic source, as a practice of transduction through electronic schematics – and how this transduction inaugurated enormous changes in Western musical practice, blurring the boundaries between composer and performer, 116

instrument and score, as well as between instrument maker and engineer. Oliver refers to McLuhan (1964): “The theremin … introduc[ed] modularity as a design philosophy and of the schematic as an operable and transmittable code.” The musical affordances of the instrument were inscribed in the electronic circuitry, and one can say that the the­ remin inaugurated the electric era in music and the pre-figured practices of electric instruments that are present to this day.

Fig. 3: Leon Theremin in performance. The instrument is surrounded by an electromagnetic field, and in order to optimise control, the performer must stand perfectly still except for moving hands and arms for precise control of the sound. (https://www.classicfm.com/discover-music/latest/weirdest-musical-instruments/theremin/)

Something similar can be said also for several of the other early electronic instruments, but since most of them have keyboard interfaces, the theremin stands out as the most radical design. It is also the only early instrument where the circuit designs circulated freely, following a short attempt at commercialisation. This free circulation was a new practice that pointed forward to the DIY approach of today, including the hacker and maker movements. The new electronic instruments 117

from the 1920s were not thought of as what we today would call disruptive technology, rather they were designed to serve as supplements to the acoustic instruments and aimed to find their place within the acoustic tradition. They did not at the time result in any particularly significant musical development.

The first great leap in music technology development came with the ability to work directly with sound– to capture (record) a sound, edit and repeat it. The first developments in this direction were wax rolls, tin foil, steel wire and celluloid film, but it was lacquer discs and magnetic tape that gave composers the tools they needed. Concrete music was invented at radio stations at approximately the same time as electronic music, just before, during and immediately after WWII. The invention of concrete music is most often credited to Pierre Schaeffer, and although he was certainly not the first to make music from recorded sounds, it was his work that had the widest-reaching consequences. In electronic music the early genealogy is less clear, but a composition practice where electronic sound-generating equipment was used to form the music developed at radio stations across Europe.

In electronic music it was easy to work in an exacting manner to control the sound material and its development, while this strong execution of principles and structures was more difficult in concrete music where the raw material was recordings of sounds from the acoustic world.

All these electronic timbres were imagined within the constraints of the technology and could not be performed outside of these affordances. While notation was adequate for the interval-based new serial music at the time, it was largely unusable in concrete music and electronic music. Notation did not capture the essential qualities of timbral development in sound-based composition and made no provision for frequencies and noise bands replacing equal-tempered pitches as the basic building blocks. The music could be represented graphic­ ally, however, and Karlheinz Stockhausen’s score for Studie II may serve as an example.

118

Fig. 4: This short excerpt from Karlheinz Stockhausen’s Studie II is represented here by an amplitude mapping (bottom), frequencey mapping (middle) and a sonogram (top). The sonogram shows the entire spectrum, with darker areas being louder.

(Montage by the author.)

Computer tools allowed for new types of representation and formalisation, first as computer code for generating note lists that could be reformatted into notation that musicians could perform from. Following this beginning of computerised algorithmic composition, generating and processing sound became the next building block of computer music, and the capabilities expanded with faster processors. In Norway composer Kåre Kolberg produced the first completely synthesised computer music piece of this kind in 1973, The Emperor’s New Clothes, a few years after Arild Boman started his artistic investigations with computers and created Ecumene (1969) for jazz ensemble and computer-­ generated sound and score. Real-time performance with computers was still out of the question, so synthesisers were the workhorses for performances during the 1970s, and when MIDI protocol for musical communication was invented and released in 1983, the integration of computers and synthesisers became possible also outside the research laboratories. (Interestingly, in Norway the University of Oslo’s Musi­ kus project launched in 1974 was a clear precursor to MIDI. However, 119

the project’s focus was more on musical analysis than on developing practical tools for musicians.)6 As computer processing and storage capacity continued to grow, digital analysis of sound also started to become practical, and textual input remained the only method for interaction with the machine until graphical interfaces made big headway from the late 1980s onwards.

Since then, development of computer graphics has been very much driven by the gaming industry, which now rivals the film industry in terms of annual revenue.7 With development of graphic interfaces, users could “see” the sound itself, select aspects, frequency and time regions, edit, process and compose by way of more or less intuitive interfaces, and new characteristics of sound became attractive arenas for artistic exploration.

The 1990s heyday of the computer music genre came as a direct result.

There were constant advances in new computer systems and affordances, and new methods for signal processing made it possible to fuse the strict algorithmic focus from electronic music with techniques from concrete music. Sounds could now be processed with the same rigour previously only applicable to sound generation. My own works When Timbre Comes Apart (1995)8 and Concrete Net (1997)9 are prime examples on this trend, in particular because they employ computer graphics to augment the connection between sound and digital processing. As the computer came closer to becoming a full musical

6 Vollsnes, Arvid and Lande, Tor Sverre (1988). Music encoding and analysis in the MUSIKUS-­ system, Oslo: University of Oslo.

7 Depending on how one calculates the value of spin-off and auxiliary products, computer games annually generate revenues larger than the Hollywood film industry. In 2017 games generated more than USD 108.4 billion, while the US movie industry generated 43.4 billion.

See: https://www.gamesindustry.biz/articles/2018-01-31-games-industry-generatedusd108-4bn-in-revenues-in-2017 and https://deadline.com/2018/07/film-industry-revenue2017-ibisworld-report-gloomy-box-office-1202425692/. For a view sceptical of that claim: https://www.quora.com/Who-makes-more-money-Hollywood-or-the-video-game-industry.

All links accessed December 19, 2018.

8 http://www.joranrudi.no/language/nb/when-timbre-comes-apart/ (accessed November 12, 2019).

9

http://www.joranrudi.no/language/nb/concrete-net/ (accessed November 12, 2019).

120

instrument, new types of representation in addition to traditional notation became useful – piano rolls, sonograms, spectrograms, computer code as text, and graphic representations of computer code in Max, PD and KYMA, to name a few high-level programming environments.

What were once contemporary practices has now been converted into digital heritage, and the rapid march of technological development makes preservation crucial. Using the above example of Kolberg’s pioneering work, The Emperor’s New Tie, the piece has already become difficult to read and impossible to realise – the synthesiser that made the sound has been dismantled, and a computer interpreter for the code does not exist. Fifty years from now, if not sooner, the same will most probably apply to today’s practices if adequate preservation measures are not taken.

Interactive and collaborative practices The development of technology-based music did not stop with what is now described as “old-school” works that are typically performed from stable media such as magnetic tape, CDs, DVDs or hard disks. The old dream of interacting musically with the computer in real time came steadily closer to realisation during the 1990s, shaped as mixed music and improvised performances where musical sounds were generated on the fly. With the opening of the Internet for non-academic use, the Internet rapidly also became an arena for artistic exploration, and webbased music and art became new genres; their essence being generative and dependent on input from visitors via browsers. (The desire to interact musically across distance has had clear historical precursors in Norway, too, and a work such as Arne Nordheim’s Forbindelser (1975) should be mentioned. Musicians in several cities performed different parts of a musical work, and these musical “streams” were brought together in a TV programme celebrating the 50th anniversary of the Norwegian Broadcasting Corporation (NRK).

The first musical collaboration on the Internet with participation from Norway was Res Rocket Surfer, an internet band of several hundred musicians across the world. The musicians sent and received 121

data to and from MIDI loops, and while the sounds resided locally, the control was global. The analogue, continuous control introduced

by Leon Theremin had been replaced by numbers. Another, and significantly more complex, example is the concert WHO – HOW – WOH (1998)10, which was a collaboration between the Sibelius Academy in Helsinki, the Warsaw Autumn Festival and NOTAM in Oslo. This was a type of research concert developed around transmission of sound and different types of control data where one of the purposes was to gain experience on how well the Internet was suited to this type of collaborative performance. One could expect significant time delays in transmission of signals between the locations and that different data streams would experience different delay times. The biggest challenges concerned transmitting digital sound, where the delay time between all three cities was 10–11 seconds. For example, it would take 20 seconds for the cellist Paulin Skoglund Voss in Oslo to hear her own performance returned from Warsaw together with the musical response from there, and 10 seconds after that she could hear the same thing over again, with the added response from Finland. The project was ambitious and showed that this type of concert could be done. However, the result was most successful with pieces written for significant time delays and without much focus on tight timing. A later development in Norway of this idea of distributed performance was the World Opera project – where the goal was for sound to be accompanied by data that should make it possible to show virtual avatars of remote singers in the different locations. The project’s aim was to realise conventional performance in a distributed, digital domain, and it did not explore new affordances of remote collaboration.

Distributing notation as MIDI-data was also explored in generative music in the many-faceted Integra project,11 where the score of one of the commissioned works was generated in real time and distributed

10 http://archive.notam02.no/warsaw/ (accessed January 28, 2019).

11 http://www.joranrudi.no/mediefiler/The%20Integra%20Project%20EMS_2011.pdf (accessed November 12, 2019).

122

to the performing sinfonietta ensemble via computer screens placed where the note stands would normally be. This work would change with every performance and became a process rather than a finished entity.

New data types Up to this point, the focus of this text has been on musical notation and digital signals generated for musical purposes. However, the binary system allows all data to be interpreted in different domains.

For example, one can “play” a picture with relative ease, print music as an image, or interpret database information as musical control data.

Generally speaking, data can be used as musical inscription – descriptive and prescriptive. These possibilities of arbitrary data mapping have opened up a new field of interaction design, where technology gets wrapped in ways intending to relate to human predispositions and preconceptions, biological and social. Some researchers also believe that there is musical cognition buried in the theatrics of gesture, such as make-believe performances on imaginary instruments. The result of these more or less arbitrary mapping processes is that all types of data can serve as types of musical notation, depending on a computer interpreter programme that formats the data for sound generation or processing. This is not esoteric but has become common practice through the use of sensors for capturing pressure, light, wind, humidity and movement, and so on. An early example of such mapping processes in Norwegian computer music is Rolf Wallin’s piece Yo (1994), where he performed a set of computer algorithms (structured as a Max patch) by stroking ribbons of conductive plastic mounted onto a “control suit,” where he could also touch a few strategic­ ally placed buttons. By way of the patch, he controlled the granulation of sampled Fig. 5: Rolf Wallin performing on material, and the dynamic notation data the first version of his control suit, was thus generated in real time from his stroking a strip of plastic on his arm. (© Eli Berge) control suit. The musical instructions 123

embedded in the use of sensors were inscribed in the technology as a set of possibilities and could not easily be notated precisely for identical future performances. This reduces the prescriptive aspect of the notation, and the representation and preservation of instructions for these types of performances become a challenge. How can we develop better representations than the ones used today?

The essence of live performances with technology is that the data streams change in real time. The most explicit genre is live coding, where listeners immediately hear the sounding results as the displayed computer code is modified and the software cycles through the loop structures. This type of change is in effect what happens within any digital instrument as it is being played, but it is easier to see and understand the changes as one can see them in the code. Live coding is a type of parallel representation of data – as text and as sound.

Fig. 6: This Chuck-code emulates the Roland TR808 bass drum sound and allows for real time changes of the instrument. The code can be found at http://kurtjameswerner.tumblr.com/ post/50274769999/chuck-tr-808-emulator-bass-drum-bd-emulation (accessed January 28, 2019).

The page contains a detailed description of the how the analogue circuitry is represented in binary code.

124

In many genres of modern technology-based music, spatial characteristics are essential and must be taken into account. This is especially important in traditional acousmatic music where spatialisation has been a significant part of performances since the 1950s. With developments from the last ten years or so, more precise techniques of wavefield technologies allow for higher precision than before, and performances have changed from real-time interpretation from a mixing console to pre-programmed placement and movement in precise loudspeaker constellations. The essential aspect of this type of precision is that composers manipulate the phase of the sound in order to simulate movement and position of the virtual sound sources, and for this type of music this is key in the compositions themselves: spatialisation is not a feature that is merely added on in performance. In order to preserve this part of the digital heritage and understand the composition methods, the spatial information must be preserved, in both text and code.

In broad terms these developments exemplify the expansion of music into the wider domain of sound-based art with new performance practices, new data types and new notions of what constitutes music in the digital cross-cultural orientation in the arts. There is no reason to assume that this expansion process is likely to slow down or stop. Machine learning, for example, is likely to make significant inroads into musical composition and performance, and because of its hidden forwarding layers in the decision-making, machine learning adds another dimension of abstraction to how music can be composed and performed. In a deep sense, the questions of digital notation come down to developing an understanding of the impact of technology in creation. Arguably, this is the most pressing question in contemporary music.

Preservation and performability As discussed, musical notation is both descriptive and prescriptive; it is simultaneously both a written work and a record of an imagined future performance. The historical paper-based notation has been shown to be robust, but the instructions must be interpreted for performance, 125

while digital inscription can often be used directly. Nonetheless, contextual information is important even there. As with conventionally notated music, preservation and archiving of technology-dependent music has three types of goals: 1) preserving data, 2) maintaining readability of these data, and 3) making the data available for musical interpretation.

This is a complex undertaking; there is a plethora of technologies, and what constitutes relevant data might change from work to work, even between performances. Software changes between versions, computers and instruments become obsolete and unavailable, and storage formats and storage media change as well. A useful preservation strategy includes the questions of why, how and what and underlines the understanding that mere preservation of documents and digital files without a clear path towards future performability is not enough. In addition to the digital files, contextual information about composers and musicians, their aesthetic projects, ambitions and aims can on occasion support a deeper understanding of the music. A reasonably large collection of all this information can make it possible to reconstruct the music with regard to the composers’ ideas, too, and not be limited to actual sound. For example, in cases where composers have left behind graphical scores or other detailed descriptions of the compositions, contextual information is essential for interpretation – what the composer’s equipment allowed him or her to do, and which sound worlds s/he was familiar with. Composer Bjørn Fongaard, for example, was unable to realise his colour handdrawn manuscripts due to lack of technical means. (Much like Antoni Gaudi’s cathedral La Sagrada Familia (1882–1926) in Barcelona, where a number of ceilings could not be completed with the technology available at the time but has been completed only recently.) Fongaard must have heard the sounds he drew quite clearly in his head, judging from his written notes and the high level of detail in his drawings.

Thus, an interpretation with the ambition of capturing what Fongaard must have had in mind depends entirely on knowledge of the nuts and bolts in Fongaard’s toolbox, as guitarist Anders Førisdal discovered in his reconstruction of several Fongaard works for 126

quartertone guitar.12 Interpreting digital representations of music depends on the same type of contextual knowledge, and when instruments, software and computers cease to function, the inscription in technology and the notation for it become impossible to interpret.

Media rot Magnetic coating disintegrates and loses its magnetisation over time; it is not a question of whether a tape or harddisk will break down and crash, but of when. However, preserving recordings and maintaining performability of works from fixed soundfiles is trivial, although remastering and cleaning of recordings is often necessary for meeting the low tolerances for unwanted noise that characterise the performance standards of today. Cleaning and remastering are sensitive activities, since details in the soundfiles can easily be washed away together with the noise. Other types of digital files such as texts and images on CDs, DVDs and harddisks are exposed to hazards as well, as media will disintegrate over time and render the files unreadable.

This problem affects all types of electronic information, and considerable effort goes into preservation work at technology museums worldwide.

Electronic instruments, computer systems and software are developing rapidly, and it is timely to remember that it was only as late as in the mid-1990s that this technology gained popular acceptance – a little more than 20 years ago. Hardware and software instruments become obsolete, and working copies can be difficult to find. From my work, I know for example that several planned performances of Lasse Thoresen’s piece Abuno (1994) have been cancelled because the instruments used in this piece are very difficult to find. Another example is Nicolay Apollyon’s interactive works for instruments and the NeXT computer from the mid-1990s. They are absolutely unique in Norwegian electronic history and are not realisable today without a major

12 The works have been published here: Anders Førisdal: Galaxe. ACD5068

127

effort in locating functioning hardware and software and reconstructing the works.

Fig. 7: This excerpt from Kåre Kolberg’s code for The Emperor’s New Tie (1973) can no longer be executed. The computer has been scrapped, the software erased and the sound generating equipment discarded. (Scan by the author.)

Computers and software are intertwined, and digital scores require functional interpreters. The first computer music piece from Norway, The Emperor’s New Tie (1973) by Kåre Kolberg, can no longer be rendered or performed from the score, since the computer is no longer in existence and the sound generating equipment has been discarded.

The same goes for composer and director Knut Wiggen’s pioneering works from the same studio; his written scores address another computer interpreter no longer in existence. Naturally, one can make quite qualified guesses about what the particular numbers in Kolberg’s computer printout mean and in this way develop ideas about the structural thinking, but the scaling of the numbers is unknown, and sonic verification is not possible. Thus, his notation is no longer immediately useful, and the only functioning representation of this piece is a recording. Uncovering organisational principles in the music becomes difficult when based only on sounding material, although notation systems for describing emergent features in the music exist.13

13 Norwegian composer Lasse Thoresen has developed a notation system which he describes in full detail in his book Emergent Musical Forms: Aural Explorations. (2015).

128

The examples of Wiggen and Kolberg are from the 1970s, but more recent digital file formats and representations also disappear at a rapid rate, and when the technology no longer exists, the music disappears if it has not been recorded. This is the case with many of my own works as well, realised on Silicon Graphics machines that no longer work, and/or in software that has not been ported to modern computers and operating systems.

Digital inscription has shown to be more vulnerable than paperbased notation, and the changes that music has undergone with this new technology are already difficult to trace and analyse at a score and inscription level. The intellectual processes fostered and facilitated by the new technological situation will be difficult to document and discuss in the same close manner as is possible with paper-based notation.

Clearly, it cannot be a goal that all music should be preserved, but in a musical heritage perspective it is necessary to secure enough material to make future performances, analyses and reflections on the early technology-based music from 1970 onwards possible, and since the tools and methods of the future are unknown, this effort must be quite encompassing.

Some recently formulated aims and initiatives Finding good methods for preserving technology-based mixed compositions and works with live electronics is not a trivial task. Composers use a plethora of methods and technologies in their work, and musicians often rely on electronic instruments and devices that have limited shelf lives. As an added challenge, electronic timbres and articulations will change from performance to performance in live music, much the same as the local acoustic conditions of different performance venues change the spatialisation in performances of fixed-file acousmatic music.

Combining the skills of composer, performer, instrument maker and technology developer has been key in computer music since the 1970s, and in many new forms of technology-based music it still is.

The willingness to transgress borders between disciplines and combine skills is an important element in the musical branch of the maker 129

movement, and there can be no doubt that unconventional thinking has brought new ideas as well as concepts into music, particularly in process- and installation works. However, these diversities in hardware and programming have increased the challenges for preservation and documentation efforts, and because instruments and coding are often known only by the person who made them, this affects future performability. In broad terms one can say that the capture of soundbased music performance is more descriptive than prescriptive. The captures of performances provide only superficial information, as there is no necessary link between the sounds from the electronic instruments and the movements necessary to perform them; the links are designed in much the same way as the sounds of the theremin. The link between sound and excitation has become arbitrary and what soundscape pioneer Murray Schafer has described as schizophonic.

Digital heritage has come into focus, and several initiatives for documenting musical works and preserving the materials necessary for securing future performances have been mounted. The first systematic documentation, Les cahiers d’exploitation, was developed by Marc Battier at the French IRCAM centre between 1991 and 2002, and the aim of these manuals was to secure performability of the mixed music written at IRCAM at the time. IRCAM has since that time also been involved several documentation projects such as the European project Caspar (2006–2009), which focused on collecting materials and presenting them in a technology-neutral manner. This project was followed by a collaboration with Groupe Recherches Musicales (GRM) in Mustica (2003–2004), organised as part of the worldwide Interpares project which concerned digital records in general. The ASTREE project (2008) was a collaboration between four French institutions and concentrated on live electronics and documentation by way of GRAME’s Faust code. The principal idea was to generate computer code for reconstructing electronic sound transformations and to include this code in scores for mixed music. The project successfully represented several musical works, both mixed and synthetic.

The current preservation efforts of electroacoustic music at IRCAM collect structured data in the Sidney database. (Lemouthon et 130

al. 2018) Several data types are required in this database: information on composers and performance versions of the work, different data file types for versions of software and hardware, as well as different performance files and video recordings of performances. IRCAM engineering staff upload all the information types as part of their work assisting composers, and an administrator oversees the entries for quality control.

Other recent documentation initiatives include New York University’s Electro-Acoustic Music Mine (EAMM) (Park and Underwood 2018), which collects recordings of works with the purpose of making these available for performance and research in music information retrieval. This initiative is not limited to works from only one institution but will use conference and festival submissions from a number of professional conferences and festivals as a starting point. It is foreseen that a later development will allow for crowdsourcing in the sense that composers themselves submit works. Crowdsourcing is also essential in Virginia Tech’s recently formulated COMPEL project, which is designed to be a community-built repository with the aim of ensuring future performability (Bukvic and Ogier 2018). It is easy to see that the interest in preservation and future performability is growing worldwide and that the number of repositories is rising.

In order to ensure performability of early works, they must be migrated from obsolete to current technology. In addition to the purely technical aspects, this also hinges on musicological information about performance practices and other contextual data. The Integra project (2007–13) aimed to migrate historical works and to commission new ones. Commissioned composers and engineers were expected to develop a set of tools according to a documentation guideline, and in this way the Integra collection was expected to grow gradually and become useful for future composers. The toolset was initially imagined as a superimposed layer that would address DSP routines in several sound engines such as Max, PD, Supercollider etc. by using a namespace communicated through the XML protocol. (Rudi and Bullock 2011) Thus, the technology would hide the basic functions and be easy to use while lowering the threshold for composers and musicians with 131

little familiarity of even high-level programming and object-oriented environments. However, the creation of this type of superstructure resulted in too much computational overhead, and the idea was abandoned. However, the software that was developed at a later stage in the project is freely available on the web14 and has maintained the ease-ofuse approach initially imagined.

The problem of maintaining performability for older technology-­ dependent works has not yet been solved, although there are a number of successful singularities – migrations of individual works. From a concert perspective these works are good starting points for expanding the catalogue, but the challenge remains: how to ensure that these works can be part of the repertoire in the more distant future, too.

Suggestions for a documentation project for technology-based music The different musical approaches in technology-based music and the documentation initiatives mentioned above make it clear that a number of different data types need to be preserved for works to be performable in the future.

A documentation project in Norway should be coordinated with international initiatives in order to make the repertoire more readily avail­ able for performance and to include it in the pool of materials needed for future research on new methods and tools for technology migration, new search criteria and musicological work. The visibility and usability of works from the Norwegian digital heritage will increase with international affiliation, and research collaboration will be simplified. Challenges in documentation that arise from new work and performance types can then be met collectively, and as the internationally distributed collection grows, structured metadata and development of new search criteria will make it possible to see musical connections across

14 http://integra.io/integralive/ (accessed February 6, 2019).

132

different contexts and genres and to facilitate research and scholarship from a broader perspective than the focus on singular works.

Key decisions are: • • •

Selection of database structure Determining data types for standardisation Establishing procedures and guidelines for data gathering and quality control.

Database In terms of choice of type and structure, it is advisable to coordinate this with other existing initiatives. In Norway the National Library has several databases for music, and it should be considered whether these could be fused and migrated into the new standard over time. In practical terms this means that data types from these databases should be included in the new initiative. This would be a forward-looking approach, since an increasing percentage of new works are created with technology, and from a current perspective the separation of music with or without technology seems obsolete in itself.

The different initiatives that have been mounted internationally have not been well coordinated, and the technology used and developed shows large variation. For example, the Sidney database mentioned above is structured into a MySql database, through the Object Relational Mapping provided by Django, while Virginia Tech’s COMPEL project uses the open-source Samvera solution for flexible combination of components and Apache Solr for search and indexing. The solutions are many, and it would make sense for the National Library to initiate broad contacts and form a consortium for a thorough and informed evaluation.

Data types In technical terms, there are several genres of technology-based music, and in aesthetic terms there are even more. A brief (and not too detailed) list of technical genres includes: 133

• • • • •

orks for fixed media – algorithmically generated and concrete W music Mixed music – acoustic instruments with electronic live proces­ sing or fixed file playback Generative music – algorithmic composition for acoustic instruments and/or electronic processing Improvisation works – live electronic music not inscribed in scores, with or without custom programming and electronics Installation works – custom electronics and configurations

The most basic documentation of all electronic music is a sound recording of the required number of tracks, and for all future performances this is an important reference. It tells us how things sounded but not much about how the sounds were made. Recordings are easy to make and preserve, and their transfer to digital media must include a migration plan for the future. However, for documentation to also have a prescriptive function, several other information types are required.

The composer can provide notes on artistic intention, either separately or embedded in programme notes. Often, these notes also include some type of analysis that points out salient thoughts or sounding elements. Essays, reviews and other texts will also be valu­ able in establishing the intellectual space the music has been performed in.

For acousmatic music or purely electronic music, performance notes and soundfiles for coding to speaker configurations must be preserved, and particular requirements for speaker placements must be kept on record as well. Ideal loudspeaker setup and all hardware requirements for performance must be described along with recordings from different performance situations where possible.

Many works are accompanied by written instructions for set-up and operation as well as manuals and riders for the technology.

For fixed-file music, diffusion notes and descriptions of loudspeaker configurations are often available. In mixed music there is scored material and performance instructions plus notes by the musicians 134

for performances. For installation works there are mounting instructions and possible notes on acoustic requirements.

Computer and instrument types and models must be recorded together with sound libraries and software programming such as “patches” in Max, PD etc. with notes on software version and comments in the programming code itself. Further descriptions in everyday language of what the technology does might be the most important documentation in the more distant future, when current software and computers have become obsolete and disappeared from use and exis­ tence. Screenshots from the actual computer performance files should be added to the database.

Huge challenges follow in the wake of the musical practices with new controllers for live use, since much of this technology is custom built and lacks documentation. Here the sound and video material is necessary for describing the works, and the hardware and software must be described in both everyday and technical language.

Validation is needed to ensure that the database records are as complete as possible, and rich data on contextual information will be helpful. There is an arguable continuum between prescriptive and descriptive data and between data on musical motivation, composition and performance, and technology that has been used. (For example, one could write that music is realised within technological frameworks and that instructions for compositions and performances are pre-­ inscribed. These types of discussions are interesting in musical analysis.

However, a more thorough discussion on this perspective goes beyond the scope of this publication.) A brief listing of different data types includes: • Musical motivation - Composers’ notes on ideas, intent and programme content

- Programme notes from concerts/performances/shows

- Analyses of the music

- Contextual information on musical genealogy and tradition, ­references to composersand traditions

135

• Methods

- Composer’s sketches, notes

- Musical notation, score

- Performance instructions to producer, tech rider

- Performance instructions to musicians and diffusionists - Screenshots of software, programming and technical set-ups Also descriptions in everyday language on signal flow, aims and effects

- Audio recordings of performances

- Video documentation of performances • Employed technology - Software types and versions, also with descriptions in everyday language - Hardware, computer types, models, versions of operating systems - Software programming and patches, everyday language description of functions and signal flow

- Soundfiles, sound libraries for instruments

Selection of works One challenge when it comes to documentation is selection, and from an historical and research point of view, it is important to document multiple music genres and how they develop. Much of the computer music from the 1990s and early 2000s has now disappeared from view, and the aims and goals of the composers, as well as their intellectual approaches to composition, have become obscure. This pioneering period is rapidly becoming forgotten, and the current development of technology-based music in Norway will seem disconnected from its early years in the 1960s and early 1970s if this heritage is not preserved and understood. For these historical periods, a curated selection of a number of works for migration and documentation is needed, and establishing the beginnings of a canon requires a curatorial effort with participation of the National Library of Norway as well as the Norwe136

gian Society of Composers (NKF) and the Norwegian Society of Composers and Lyricists (NOPA). Ideally, this initiative should be structured as an extension of the musical heritage project and also include representation from higher music education.

Curated content should form the core of the database. However, it should not be limited to a curated repertoire. With today’s rapid expansion of music technology into new fields and practices, a curatorial committee should oversee that core works are included, but for a broad and more encompassing database, composers and performers should be encouraged to enter musical works themselves. This type of “citizen-science” approach should be easy to implement and sits well with modern technological skill sets, where self-publishing in participatory Internet media has developed over the last twenty years.

A broad and large representation of works would benefit future research on music information retrieval, and composers should be encouraged to take responsibility for maintaining future performability of their music, as is already common in the more commercial sectors in the music business where presence on social media is important for maintaining interest among followers and fans. Some composers will undoubtedly be more eager than others to include their works, and the collection will most probably be skewed, but as long as the curatorial effort of genre inclusion and aesthetic balance is maintained, the negative consequences of skewing will be manageable.

In order to generate interest in entering information in the music database, different measures for motivation should be tested, and some concrete evidence of the benefits must be presented. A concert series, a series of music analyses, a series of publications and a streaming service should all be considered in order to promote the preservation project to both contributors and users.

Future research Over the last twenty years generations of software and computers have come and gone, new file formats have been developed and others have disappeared, media formats have been replaced by new formats – and as a result, much music can no longer be performed as it was at the 1 37

time of its creation. The problem has been recognised, but it has no simple solution.

Migrating works to a technology that will enable their performance in the future is a major, unsolved problem which the research initiatives discussed above have been unsuccessful in solving on a general level. In Integra, several works were migrated to modern technology, but on a singular basis – the idea of generalisation was abandoned.

The ASTREE project revolved around representing synthetic music mathematically. Its approach proved successful for another subset of technology-based music but remains non-applicable to the broader field.

There is significant international interest in the issue of preserving digital heritage, and it is expected that machine learning will be useful in migrating data for technology-based music in the future – possibly also in developing common standards for sound, code and data for composition and performance. A successful documentation project for technology-based music should be developed in collaboration with computer science departments around the country, and master and PhD students could be attracted to tackling challenges in this field. The National Library of Norway should also establish international collaborations within its network of institutions with similar missions and tasks, and the professional music community could contribute the first-hand musical expertise necessary to capture the essential characteristics of the music. This is a growing field with great potential for development in combining research on sounding music with representation issues. It should also be of interest to departments of musicology.

It is not a particularly radical assumption that machine learning will change how music is made, how it is adapted and how humans will use it, and that there might be other interesting parameters for describing music than those we are using today. Ongoing research and development in this area might also contribute to the migration processes where older works are migrated into current technology.

138

Conclusion Technology-based music with musical inscription in technology has been underrepresented in heritage and preservation initiatives, despite the documented vulnerability that results from rapid technological development and continual revolving of tools. As a consequence, important parts of the history of early digital music are no longer performable or available as subjects for reflection and research. Furthermore, there is every reason to believe that this trend will continue if a documentation and preservation initiative is not initiated and appropriate action taken. There can be no doubt that this will weaken our understanding of our early digital heritage.

A documentation and preservation project will be well served by a combination of performances and publications reflecting on and describing this cultural heritage. The initiative can easily be linked to other initiatives internationally and also be positioned for research on methods for music representation as a collaboration between relevant institutions in Norway and an international consortium.

139

Literature, recordings

Baudouin, Olivier 2007.“A Reconstruction of Stria.” In Computer Music Journal.

31:3. 75–81.

Bukvic, Ivica I. and Andrea L. Ogier 2018. COMPEL: “A Crowdsourced Community-Building Repository for Reproducible Computer Music.”

In Proceedings of The International Computer Music Conference, Seoul. 130–133.

Chowning, John 2007. “Stria: Lines to Its Reconstruction.” In Computer Music Journal 31:3. 23–25.

Clarke, Michael 2006. “Jonathan Harvey’s Mortuos Plango, Vivos Voco.”

In Simoni, Mary (ed.) Analytical Methods of Electroacoustic Music. London: Routledge.

Dahan, Kevin 2007. “Surface Tensions: Dynamics of Stria.” In Computer Music Journal Fall. 31:3. 65–74.

Førisdal, Anders: Galaxe. Aurora ACD5068.

Groven, Eivind 1968. Renstemningsautomaten. Oslo: Universitetesforlaget.

Kilmer, Anne Draffkorn, and Miguel Civil 1986. “Old Babylonian Musical Instructions Relating to Hymnody.” Journal of Cuneiform Studies, 38, no. 1:94–98.

Lemouton, Serge, Alain Bonardi, Laurent Pottier and Jacques Warnier 2018.

“Electronic music documentations, practices and models.” In Proceedings of The International Computer Music Conference, Seoul. 123–129.

McLuhan, Marshall 1964. Understanding Media: The Extensions of Man. New York: McGraw-Hill.

Meneghini, Matteo 2007. “An Analysis of the Compositional Techniques in John Chowning’s Stria.” In Computer Music Journal, 31:3. 26–37.

Oliver, Jaime 2018. “Theremin in the Press: Instrument Remediation and Code-Instrument Transduction.” In Organised Sound, 23:3. 256–269.

Park, Tae Hong and Paul Underwood 2018. “The Electro-Acoustic Music Mine (EAMM): An Initiative for the Archival Preservation of Electro- Acoustic Music.” In Proceedings of The International Computer Music Conference, Seoul.

138–141.

Rudi, Jøran and Jamie Bullock 2011. “The Integra Project.” Proceedings of the Electroacoustic Music Studies Conference, New York.

140

Literature, recordings

Rudi, Jøran 2019a. “Representation, complexity and control – three aspects of technology-based sonic art.” In Toft, T. (ed.) Digital Dynamics. Chicago: Intellect/ University of Chicago Press. 161-178 Rudi, Jøran 2019b. Elektrisk lyd i Norge fra 1930 til 2005. Oslo: Novus.

Schafer, Murray 1977. The Tuning of the World. New York: Knopf.

Thoresen, Lasse 2005. Emergent Musical Forms: Aural Explorations. London, CA: University of Western Ontario.

Zattra, Laura 2004. “Searching for lost data: Outlines of aesthetic-poietic analyses.” In Organised Sound 9:1, 35–46.

Zattra, Laura 2007. “The Assembling of Stria by John Chowning: A Philo­ logical Investigation.” In Computer Music Journal. 31:3. 38–64

141

Computational Musicological Analysis of Notated Music: A Brief Overview

Olivier Lartillot

I present a short overview of computational methods for musicological analysis of notated music. We first need to clarify the various levels of computational representations of music: on one side, notated music, on the other, audio recordings, and in the middle, a note-level representation of music performance where higher-level musical descriptions are absent. The article provides a synthetic and partial panorama of the different types of music analysis that have been systematised and automated using computers. While pioneering works were mainly focused on statistical descriptions of the surface of music, other dimensions of music analysis such as harmony, metre and structure have been taken into consideration since. I conclude by sketching my personal vision of the future of computational music analysis.

Musicological importance of computational approaches The use of computers in music analysis can offer various benefits. It automates aspects of music analysis that are particularly cumbersome to carry out manually. Besides, the automation enlarges the perspective by enabling analyses that can be significantly more extensive (through the analysis of very large music catalogues) and intensive (studying each piece of music at a note level). But automating music analysis is far from trivial. On the contrary, it remains an open problem, far from 142

being resolved. One very interesting underlying issue is that auto­ mation requires an explicit formalisation of the core process under­ lying music analysis. Computers can also be considered a means for developing and testing existing as well as new theories related to music analysis and music cognition.

Articulating computational music analysis with traditional methods enables us to reconcile historical evidence with content-based data. Besides, the analysis of large corpora allows the design of powerful tools for content-based search.

Levels of music representations Music can be represented in various ways. Notated music, or sheet music, is a traditional form of representation of music, embodied into scores. But at the same time music is first and foremost a performance that often (but not always) consists of a rendering and interpretation of a predefined score. The large degree of freedom added by the interpretation itself forms a significant constituent of music.

In improvised music, music from oral tradition as well as electro-­ acoustic music, to name a few examples, the score does not play the role of a reference representation anymore. When a score is established from these types of music, it is not “prescriptive” but “descriptive”: it can be considered an attempt to describe what has been played or produced; this description generally covers a part (often minimal) of the music that was created and performed. Studying the computational representations of music therefore requires us to consider these two opposite poles in music representation: notated music and music performance.

As for music scores, various representation formats exist depending on the purpose of use. Music notation software, enabling us to input and edit sheet music digitally, was initially designed to represent music merely as a combination of graphical elements, with the unique aim of printing the result on paper or on screen. The graphic appearance of a score can vary significantly without changing the actual musical content (for instance by simply modifying the horizontal spacing between notes). Hence a large part of those graphic 143

considerations is of no importance with respect to the representation of music itself. Other representations of music focus solely on the part of music score description that relates directly to the musical content itself, while ignoring the graphical considerations that have no impact on the way music is played. Reference formats in Western musical notation are MusicXML (Good 2001) and MEI (Music Encoding Initiative) (Hankinson et al. 2011), which were preceded in particular by the Humdrum **kern encoding scheme (Huron 2002).

Music performance, the opposite pole of music representation, can be simply encoded in the form of audio recordings. Evidently, this representation, reduced to sound waveforms, does not contain any explicit representation of the music. But listeners can detect most of the notes and the underlying musical structure. Algorithms have similarly been designed to automatically detect those notes and perform music analysis. Thanks to the automation of music production, a new type of representation of music performance, provided by the MIDI standard, allows us to describe the particularity of each performance while at the same time representing the actual notes that are played. It basically indicates at which exact time (in seconds) each note appears, with characteristics related to each note such as pitch height, dynamics, instrumentation, duration etc.

This MIDI representation, while building a bridge between the opposite poles of notated music and music performance, is at the same time less rich than each of those poles. First, this decomposition into discrete notes does not do justice to the actual audio performance, where each note might feature a subtle dynamic timbral evolution, for instance. The notes are represented on a temporal axis that enables an exact rendering of the performance in terms of time and pitch. The underlying metrical grid can be represented as well, on the other hand, the harmonic context is ignored. Other aspects generally represented in the score, such as the decompo-sition of the music into voices, are sometimes absent as well.

Conversions from the opposite poles of computational music representation are possible, but challenging. Starting from notated music, an interpretation of the score would require a choice concerning 144

the tempo and its dynamic evolution over time, as well as the dynamics of each note, among other aspects. This leads to a MIDI representation, which can then be turned into an audio recording by selecting a synthesis technique (a particular instrument, say) for each voice. An automation of this process would generally lead to very robotic music instrumentation, although recent advances in automated music rendering show some progress.

Reversely, going from the audio recording to the notated music is also extremely challenging. This problem is often known as automated music transcription. To oversimplify, this could be understood as a process in two steps: first converting the audio into MIDI, and then MIDI into score. If we ignore the first step, which is by itself highly challenging, we can decompose the second step into three sub-problems. First, reconstructing the underlying pulsation and metrical grid, enabling us to locate each note in the metrical grid and infer the rhythmic value (e.g. crotchet) associated with each note duration (e.g. one second). Second, reconstructing the underlying tonal or modal structure (or at least part of it) in order to represent each pitch height in a diatonic scale, which enables us to distinguish between C# and Db, for instance. Finally, the rhythmic value of each note in notated music is represented with respect to the subsequent value.

This next event can also be a silence, which has itself a rhythmic value and a succeeding event, note or silence. It is therefore necessary to chain the notes together into voices.

As we can see, transcribing music is already a form of music analysis.

Panorama of computational music analysis Computational music analysis started as soon as computers were made available. Pioneering works can be traced as far back as 1937, for instance, which is when a scholar named Otto Ortmann wrote an article entitled “Interval Frequency as a Determinant of Melodic Style” (Schüler 2005, 34). For several decades the early works were focused on statistical and information theory analysis, where basic elements of music are counted and statistics are computed, with determination of 145

frequencies, of “transition probabilities”1 between successive intervals, of “entropy”2 etc. And in fact, interest in these types of quantitative analysis emerged before the rise of informatics: pioneering work in this domain was carried out through manual analyses.

In the middle of the twentieth century information theory was considered a scientific discipline that could profoundly revolutionise music analysis, music theory and music aesthetics. Richard Pinkerton wrote already in 1956: Information Theory may well prove generally useful for studying the creative process of the human mind. I don’t think we have to worry that such analysis will make our art more stilted and mechanical. Rather, as we begin to understand more about the property of creativeness, our enjoyment of the arts should increase a thousandfold. (Schüler 2005, 32)

One main application of the early statistical approaches was to establish models that discriminated between different music genres or between contrastive styles related to different music composers. The aforementioned work by Ortmann enters this category. Another application concerns the representation of music styles in the form of a network of statistical transitions between successive notes – such as “Markov chains”3 – which can be used to generate new pieces of music following the same style.

Besides counting occurrences and statistics, computational analysis can reveal particular structures in scores and draw connections between structures. Allen Forte’s Set Theory, where music analysis is based on a systematic categorisation of sets of pitch classes, has been largely conditioned by the rise of computational tools:

1

Probability of going from one interval to the next one, based on statistics.

2 The information entropy is a basic quantity in information theory, indicating the average level of “information”, “surprise” or “uncertainty”.

3 A Markov chain is a statistical model that describes a sequence of possible events such that the probability of each event depends only on the previous event.

146

The computer can be programmed to deal with complex structures – such as musical composition – very rapidly. […] A second reason for using the computer derives from the requirements of completeness and precision that form the basis of every computer program. […] The design of an algorithm, the formulation of a decision-structure to solve a problem, the careful checking out of a malfunctioning programme – all these activities provide clarifications and insights which would be difficult, perhaps impossible, to obtain otherwise. (Forte, Allen. 1967a. “Computer-­ implemented Analysis of Musical Structure”. Papers from the West Virginia University Conference on Computer Applications in Music, ed. by Gerald Lefkoff. Morgantown: West Virginia University Library. 29–42.

Excerpt from pp. 33–34, cited in Schüler 2006, 9–10)

Set theory analyses are generally sufficiently formalised to be directly implemented in the form of algorithms. A significant part of computational analysis around the mid-1960s was related to set theory analysis.

Among them are also Milton Babbitt’s analytical approaches to do­ decaphonic and set structures.

Another interesting emerging application is the automated detection, or “retrieval”, of particular moments in the music score where a particular structure – defined by a specific “query” – has been found. For instance, finding the positions of a particular motivic structure or a fugue entry etc.

Metrical analysis Computational harmony analysis emerged in the 1960s and was preceded by early attempts to perform statistical analyses of chord structures and root movements, particularly in Bach chorales (Schüler 2005, 35). Another area of analysis is related to rhythmical and metrical analysis. It is not the intention and ambition of this paper to establish a chronology and state of the art of those different dimensions of computational music analysis.

We propose here to focus on one particular reference software for music analysis: David Temperley’s Melisma Music Analyzer (Temperley and Sleator 1999). The core principles of the approach are highly influenced by Fred Lerdahl and Ray Jackendoff’s Generative Theory of Tonal Music (GTTM): for each type of analysis the methodology is 1 47

governed by a set of rules that compete with each other. On one side, well-formedness rules specify particular structural descriptions; on the other, preference rules enable us to compare different possible analyses and choose the one that is most relevant (Lerdahl and Jackendoff 1983).

For instance, the metrical analysis in Melisma aims to infer the metrical structure from a score and is particularly useful when analysing a MIDI representation where that metrical structure is not indicated. In other words, from a series of notes with only pitch, temporal location in seconds and duration in seconds we would like to reconstruct the pulsation along the different metrical levels (cf. Figure 1).

The preference rules are the following: • • •

Event rule: prefer a structure that aligns beats with event onsets4 L ength rule: prefer a structure that aligns strong beats with onsets of longer events Regularity rule: prefer beats at each level to be as evenly spaced as possible

Fig. 1: Metrical and harmonic analysis of “Oh Susannah” by Melisma (Temperley and Sleator 1999, Figure 1, reprinted courtesy of MIT Press). Each row of dots indicates a level of the metrical structure; each dot indicates a beat on the onset of the note below. Each chord symbol represents a chord span, beginning on the note below and extending to the beginning of the following chord span.

4 An onset simply designates the temporal position of the beginning of a note.

148

Let’s consider the example in Figure 1, supposing again that we listen to the music played in a regular tempo but without knowing what the pulsation and metrical structure are. The event rule specifies that we would prefer to tap the beat at the same time as note onsets instead of tapping between two notes. Thus we would prefer to synchronise the pulsation with as many notes as possible. But at the same time the regularity rule constrains the duration between successive pulsations at a given metrical level to be as regular as possible. Finally, we would also tend to synchronise the locations of higher (slower) metrical levels with onsets of longer notes.

Harmonic analysis Harmonic analysis in Melisma aims to reconstruct the chord sequence underlying a given melody. This process is governed by another set of preference rules:

•

5

Compatibility rule: prefer chord roots5 that result in certain pitch-root relationships. From most preferred to less preferred, the relationships are: 1, 5, 3, 3b, 7b, 5b, 9b and finally ornamental.

In other words, for a given pitch in the score (let’s say C).

- The most obvious root would be pitch 1 (C would correspond to C major or minor chord, for instance).

- Or that pitch could be the fifth degree 5 above the root (C would correspond to F major, for instance).

- Or else the major third degree 3 (C would correspond to Ab major chord).

- Or the minor third degree 3b (C would correspond to A minor chord).

- Etc. - In the “worst” case, that pitch is not related to the root and corresponds to an ornamental.

For instance, the root of chord C-E-G is C.

1 49

•

• •

rnamental dissonance rule: when labelling events as ornamenO tal, prefer events that are - closely followed by another event a half step or whole step away - metrically weak Harmonic variance rule: prefer roots such that roots of nearby chord spans are close together on the line of fifths.

Strong-beat rule: prefer to start chord spans on strong beats.

In the same example in Figure 1, from bar 0 to 3, pitches D and A are considered ornamental because they appear in metrically weak positions and are followed by whole-step intervals. The first note (C) could define C as the root. Note E in bar 1 would preferably be considered the third degree of root C rather than defining a new root E because the movement between C and E does not follow the line of fifths. The following note G in bar 1 could actually define a root G, but as this change of roots would not appear on a strong beat, this is avoided. We could imagine root G starting at bar 2 and going back to C at the second beat of that bar, but because that return to C would appear on a not very strong beat, this could be avoided as well. Conversely, the D in bar 4 can lead to root G, where D would be the fifth degree of that root. And so on. More refined harmonic analyses of more complex music can be found in (Temperley and Sleator 1999).

It is not the aim of this paper to give a complete overview of the Melisma Music Analyzer. Besides, significant progress has been made, both by the author David Temperley and by a community of researchers. For instance, while harmony in the first version of Melisma is represented merely as a succession of roots that are constrained to follow the circle of fifths, more recent approaches integrate functional analysis: each chord is expressed as a scale degree (such as I or V) – with a function (such as tonic or dominant) – within a particular tonality.

For instance, the hierarchical organisation of tonal music has been represented in the form of a context-free generative grammar (Rohrmeier 2011). The grammar incorporates four levels: a phrase level divides a piece into phrase; a functional level specifies the functional role a certain scale degree has within a phrase, the scale degree cap150

tures the relationship between the chord and the key and the surface level expresses the actual chord with all its possible additions, inversions, etc.

Fig. 2. Automatic analysis, using HarmTrace, of the first two phrases of J.S. Bach’s Ach Herre Gott, mich treibt die Not, BWV 349, in the key of F.

Figure 2 shows an example of analysis obtained through an implementation of this grammar called HarmTrace (de Haas et al. 2011). The analysis is displayed in the form of an upside-down tree, made up of a series of top-down branches, each one representing one of the successive chords in the piece. For instance, the first branch on the left represents the first chord, an F major chord (“F”), first degree (“I”), hence tonic (“Ton”). We can see that the fifth chord is exactly the same. The previous (fourth) chord is the dominant (“Dom”), hence fifth degree (“V”) C major chord (“C”). Similarly, the previous (third) chord is the subdominant (“Sub”), hence second degree (“IIm”) G minor seventh (“Gm7”). Again, the previous (second) chord is the fifth degree above, represented as “Vd/II”, hence sixth degree (“VIm”) D minor chord (“Dm”). At each level in the tree, branching indicates descending fifth transitions, in particular from subdominant (“Sub”) to dominant (“Dom”).

Computational models did not focus solely on Western music.

For instance, with my colleague Mondher Ayari I tried to model the modal development of traditional Arabic maqam music and in particular the Tunisian repertoire (Lartillot and Ayari 2011). Western 1 51

Fig. 3: Modal structure of Mhayyer Sîkâ D, a Tunisian maqām mode (Lartillot and Ayari 2011). The ajnas constituting the scales are: Mhayyer Sîkâ D (main jins), Kurdi A, Bûsalik G, Mazmoum F, Isba’în A, Râst Dhîl G, and Isba’în G. Pivotal notes are circled.

Fig. 4: Computational modal analysis of a transcription of the beginning of a traditional instrumental improvisation performed by the Tunisian Nay master Mohamed Saâda, developing the fundamental elements of the Mhayyer Sîkâ D maqām mode (Lartillot and Ayari 2011).

152

classical harmony is defined by the use of particular tonal scales, where for each scale the discourse is composed of a succession of chords, made up mostly of triads based on notes taken from that scale.

Maqam music is defined by the use of modal scales, and the discourse is structured by the use of particular subscales (also called jins, plural ajnas) made up of successive notes from those scales, generally four (tetrachord) or five (pentachord), where some of the notes on those subscales are pivotal, serving as stable notes. Figure 3 shows an example of modal structure.

We have attempted to automatically perform this type of modal analysis. Figure 4 shows an example of analysis. The succession of most likely ajnas is indicated below the staves. Important notes (as opposed to ornaments) are circled, and pivotal notes are highlighted with grey ovals that encompass the whole underlying ornamentation.

Structural analysis David Temperley’s Melisma Music Analyzer was greatly influenced by Lerdahl and Jackendoff’s GTTM, which integrates various dimensions of analysis, including metrical analysis. Another core analytical principle studied in the GTTM is reduction, which is further modelled in two ways: through the inference of “prolongation” and “timespan trees”. Attempts at automation of these reduction principles have been proposed (Hamanaka et al. 2006; Marsden 2010).

Another dimension of music analysis considered in the GTTM is the grouping of notes in multiple hierarchical levels. The idea is to show the decomposition of the musical discourse into successions of phrases, themselves decomposed into sub-phrases, and so on. This is another complex aspect of music analysis that is hard to model. In our collaboration with Mondher Ayari we have tried to develop a computational modelling of music segmentation6 and have shown how it is

6 “Grouping” and “segmentation” are basically the two sides of the same coin, one looking at how notes are grouped together, while the other considers how the melody is cut into pieces.

153

governed by a large range of heuristics based on local configurations (note durations, pitch interval) but also higher-level aspects such as harmony and rhythm (Lartillot and Ayari 2011).

Computational attempts have been made at analysing monodies and modelling low-level aspects of grouping related simply to local aspects such as note duration and pitch intervals. Emilios Cambouropoulos’ Local Boundary Detection Model (LBDM), for instance, assigns a “boundary strength” (which can be understood as a segmentation probability) to each interval between successive notes based on the pitch and temporal intervals of previous and subsequent notes (Cambouropoulos 2006). Figure 5 shows an example of a segmentation curve, where peaks indicate location where segmentation is predicted.

We have introduced a new approach that enables us to reveal a multi-­level hierarchical construction of local grouping. Figure 6 shows this hierarchical local grouping for the beginning of Mozart’s Variation XI on “Ah, vous dirai-je maman”.

Fig. 5. The first few bars of Chopin’s Etude Op10, No3.

The curve depicts the LBDM boundary strength profile.

Peaks in this curve indicate potential local boundaries (Cambouropoulos 2006).

Fig. 6. Analysis of the melody at the beginning of Mozart’s Variation XI on “Ah, vous dirai-je maman”, K.265/300e. Each box indicates a local segment based on note durations.

Box colour is related to segmentation level and defined by the longest duration between notes in the box.

Grey is related to dotted quavers, yellow to quavers, orange to dotted semiquavers, red to semiquavers, and so on.

154

We have also studied the applicability of these approaches to non-­ Western music, in particular traditional Turkish makam (Lartillot et al.

2013). Figure 7 shows an example of an analysis of a Nihavend maqam in Kar form according to three segmentation models: one presented in (Tenney and Polansky 1980), the LBDM and our own model.

Fig. 7. Analysis of a Nihavend maqam in Kar form (Lartillot et al. 2013). Comparison of the boundaries (i.e., where the melody can be cut into segments) given by the three computational models – Tenney and Polansky’s (TP) model, LBDM and our proposed model (“New Model”) – with segmentations given by listeners, shown above the stave.

Another core aspect of music is motivic repetition, i.e. the multiple repetition of musical sequences of various scales, from short motivic cells to long thematic phrases. This is taken into account in the GTTM under the term “parallelism”. I have shown how parallelism actually 155

plays a central role in Lerdahl and Jackendoff’s theory (Lartillot 2010).

Yet the GTTM’s authors recognise that they did not manage to formalise this core aspect in their model. “Our failure to flesh out the notion of parallelism is a serious gap in our attempt to formulate a fully explicit theory of musical understanding” (Lerdahl and Jackendoff 1983, 53).

Computational automation of motivic analysis has been attempted during the past few decades. I have personally spent a great deal of effort in this area of research. In my approach the idea is to reveal the motivic structure with a maximum of detail while at the same time keeping the analysis as simple and clear as possible (Lartillot 2016). Due to the complexity of the problem, my approach is restricted so far to the analysis of monodies, or superpositions of monodies such as counterpoints. Although various music dimensions, such as rhythm, diatonic and chromatic pitch, are also taken into consideration.

Fig. 8. Automated analysis of J.S. Bach’s Invention in D minor BWV 775, first 14 bars (Lartillot 2007, 306).

The motives for each voice are indicated below the corresponding staves with horizontal lines. Groups of motives are denoted with a letter (larger motives A, B, etc.; smaller cells a, b, etc.) indicated on the left side of each line.

156

Figure 8 shows an example of computational analysis, using my approach, of the first 14 bars of J.S. Bach’s Invention in D minor BWV 775 (Lartillot 2007, 306). The opening motive A, with all its occurrences throughout the piece, has been accurately retrieved by the machine. The accompanying figure B has been detected too and is decomposed into a succession of two successive and similar shapes.

The first sequence unit A’ is detected and identified as a variation of the opening motive A. The computational system has been able to retrieve the most salient structures of the piece, which are congruent with a musicologist’s analysis (Kresky 1977). However, the subtlest configurations discovered by the musicologist cannot be detected with the algorithm. Indeed, the use of computers here is not intended to replace the musicologist’s skills, but rather at to experiment with a formalisation of the basic principles of music understanding. Besides, this automation may enable the annotation of large music databases.

Automated analysis of fugues has also been attempted, detecting subjects and countersubjects as well as partial harmonic sequences inside episodes (Giraud et al. 2015).

Applications As the GTTM has shown, and particularly when considering the formalisation shown in Figure 9, the various dimensions of music analysis (metrical, tonal/harmonic, motivic, structural, etc.) are highly interdependent. Melisma Music Analyzer, for instance, does integrate various components of music analysis, but it does not model their interdependencies. I am conceiving a computational framework, called the MiningSuite, that integrates a large range of analyses, both from audio and from scores (Lartillot 2011). One main motivation of this project is precisely to model and take into consideration these interdependencies.

The MiningSuite integrates other previous projects, all developed in the Matlab environment: in particular the MIRtoolbox (Lartillot and Toiviainen 2007), a leading tool for music analysis from audio, and MIDItoolbox (Eerola and Toiviainen 2004), for the analysis of MIDI representations.

1 57

Fig. 9. Schematic understanding proposed in (Lartillot 2010, 196) of the general conceptual framework developed in Lerdahl and Jackendoff’s GTTM. In italics are notions related to accents, in bold aspects related to parallelism and in grey the part of the theory related to grouping structure.

Other software solutions have been developed and used in the research community, such as the Humdrum toolkit for pattern matching, for search for similarity in music and for statistical measurements (Huron 2002). Other solutions include Music21, in the Python language, for complex query retrievals, statistics and analytical models (Cuthbert and Ariza 2010) and JSymbolic (McKay and Fujinaga 2006), also to extract statistical information.

Partial or complete automation of music analysis opens a broad range of opportunities in terms of application to every-day music activities by the general public. Music analysis could help listeners better understand music by offering visual guiding tools, which could be designed to be as immersive and engaging as possible. This technology can also help design new ways of automatically structuring music catalogues to get a clear idea of their content and of automatically classifying the catalogue in terms of genres, composition styles, cultural or geographic origins, moods etc. Another application concerns the detection of pieces of music that are variants of the same composition or that share the very same musical characteristics and can help detect cover songs and plagiarism.

158

This quest towards the development of artificial musical intelligence can also unlock the development of a form of computational creativity. By understanding how music works, machines can then learn to automatically compose new music as well as learn to perform music, e.g. interpret classical music with rubato and expression.

Acknowledgements This work is partially supported by the Research Council of Norway through its Centres of Excellence scheme, project number 262762, and the MIRAGE project, grant number 287152.

1 59

Literature

Good, Michael. 2001. “MusicXML for notation and analysis”. In The Virtual Score: Representation, Retrieval, Restoration, eds. Walter B. Hewlett and Eleanor Selfridge-­Field, 113–124. Cambridge, Massachusetts: MIT Press.

Hankinson, Andrew, Perry Roland, and Ichiro Fujinaga. 2011. “The Music Encoding Initiative as a document encoding framework.” Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), eds.

Anssi Klapuri and Colby Leider, 293–298. Miami: University of Miami.

Huron, David. 2002. “Music information processing using the Humdrum Toolkit: Concepts, examples, and lessons.” Computer Music Journal 26(2): 11–26.

Schüler, Nico. 2005. “Reflections on the history of computer-assisted music analysis I: Predecessors and the beginnings.” Musicological Annual 41(1), 31–43.

Schüler, Nico. 2006. “Reflections on the history of computer-assisted music analysis II: The 1960s.” Musicological Annual 42(1), 5–24.

Temperley, David, and Daniel Sleator. 1999. “Modeling meter and harmony: a preference-rule approach.” Computer Music Journal 23(1), 10–27.

Lerdahl, Fred, and Ray Jackendoff. 1983. A generative theory of tonal music.

Cambridge, Massachusetts: MIT Press.

Rohrmeier, Martin. 2011. “Towards a generative syntax of tonal harmony.”

Journal of Mathematics and Music 5(1), 35–53. de Haas, W. Bas, José Pedro Magalhães, Frans Wiering, and Remco C. Veltkamp. 2011. HarmTrace: Automatic functional harmonic analysis. Technical Report UU-CS-2011-023. Utrecht: Utrecht University.

Lartillot, Olivier, and Mondher Ayari. 2011. “Cultural impact in listeners’ structural understanding of a Tunisian traditional modal improvisation, studied with the help of computational models.” Journal of Interdisciplinary Music Studies 5(1), 85–100.

Hamanaka, Masatoshi, Keiji Hirata, and Satoshi Tojo. 2006. “Implementing ‘A Generative Theory of Tonal Music.’” Journal of New Music Research 35(4), 249–277.

160

Literature Litteratur

Marsden, Alan. 2010. “Schenkerian analysis by computer: A proof of concept.”

Journal of New Music Research 39(3), 269–289.

Cambouropoulos, Emilios, 2006. “Musical parallelism and melodic segmentation: A computational approach.” Music Perception 23(3), 249–268.

Lartillot, Olivier, Z. Funda Yazıcı, and Esra Mungan. 2013. “A more informative segmentation model, empirically compared with state of the art on traditional Turkish music.” Proceedings of the 3rd International Workshop on Folk Music Analysis (FMA), 63–70.

Tenney, James, and Larry Polansky. 1980. “Temporal Gestalt perception in music.” Journal of Music Theory 24(2), 205–241.

Lartillot, Olivier. 2010. Reflections towards a generative theory of musical parallelism. Musicae Scientiae Discussion Forum 5, 195–229.

Lartillot, Olivier. 2016. “Automated motivic analysis: An exhaustive approach based on closed and cyclic pattern mining in multidimensional parametric spaces.” In Computational Music Analysis, ed. David Meredith, 273–302. Springer.

Lartillot, Olivier. 2007. Motivic matching strategies for automated pattern extraction. Musicae Scientiae Discussion Forum 4A, 281–314.

Kresky, Jeffrey. 1977. Tonal music: Twelve analytic studies. Philadelphia: Indiana University Press.

Giraud, Mathieu, Richard Groult, Emmanuel Leguy, Florence Levé.

“Computational fugue analysis.” Computer Music Journal 39(2), 77–96.

Lartillot, Olivier. 2011. “A comprehensive and modular framework for audio content extraction, aimed at research, pedagogy and digital library management.”

Proceedings of the 130th Convention of the Audio Engineering Society.

Lartillot, Olivier, and Petri Toiviainen. 2007. A Matlab toolbox for musical feature extraction from audio. Proceedings of the International Conference on Digital Audio Effects, 237–244.

Eerola, Tuomas, and Petri Toiviainen. 2004. “MIR in Matlab: The MIDI Toolbox.” Proceedings of the International Society for Music Information Retrieval (ISMIR), 22–27.

Cuthbert, Michael Scott, and Christopher Ariza. 2010. “music21: A toolkit for computer-aided musicology and symbolic music data.” Proceedings of the International Society for Music Information Retrieval (ISMIR), 637–642.

McKay, Cory, and Ichiro Fujinaga. 2006. “jSymbolic: A feature extractor for MIDI files.” Proceedings of the International Society for Music Information Retrieval (ISMIR), 302–305.

1 61

Contributors

Jennifer Ward studied musicology and German at the University of Wisconsin and library science at the University of Illinois. She worked as a music librarian at Northwestern University before moving to Germany. She has been an editor at the Central Office of RISM in Frankfurt since 2010. Jennifer is active in IAML as the web editor and secretary of the Bibliography Section. Her research has appeared in journals including Notes, Journal of Musicological Research and Die Tonkunst. She is currently working on a dissertation about a collection of 17th-century printed music kept at the University Library in Frankfurt.

century, after which he taught music history and music theory in Copenhagen. In 2013–2018 he worked as a researcher at the Danish Centre for Music Editing at the Royal Danish Library, where he published critical editions of choral music by the Danish nineteenth century composer Niels W. Gade. In 2017–2021 he is participating in the project “Music and Language in Danish Reformation Hymns” at the Society for Danish Language and Literature.

The project is concerned with publishing digital editions of sixteenth century hymn books and missals at <salmer.dsl.dk> and <melodier.dsl. dk>.

Axel Teich Geertinger, former Head of the Danish Centre for Music Editing, now Research Librarian at the Royal Danish Library. His main fields of interest in recent years have been critical editing, digital musicology, thematic catalogues and Danish 19th-century music. He studied engineering and musicology and earned his PhD in musicology from the University of Copenhagen in 2009 with a dissertation on the Italian baroque opera sinfonia. He is the main developer of the MEI-based metadata editor MerMEId. He is also employed as Senior Editor at the Society for Danish Language and Literature, working on a digital edition of Danish 16th-century hymn books.

Julia Craig-McFeely is DIAMM research fellow at the University of Oxford. She studied at Edinburgh University and completed her DPhil on English Lute Manuscripts and Scribes 1530–1630 at the University of Oxford in 1994.

She has been involved with DIAMM since its inception in 1998 and a director since 2007. She is known internationally as an expert in archive-quality imaging of delicate documents, has lectured on intellectual property rights in relation to digital images and has consulted to a number of organisations including the National Library of Ireland and the Israel Antiquities Authority. In 2008 she was one of the team of specialists who undertook the pilot project to digitise the Dead Sea Scrolls in Jerusalem. She now divides her time between DIAMM, DIAMM Publications, research in early modern British music and as an editor and publisher of books on musicology and theology.

Bjarke Moe studied musicology at the University of Copenhagen and later in Greifswald (Germany). In 2010 he received his PhD degree in Copenhagen with a dissertation on music and cultural exchange in the seventeenth

162

Contributors

Jøran Rudi’s first academic training was in the social sciences, then followed a few years as a rock musician in one of the influential bands that emerged at the end of the 1970s.

This brought him into contact with electronic instruments and electroacoustic music, and he travelled to the United States for studies in computer music at New York University. In 1990 he returned to Norway and was brought in to be the founding director of NOTAM in 1993.

From 1993 to 2010 he was responsible for the academic and artistic profiles of the institution, its research and development, mediation, education, administration and finance. Jøran Rudi returned to his role as a researcher at NOTAM in 2010.

Olivier Lartillot is a researcher at the RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, University of Oslo, working in the fields of computational music and sound analysis and artificial intelligence. He is currently leading a research project called MIRAGE – A Comprehensive AI-Based System for Advanced Music Analysis, funded by the Research Council of Norway until late 2023. He has collaborated on various projects around the topics of artificial intelligence, signal processing, cognitive science, neuroscience, music analysis, ethnomusicology and music therapy.

manuscripts, digital solutions for notated music, and source publishing of music-related manuscripts. Bue is also involved in the Music Encoding Initiative (MEI), Répértoire International des Sources Musicales (RISM) and Répértoire International des Littérature Musicales (RILM).

Annika Rockenberger studied modern German literature, European history and communication science at Freie Universität Berlin, where she specialised in textual scholarship and book and print history of the early modern era. She earned her PhD in literary studies and philosophy from the University of Oslo. She works as senior academic librarian for digital research methods in the humanities and social sciences at the University of Oslo Library. She is co-founder and vice-chair of Digital Humanities in the Nordic Countries (DHN).

Margrethe Støkken Bue studied musicology at the University of Oslo, specialising in sheet music editing, music philology and Norwegian music history. She is employed by the National Library of Norway, where she works with music

163

Nota bene series Litteratur

Nota bene is the National Library of Norway’s channel for disseminating research findings built on its collections, and research of relevance to these collections. All manuscripts are peer reviewed. Nota bene has a wide thematic profile. In order to mirror the full breadth of our collection, the publications, which include monographs, critical editions and collections of articles, may be based on manuscripts, printed material, film, photography, music, broadcasting, and digital media.

Nota bene 1 Det nasjonale i Nasjonalbiblioteket | Marianne Takle | 2009 Nota bene 2 The Archive in Motion. New Conceptions of the Archive in Contemporary Thought and New Media Practices | Eivind Røssaak (red.) | 2009 Nota bene 3 Axel Charlot Drolsum. Brev 1875–1926 | Bjørg Dale Spørck | 2011 Nota bene 4 Opplysning, vitenskap og nasjon. Bidrag til norsk bibliotekhistorie | Ruth Hemstad (red.) | 2011 Nota bene 5 Latin Manuscripts of Medieval Norway. Studies in Memory of Lilli Gjerløw | Espen Karlsen (red.) | 2013 Nota bene 6 Den engasjerte kosmopolitt. Nye Bjørnson-studier | Liv Bliksrud, Giuliano D’Amico, Marius Wulfsberg og Arnfinn Åslund (red.) | 2013 Nota bene 7 Naturen og eventyret. Dokumentarfilmskaperen Per Høst | Gunnar Iversen | 2014 Nota bene 8 Å bli en stemme. Nye studier i Camilla Colletts forfatterskap | Trond Haugen (red.) | 2014 Nota bene 9 Propagandakrig. Kampen om Norge i Norden og Europa 1812–1814 | Ruth Hemstad | 2014 Nota bene 10 Small Country, Long Journeys. Norwegian Expedition Films | Eirik Frisvold Hanssen og Maria Fosheim Lund (eds.) | 2017

164

Nota bene series

Nota bene 11 Reformasjonstidens religiøse bokkultur cirka 1400–1700: tekst, visualitet og materialitet | Bente Lavold og John Ødemark (red.) | 2017 Nota bene 12 I dørtrekken fra Europa. Festskrift til Knut Sprauten. I anledning 70-årsdagen 22. juni 2018 | Ola Alsvik, Hans P. Hosar og Marianne Wiig (red.) | 2018 Nota bene 13 Litterære verdensborgere. Transnasjonale perspektiver på norsk bokhistorie 1519–1850 | Aasta M.B. Bjørkøy | Ruth Hemstad | Aina Nøding og Anne Birgitte Rønning (red.) | 2019 Nota bene 14 Lov og lovgivning i middelalderen. Nye studier av Magnus Lagabøtes landslov Anna Catharina Horn og Karen Arup Seip (red.) | 2020 Nota bene 15 Notated Music in the Digital Sphere: Possibilities and Limitations Margrethe Støkken Bue and Annika Rockenberger (eds.) | 2021

165

© National Library of Norway, Oslo 2021 ISBN 978-82-7965-463-6 (trykt) ISBN 978-82-7965-464-3 (e-bok) ISSN 1891-4829 (trykt) ISSN 2535-4337 (e-bok) Design: Superultraplus Designstudio AS, www.superultraplus.com Print: Erik Tanche Nilssen AS This material is protected by copyright law.

Without explicit authorisation, reproduction is only allowed in so far as it is permitted by law or by agreement with a collecting society.

Notated Music in the Digital Sphere: Possibilities and Limitations explores the different research areas of notated music in a digital setting. In this collection of articles, European scholars investigate varied issues such as databases, encoding, computational analysis, long-term storing of born-digital material, and digital restoring.

NOTA B E N E

Nota bene is the National Library of Norway’s channel for disseminating research findings built on its collections, and research of relevance to these collections. All manuscripts are peer reviewed.

Nota bene has a wide thematic profile. In order to mirror the full breadth of our collection, the publications, which include monographs, critical editions and collections of articles, may be based on manuscripts, printed material, film, photography, music, broad­ casting, and digital media.

tekst, visualitet og materialitet

Bente Lavold og John Ødemark (red.)

11 I S B N : 9 7 8 - 8 2 -7 9 6 5 - 4 6 3 - 6

9

788279

654636

NASJONALBIBLIOTEKETS SKRIFTSERIE

Reformasjonstidens religiøse bokkultur cirka 1400–1700: