This study is founded upon the model of subjective tone duration and represents an analysis referred to musical application of the psychoacoustical phenomenon of temporal masking effects. Within an empirical experiment ten musicians performed phrases of the F-major-Invention by J.S. Bach and were recorded by the means of a MIDI-sequencer. A variation of the used synthetic piano-sounds and organ-sounds was executed, relating to the parameters articulation, envelope and dynamics. Sounds with subjectively longer auditory impression of an objectively measurable shorter sound sensation were contrasted with objectively longer sounding tones. The realized statistic evaluation confirmed a significant influence of the specific envelope for the performed notelength, especially for the articulation portato.

Diese Studie basiert auf dem Modell der subjektiven Tondauer und stellt eine auf die musikalische Anwendung bezogene Analyse dieses psychoakustischen Phänomens dar. In einem mit zehn Probanden vorgenommenen praktischen Versuch wurden Phrasen der F-Dur-Invention von J.S. Bach mit einem softwaregestützten MIDI-Sequenzer aufgezeichnet. Das verwendete synthetisierte Klavier- und Orgelklangmaterial unterlag dabei einer Variation der Parameter: musikalische Artikulation, Hüllkurve sowie Anschlagdynamik. Klänge mit subjektiv längerem Höreindruck eines objektiv meßbar kürzeren Schallereignisses wurden im Experiment objektiv länger klingenden Tonimpulsen gegenübergestellt. Die statistische Auswertung zeigte besonders für die Artikulation portato einen deutlichen Einfluß der Hüllkurve eines Ausgangsklanges auf die tatsächlich gespielte Notenlänge.

Cette étude - fondé sur le modèle de la durée subjective du son - traite de l’application musicale du phénomène psychoacoustique des effets temporels dans le masquage. Dans une étude empirique dix musiciens ont eu pour tâche de jouer des phrases de l’ Invention en fa-majeur de J.S. Bach, enregistrés par un MIDI-sequencer. Les sons synthétiques du piano et de l’orgue, qu’on a pris pour ce test, ont été variés au point de vue de l’articulation musicale, de l’envelope et de la dynamque. Des notes qui donne l’impression d’être plus longues qu’ils sont en realité ont été comparées avec des notes effectivement plus longues. L’exploitation statistique confirme que l’envelope a une influence significative sur la longueur de la note, surtout dans le cas de l’articulation portato.

cognitive musicology, envelope-variation, masking, MIDI, MIDI-sequencer, musical interpretation, psychoacoustics, psychomusicology, sound analysis, subjective tone duration, temporal masking effects.

The subject of masking is of considerable importance to music. There are some well-known papers (Wegel and Lane 1924, Egan and Hake 1950, Feldkeller and Zwicker 1967) and experiments refering to the masking effect and other psychoacoustical phenomena in this century. New aspects have been explored by Zwicker (1970) and after all by Fastl (1974, 1975, 1979, 1981, 1982). The past research primarily explored the basic knowledge of masking in an universally valid way. Based on scientific findings about temporal masking effects, this study especially inspects the human response and attitude in a music-orientated application at the appearance of masking patterns. As a result of this paper performance rules were elaborated to make the findings available for MIDI-software-developing and other interested persons or institutions as a point of attachement.

Every sonic phenomenon requires some duration. The subjective toneduration Ts is characterized as the specific auditory sensation, for the duration of sound with given physical data Ti (Fastl 1975). Fig. 1 shoes an idealized model for a masking threshold. The dotted line represents the threshold of audibility, the subjective toneduration Ts is defined (in this connection) 10 dB above the absolute threshold, which is the minimum stimulus that evokes a response in a specified fraction of the test. It is noticeable in Fig. 1, that the masking effect is extended over the objective toneduration, that means it is not limited to the physical presence-time of the test-tone impulse.

In this paper no additional experiments connected to basic masking research are explained. The main interest of this study is the practical application of the actual psychoacoustical knowledge by performing musicians. Evident researches on subjective auditory sensation should be applied instinctively by instrumentalists; artists should play notes of existing scores automatically in the subjectively correct length; to find out the need for an implementation of this psychoacoustical knowledge in existing MIDI-sequencers was the secondary interest of the experiment.

The purpose of this study was to draw a conclusion how these interactions between objective physical duration-parameters and the specific interpretation are managed by performing musicians. Therefore the following experiment was executed:

By the aid of a digital sampler (Roland S 760, 44.1 kHz, 10 MB Sample-RAM) the envelopes of synthetic pianosounds and organsounds were varied. This was done by the use of an envelope follower (time variable amplifier, TVA). The same sound was supplied with three different dynamic envelopes (Fig. 2): dynamic envelope 1 represents a piano-sound or organ-sound (Ti = 121ms) reaching the maximum intensity (0 ms attack-time) rapidly. After the MIDI-note-off there will be no decay, the sound will fall back to silence without any delay (0 ms release-time). For the second dynamic envelope the attack-times and release-times were selected shorter than the masked threshold (Fastl 1975: 289). In this experiment Ti for envelope 2 was fixed between 160 ms and 170 ms, depending on sound and frequency. The attack-time was less than 15 ms, the release-time therefore about 30 ms. In case of the dynamic envelope 3 the objective toneduration was selected longer than the masking threshold of the original piano-impulse or organ-impulse used with envelope 1, that means, the objective and the subjective auditory sensation is approximately equal (Ti = Tm’).

"The attack is normally the most tonally complex part of the envelope, and provides most of the auditory attributes by which one instrument is distinguished from another. Although only a small fraction of a second of the sound will have been removed, this is often sufficient to render the resultant sound unrecognizable." (Dobson 1992, 68-69) There are many reasons to suppose, that musicians will adapt their way of playing by the different appearance of dynamic envelopes. Envelope type 1 and type 2 hypothetically should be treated in the same way, while envelope 3 should cause a shorter performance of the note length.

The experiment (Fig.3) was executed with 10 pianists, who met all requirements for the musical qualification. Each of them had to play the first phrase (4 measures of the soprano-voice) of the F-major-Invention by J.S. Bach for several times in the articulation legato, portato and staccato. The four measure phrase was played on a masterkeyboard (Roland A 80) and recorded on a MIDI-sequencer (Cubase Score 1.0). Both the piano-sound and organ-sound were performed with an enabled MIDI-velocity-sensitivity, within one execution of each articulation type and envelope type, the MIDI-velocity of the samplers organsound was disabled. This was in order to find out if there is an influence in the instruments touch-sensitivity on the musical performance. The tempo of the performance was 120 quaternotes per minute (1/8-note = 250 ms, 1/16-note = 125 ms) for all articulation-types. So it was faster than it would be performed in a usual interpretation. This was necessary to fullfill the limit for masking thresholds (Zwicker 1970, Fastl 1982).

To execute all possibilities, the four-measure-phrase had to be played 27 times; to avoid arbitrary results, a total of 54 repetitions had to be performed. The overall duration of the experiment was about 30 minutes. The succession of envelope-variation was a matter of chance. The instrument-sounds (organ or piano) and the chronological order of the articulation were not mixed to guarantee a constant musical feeling during the testsession and to avoid a distraction of the musicians. All persons were instructed to retain the articulation as requested, irrespective of possible nonsense for the interpretation or their musical realization. Before starting the experiment, an unrestricted practice-time for each person was included, to give all musicians the possibility to feel comfortable on the keyboard, and to get an auditory impression of the forthcoming soundmaterial.

To keep a maximum of flexibility for the evaluation and interpretation, every single MIDI-recording was transformed seperately to an ASCII-file (per person 54 files, each with 35 MIDI-notes containing 35 velocity- and notelength-values). This procedure was done by a special PC-program written in C. The statistical computer-analysis was made with the program SPSS 6.0.1^® for Windows^®.

Figure 4 gives an example of a single performed phrase-profile, including the parameters notelength and MIDI-velocity of one person. This style of representation allows an exact and individual interpretation of the musical phrase shown below. Each note is determinable in notelength and velocity at first sight, characteristic attributes of the phrase are well readable. The last note (N^o. 35) was excluded from the evaluation. The sequencer value for the MIDI-notelength (variable "Notenlänge") is represented by the ppq-factor (ppq = pulses per quaternote). In this sequencer application one quaternote is subdivided in 384 ppq, therefore 1ms is approximately equivalent to 1.3 ppq at a tempo of 120 quaternotes per minute. The degree of accuracy in this experiment was better than 0.7 ms.

The following schedule gives an impression of the evaluation for ten people: this summary of the 27 possibilities is classified in two categories: the arithmetical mean of quavers and semiquavers of the whole phrase. According to expectation, the notelength for the envelopetypes 1 and 2 indicates a significant correlation for all articulations and sounds. The comparison of the notelengths for envelope 3 shows interesting results: the arithmetical mean displays no or miniscule deviation, in most cases less than 3 ms in the notelength (for all instrument-sounds played under the requested articulation legato). Concerning the articulation staccato nearly the same result appeares, excepting irregularities. Finally, the prognosticated variation for the articulation portato appears: playing with envelope 3 the notelength was shortened up to 11ms.

Abbr.: env. = envelope, Xam = arithmetical mean, S = standard deviation, Xme = median.

A more detailed evaluation of the whole sample as shown above made some global trends evident (Junker 1995). One step further a clusterevaluation was made to analyse optional similarities of single musicians.

The clusterevaluation was used to establish groups of persons with regular attributes. Due to the most evident deviations the primary emphasis was put on a significant playingstyle and a typical choice of notelength for a specific envelope-variation of the articulation portato. Other attention of the analysis was directed to the possible sensitivity of people for different instrument sounds. Irregular data of staccato-phrases was excluded from the analysis. As a result of the primary constant regularity for the legato-notelength, this articulation type had a minor influence on the final results.

Two categories with similar attributes could be isolated, one person was excluded from the interpretation because of irregularities:

Five musicians form cluster 1 (this group of people can be characterized as follows): the variation of the envelope has no significant influence on the deviation of the performed notelength (Fig. 5). The comparison made for all three envelopetypes results in an average deviation less than 3 ms. Equally, no sensitivity could be detected in this category for specific instrument-sounds.

The description of Cluster 2 (4 persons) is much more interesting: as with cluster 1, there is no significant influence in this group for the envelope-variation playing the phrases legato and staccato; under the premise to articulate portato there appear especially remarkable differences in the notelength. The correlation between phrases performed with envelope 1 and 2 is significant, for the envelope type 3 the performed notelength is shortened almost to 40 ms (Fig. 5), in single cases the shortening was even more. It is noticeable for cluster 2, that the major deviations are at note N^o.17 to N^o. 21. Especially remarkable is the tendency after note N^o. 3: detectable is a more or less parallel proceeding of the graphes "LANG1" and "LANG2". Curve "LANG3" shows a significant deviation, obviously from this point the persons of cluster 2 adapted their performance behaviour because of the modified envelope. From note N^o. 4, the notelength-values for "LANG3" are almost always shorter.

Opposed to the more insensitive musicians of cluster 1 (Fig. 6), the persons of Cluster 2 show a sensibility for specific instrument-sounds, especially at the articulations portato and staccato. Phrases played with the piano-sound resulted in an eminently shorter notelength. The maximum notelength-reduction is about 35 ms at note N^o. 18. For the semiquavers there is no significant regularity of deviation detectable. The fascinating thing is that the deviation seems not to be forced by any specific envelope. There is no adaption after some played notes, the reduction of the notelength for cluster 1 starts with the first performed note. Obviously, people in this group have a different affinity or relationship to the alternately used instrument sounds in this experiment. It is an interesting fact that this characteristic feature did not appear for the articulation legato.

The next graph (Fig. 7) refers to the question: "If there is an influence of different articulation styles on the performed MIDI-velocity?" A preponderate regularity is indicated, the progress of the curve diagram is parallel; just for the articulation staccato appears an irregularity at note N^o. 9. On an average the articulation "portato" has the highest dynamic coefficent, tendentious the velocity-value for legato is the lowest. The analysis, referring to the influence of the envelopetype, did not result in significant facts. Figure 8 gives a demonstration for the remarkable correlation, all envelope adjustments result in almost congruent developments of the curves. In the same way, there couldn’t be established any typical influence for the use of the non velocity sensitive organsound. Additional researches resulted in no significant indication of a specific sensitivity for using touch senitive or not touch sensitive sounds.

Proceeding on the hypothesis that the parameter notelength and velocity are inversly proportional this study did not result in gratifying confirmation. No significant or fundamental characterization which indicates a shorter notelength for a high velocity-value (and vice versa) could be remarked, neither for a specific articulation, nor for any instrument sound or envelope used in this experiment.

Figure 9 shows an example of the confrontation of notelength and velocity. Both variables (var.: LANG and VELO) are indicated by the arithmetical mean (fat line), the standard deviation (box) and maximum/minimum (thin line, cumulation 95%). Any proportion can be inspected and interpretated note by note. For example the extended notelength ranges for note N^o. 10, N^o. 13 and N^o. 22, or the relative constant value for the MIDI-velocity of each single note.

The results of this study are:

• Persons, who attended the experiment, can be seperated into two groups with regular attributes: one group of more or less insensitive musicians with no significant deviations on any variation of the experiment; a second group of instrumentalists with interpretable reactions to specific parameter settings.

• The sensitive group is characterized by the following facts: the performed notelength depends on the instruments sound; for the articulations legato and staccato the way of playing is mechanically dominated, the variation of the sound’s envelope is not of importance; phrases articulated portato showed significant deviations in notelength reductions for sounds of envelope type 3.

• The musicians key depression is not changed, neither at a variation of the envelope, nor at the appearence of different instrument sounds.

The experiment, using an softwarebased MIDI-sequenzer, turned out to be a reliable method. To improve the psychoacoustical performance of software based MIDI-sequenzers the aspects of the following model could be applied (Fig. 10).

Figure 10 shows a simplified description of a MIDI-sequenzer listeditor, including six crotchets. Different dynamic values are indicated by shading (pp to ff). The graph below the listeditor displays in a simplified way the expected masking threshold of each single note depending on the velocity and the envelope type. MIDI-notes over the whole length represent sounds performed with envelope type 1 and 2. MIDI-notes without the brigth shaded elongation stand for sounds with a long attack-time and release-time. It is quite natural that in this case the objective toneduration for the synthetic instrument is longer than the duration of the MIDI-note, and logically longer than the intellectual model of masking threshold for the MIDI-notes.

This paper is a abridged version of a more detailed study (Junker 1995). The description in rough outlines represents not a final result. To advance a model for practical realization more extensive series of experiments would be necessary. This article introduces just a method for finding a future way to design a better (more humanized) generation of computers for the musical creativeness.

Ackermann, Philipp

Backus, John

Bengtsson, Ingmar and Gabrielsson, Alf

Burghardt, H.

Butler, David

Cogan, Robert

Desain, Peter and Honing, Henkjan

Dobson, Richard

Egan and Hake

Fastl, Hugo

Feldkeller and Zwicker

Fletcher, Neville H.

Junker, Gerhard

Moore, Brian C.J.

Palmer, C. and Brown, J.C.

Repp, Bruno H.

Roederer, Juan G.

Sundberg, Johan

Sundberg, Johan

Wegel and Lane

Zwicker, Eberhard