Articles related to the study

PNAS paper + photos

FAQ

Response to criticisms of Preferences among old and new violins

A number of criticisms have been leveled (in one form or another) at this study. Unless directly attributed, they have been paraphrased in bold at the head of each section. Joseph Curtin and Claudia Fritz have compiled and edited the authors’ responses, and have included various online comments that seemed eloquent and to-the-point. These are attributed to their (sometimes anonymous) authors, and posted in italics.

It’s not the player who should be judging the sound, but the listener!

Who is talking about "judging the sound”? The main goal of the experiment was to investigate player preferences, and so we worked with players. We were also interested in whether violinists could distinguish old violins from new by their playing qualities alone. Many earlier (albeit informal) tests have suggested that listeners can't reliably tell the difference between old and new. Nor do they consistently prefer old over new. Ironically, a frequent criticism of these earlier tests is that they were based on the judgments of listeners rather than players!

We believe there are many viewpoints from which to assess violin quality – listeners, players, ensemble leaders, recording engineers – and no a priori reason that any two should agree. The important thing from the scientific viewpoint is that each be studied by an appropriately designed experiment. Given that players alone can assess playability, and that players ultimately choose their own instruments, player-preferences seem a sensible place to begin. If subsequent research shows that player-preferences don’t often agree with those of listeners, that itself will be an important finding.

Players can’t possibly judge an instrument’s projection.

We agree. But players do routinely estimate projection, and typically acknowledge (as many of our players did) the need to re-test in a hall with trusted colleagues listening. Remember though that this experiment deals with the subjective impressions of players, not the objective characteristics of instruments. Correlating the two is an important topic for future research.

In anticipating a study of projection, many interesting questions arise. Does it make sense to test projection in an empty hall (as is routinely done), considering that the presence of an audience can substantially change the hall’s acoustics? Is playing a sequence of unaccompanied violins a true test for projection? After all, does not ‘projection’ imply being heard above an orchestra or some other musical background?

The fact that the three distinguished instrumentalists who publicly replied to our paper (Steven Isserlis, Earl Carlyss, and Frank Almond) all raised the issue of projection suggests it is an important one to investigate. Steven Isserlis asserts that “a famous (and curious) feature of Stradivarius instruments is that their tone seems to increase with distance.” Earl Carlyss, quoted in the New York Times, says: “The modern instruments are very easy to play and sound good to your ear, but what made the old instruments great was their power in a hall.” And Frank Almond writes that “a peculiar (and sublime) aspect of great old Italian instruments is that the sound somehow expands and gains more complexity from a distance, especially in a concert hall. With many “modern” instruments, it is common that in a confined space (under the ear), the sound may also be huge and seem quite promising. But frequent criticism of some newer instruments (not all) is that in a hall it sometimes doesn’t carry past the 6th row, and the sound is often bland or one-dimensional.”

We would like to invite these artists (and their instruments!), along with any others who are willing, to participate in a study exploring the nature of projection. This is a serious offer! The chance to work with top-level players and instruments is an invaluable opportunity to further our understanding of this fascinating subject. The anonymity of players and instruments is guaranteed. Please contact Claudia Fritz if you are interested.

Fine violins are built to sound good in concert halls, not hotel rooms. The test space was unsuited to evaluating old Italian violins.

Violins are expected to sound good in concert halls, but also in many other contexts, including for example chamber-music settings, where they must blend in with an ensemble, rather than project over an orchestra. Players use violins in all manner of practice rooms, and in the case of touring soloists, they are routinely played in hotel rooms!

None of our players gave any indication they felt the room we used was unsuitable for trying violins. In our experience, violinists consider an acoustically dry (though not dead!) room best for initial try-outs, as the direct sound from the instrument is not so much colored by room reflections. For this reason, relatively dry rooms were used for all the playing tests designed by Claudia Fritz and her collaborators (see some articles here). The question of how well judgments made in one room carry over into another is an interesting one, but outside the scope of this experiment. For our purposes, the important thing is that all instruments were tested in the same room.

The acoustics of the test room will affect the choices made by players.

Quite possibly, though this has not yet been studied scientifically. As every room has its own acoustics, the question becomes: which are best for evaluating violins? Judgments made in a concert hall reflect the acoustics of that particular hall, and the acoustics may change dramatically when a hall is full, making an empty hall a less than convincing test space. There is as yet no evidence to indicate that preferences made in one concert hall will carry over into another. Nor is their evidence that any single venue can be considered sufficient for judging violins in a definitive way. Professional violinists need instruments that work in a wide variety of musical contexts, and before purchasing an instrument, they tend to test them in a variety of spaces.

You can’t judge an instrument in a minute, or even twenty minutes. It can take years to get the best out of a Stradivari!

Or a new violin! Still, a player in search of an instrument will routinely eliminate violins after playing them for a few minutes. First-impressions have real-world consequences, and are therefore worth investigating. Of course, decisions based on quickly-formed preferences may or may not correlate with long-term satisfaction – an interesting subject for future research. And it may be that players expect more from Old Italian violins than new ones, and are therefore willing to work harder and longer at getting the best out of them. This was nicely expressed in a post by Magnus Nedregard: I think the greatest psychological difference between the grand oldies and the new instruments is how much we think they can do. Since you expect pretty much everything from an old Italian you keep looking and trying until you can find it. (Hence you need weeks). For a "modern" most peoples imagination seem to stop at a healthy and strong sound, and that's why people stop looking when they've found just that. Expectations are decisively important, so this test is highly relevant.

Even if it were eventually shown that it takes longer to “get the best” out of an Old Italian, that wouldn’t invalidate the results of this study. We have simply shown that, lacking any information on an instrument’s identity or provenance, violinists tend to select for further testing the instrument (whether new or old) they think best at the time.

You don’t buy a violin in twenty minutes!

Agreed. This is one reason we asked players to consider which violin they preferred in the moment, under test conditions, rather than which they thought the better violin, and then which they would like to take home for further testing, rather than which they might like to purchase. The idea was to emulate a typical real-life situation, where a player first tests a number of violins in a shop, then decides on one to take home for a longer trial.

Only violinists with long experience playing Old Italian violins can get the best out of them. Being an advanced student or experienced orchestral player does not guarantee the necessary qualifications to evaluate a Stradivari. Most of your subjects were therefore unsuitable.

Consider first the converse of this statement: Only players with long experience playing new violins can get the best out of them. Violinists who habitually play old Italian violins may therefore be unqualified to evaluate a new one. Both the original statement and its converse are dubious for at least two reasons: (1) they are neither self-evident nor supported by empirical research, and (2) they presuppose some general difference in playing qualities between old and new instruments – a notion this experiment is explicitly designed to test.

Remember too that the experiment was not designed to evaluate the quality of the instruments themselves (a project whose credibility might well depend upon its subjects being top-notch violinists). Rather, it was designed to test player-preferences, and the preferences of mid-level and even amateur players cannot be dismissed out of hand. Indeed, discovering that high- and low-level players differ in their preference for old or new violins would be an important finding.

Martin Swan writes: I don't think the Fritz test is trying to define what "top musicians" look for in a violin, but to establish some sort of protocol for examining how any and all musicians evaluate violins. So I don't understand the focus on virtuosos .... why would a virtuoso be a more useful subject for the Fritz experiment than a journalist who plays well? I accept that soloists' requirements are very specific, and that "nice tone' may be low on their list. However, they are particularly susceptible to labels - they know that when they walk out in front of an orchestra, the whole violin section is just thinking "what's he/she playing on .....?". I think it's nigh-on impossible these days for someone to give a high-profile concerto performance on a violin that cost less than 6 figures, unless they're very established and respected.

In fact our study showed no significant correlation between the playing-experience of the violinist (or the value or age of the violin they were currently using) and the age of their most-preferred instrument. In terms of long experience playing Old Italian violins, the two jury members (each of whom owns a Stradivari or Guarneri del Gesu) were the “most-qualified”. Both selected new instruments as their take-home choice.

Different bows were used by different players, and this introduces an unwarranted variable.

The question of which bow best suits a given violin and violinist is an interesting and important one. Given the lack of any research on the subject, we simply followed common practice: When violinists test instruments, they typically use their own bow, to which they are accustomed. This avoids the introduction of a new variable into their experience of trying violins.

We asked players to bring their own bows, and for those who came without, we provided a good-quality bow we felt was entirely adequate to the task. The fact that none of the subjects expressed concerns about bows suggests they did not feel them to be much of an issue.

Unlike the new violins used in this experiment, the Old Italians were not the best of their kind.

The choice of old instruments was limited by the (extreme!) difficulty of finding worthy examples whose owners were willing to loan them for an experiment of this kind. We would certainly have included other Old Italians, had any been offered. Furthermore, we would welcome the chance to redo the experiment using ‘better’ examples – though the question then becomes: How do we know they are ‘better’? By their reputations? By our own impressions? By those of an expert player? By a preliminary round of double-blind tests?

A surprising result of this and other research is the variety of tastes among players. Except for N2 & N3, each violin in this study was chosen as take-home by at least one player – and as least-favorite by at least one other! This carries through into the four categories, where most of the instruments were designated both best and worst at least once in each category.

That said, the drift toward N2 and away from O1 confirms the notion that some instruments are generally preferable to others. Given access to a sufficient number of instruments for a large-scale study, the generally-preferable old and new violins could perhaps be separately identified by preliminary double-blind tests, and then brought together for a final round.

Regarding our choice of new violins, since the old violins were by makers widely regarded as the greatest who ever lived, it made sense to find new instruments that were representative of the best contemporary work. To this end, we assembled a pool of about ten new instruments. From these we selected the three we felt would (1) best stand the comparison, and (2) present a range of tonal characters appealing to a range of tastes.

A different set of old or new instruments would doubtless lead to a different set of individual preferences. It might also shift the balance in favor of the old. We doubt, however, that players would suddenly get better at distinguishing old from new. But please help us find out! If you have access to a distinguished old Italian instrument and are willing to participate in future research, please contact Claudia Fritz.

Martin Swan writes: Let's forget for a while about what the violins were and concentrate on what's revolutionary about this test. It's a peer-reviewed experiment, well conducted and published in a major scientific journal, which is not about impact hammers, ffts and spectrum analysis, but about what players actually look for in violins. Maybe people should take a time out and allow that idea to actually sink in ...

The Old Italians were not optimally set-up or adjusted.

This wasn’t the opinion of their owners, who asked us not to change anything. The instruments were periodically checked during the experiment to make sure nothing had changed.

The question of set-up and adjustment is, however, an important one, and it raises many others. To whose taste should an instrument be adjusted? That of the owner? A panel of‘neutral’players? An expert adjuster? While we readily acknowledge the importance of set-up and adjustment, it is by no means easy to incorporate such variables into an experimental design. David Burgess puts it well: Let's face it. There is no way any such test can be done, that someone can't come up with a way to say, "but-but-but-but". . . If the instruments had all been adjusted immediately prior, then people would be complaining that the instruments hadn't had enough time to settle in after the adjustment. If it was done a month before, then "oh, they probably had gone out of adjustment". And either way, it could be said, "The adjuster may have imposed his taste in sound on the instrument, so it wasn't the instrument alone that was being heard". . . These are all factors which could come into play in any musician's real-world typical selection process, and could be just as much of a disadvantage to the new instruments as to the old.

The Old Italians may not have been properly played-in prior to the experiment.

The issue of “playing in” an instrument is another fascinating subject, but beyond the scope of this study. All of the Old Italians had been played regularly prior to the experiment, while one of the new violins was only a few days old!

Sample sizes were too small to draw conclusions.

We believe (as did the PNAS reviewers and other scientists) that sample sizes were quite adequate for the rather modest conclusions we did draw – though not for the sweeping generalizations expressed by outside commentators and media headlines. The violin-world being what it is, we doubt that fine old violins will ever be available in sufficient numbers for a large-scale experiment. This means that many small-scale tests are needed in order to get a clearer understanding of the many complex issues involved.

This was posted by mikeyk1: To me as a scientist [the study] looks fundamentally solid, and a 21 x 6 sample is large enough for a statistically significant result, which is that there is no correlation. Of course you can always ask for more and more and evermore research before you're satisfied, but these sorts of things take time and money ...

Generations of violinists are either fools or this research is flawed.

mikeyk1 replied: Don't underestimate the power of mythmaking. This wouldn't be the first time scientific research dispelled some long-held and cherished myths. It sounds more like you . . . cannot come to terms with a piece of research that contradicts a piece of orthodoxy held in some musical circles:The apparent belief that nobody these days can ever make violin as good as an old Stradivarius.Yet there seems to be nothing beyond prejudice to back this belief up.

The chief failing of this test, as reported in the article, is that the players are not identified. Who were they? (Steven Isserlis)

mikeyk1: The identity of the violinists is irrelevant . . . [not] least because it would become a discussion about the participants, not a discussion about the science. Participant anonymity is a well established research principle, enabling anyone to take part in research and express their honest views and findings without reprisal. Not every musician is eager to tell their boss they took part in an experiment that may undermine the boss's sensitive and sacredly held beliefs.

There’s no such thing as an objective assessment of tone quality.

Yes, quality assessments are subjective, but scientific experiments that compile and correlate many subjective assessments can yield objective information about what players tend to like or dislike. Only with this kind of information in hand does it make sense to go after that long-standing goal of violin research: objective physical measurements of instruments that correlate with player-assessments of their playing qualities.

Tests like this might work on wine, but violins have far too many variables.

Tests like this work on humans. Keep in mind that our results were obtained under specific experimental conditions, so many more tests are needed to cover the depth and complexity of the subject. As progress depends on collaboration with players, makers, and collectors, we once again invite anyone interested to please contact the authors!