Double blinding violins (prasad)

If you enjoy a good debunking, this one's a doozy. A scientist and contemporary violin maker conducted blind comparisons, getting professional violinists to try and choose among three old Cremonese violins, including two by Stradivari, (total value 10M) and three high-end modern violins (total value 100k). They used a clever blinding protocol:

With modified welders’ masks on their faces, to restrict vision, and a dab of scent under the violins’ chin rests, to mask tell-tale odors, the participants got to play compare six violins, including two by Stradivari and one by Guarneri. In an initial test, they quickly compared 10 pairs of violins — nine distinct new-vs.-old pairings and one repeat pair. They got to play each instrument for one minute, without visiting the first instrument played, in a room with neutral acoustics. In a second test, they were allowed to compare and contrast the six instruments in a more natural way, playing them all however they wished, for 20 minutes.

The results are unsurprising to any decent cynic:

In the head-to-head tests, the players preferred new and old instruments at equal rates, except in one case: One violin by Stradivari, crafted in roughly 1700, was clearly preferred less often than the others. “It seems that under these test conditions, only a conspicuously least-preferred violin differentiates itself,” the authors write. (What’s more, when the violinists compared the same two instruments twice, they made the same choice only 52% of the time.)

In the more leisurely comparison, the same Stradivari model was again voted as worse than the other violins (new or old); four instruments received statistically indistinguishable ratings. This time, however, a distinctly preferred model arose — and it happened to be a new violin.

The story has been received enthusiastically on science and technology sites, and the analogy to high-end wine has been drawn more than once. I don't think that's quite right though. This isn't really like the claim that wine snobs can't really tell plonk from Chateau whatsisface - the new violins themselves cost tens of thousands of dollars each. Indeed, even in this group of six superb violins, there appears to be some degree of agreement at least on the best or worst instruments. Nor is the psychology really related to the rubbish about finding notes of badger and ear-wax in a single-malt; This is about love, not bull-shit. Great musicians fall in love with great instruments they've played with.

But we do need to expand our sense of what a great instrument (and instrument-maker) is. It's remarkable that craftsmen hundreds of years ago could make instruments that can (almost) hold their own against our labs and years of accumulated practice. But clearly there are superb violin-makers today, and the very best might even be better than the Great ones. It seems like a shame for them to not be well-known, and positively wrong to impose upon them the burden of predetermined, unwinnable comparisons with the long-dead. And it's long past time for the audiophile-type magic-fiddle stuff to die.

Comments

[Ruchira]

Ah, Prasad the philistine strikes again! Don't break the hearts of the romantics and cognoscentis via science & technology.

Now I will go and check my meager wine rack and see if I have any Chateau whatsisface. Probably not, I rarely pay more than $12.99 for a bottle.

[Lester Hunt]

This makes perfect sense to me. I've always thought that the advantage, if there is one, that an old Strad has over the products of the best modern makers would have to do with the aging of the wood (micro-cracks that make the timbre more mellow, more smoky, etc.). The idea that Stradivari himself was better than the best modern makers seems implausible to me. After all, this is a craft, not an art. In the crafts, if not in the arts, you have progress.

[Sujatha]

If the comparison is between the best outputs of the current violinmakers and centuries-old Stradi, I am totally unsurprised that some modern violin sounded better to the violinists' ears than a 1700s Stradi. If the labels had been uncovered, there might been no contest between the sound from the Stradi or Guarneri vis-a-vis the newer violins. A violinist's perception of how the instrument sounds is still reliant on his/her subjective perception. But it is still quite possible that the 'clunker' Stradi would have received only a middling review, instead of being the least-preferred.

There is such a thing as the awe-factor, especially when pricing antique violins. Collectors of those are paying as much for the antique value and the anticipation of future price-rises as the instrument and its musical value alone.

What do I know? I'm not a professional violinist, but my son plays a $600 violin purchased off of eBay that has a sound quality that rivals many a $2000 violin. And, as my son's violin teacher says, a good violin will start to sound better the more it is played and as it ages.

[Pete Chapman]

It's the welders' masks...don't you see? They blocked the psychic emanations from the older instruments and prevented the players from feeling the thought signatures of all of the other players who had used those instruments...Yes, this levelled the playing field but at such a cost! The only way to do it fairly would be to place a thought wave amplifier (copper funnel) over the bridge of the violins but that would alter the sound...oh well I guess we'll never know.

[Norman Costa]

Let's face the music and dance (or play or listen):

Being a scientist, and familiar with experimental design, I thought I would contribute something unique, clever, and incisive to the discussion. After all, this is my area. Sensory perception was one of the earliest experimental study topics for psychology. However, Pete Chapman beat me to it. And I'll bet he's not a researcher in psycho-physical phenomena.

The experimental procedure and apparatus introduces its own variables into the experimental equation. The welder's mask, the scent, the requirement to play the instrument (What about listening only?), and the selection of musical pieces contribute to the results in their own way. A good experimental design tries to eliminate all extraneous variables - this is what is called control. Sometimes, variables cannot be eliminated, so we distribute their effects as evenly as possible to each of the conditions. This is not control. It's balance.

Of course, there is the sample of musicians in this study. Whom do they represent? Until there are more studies on the subject, one can only say that the results are valid for this sample, for these variables, and for the time and circumstance existing at the time. Temperature and humidity? What about the sampling of violins? Are they representative of the larger population of Cremonese products of 400 years ago?

@ Lester: You wrote: "After all, this is a craft, not an art. In the crafts, if not in the arts, you have progress." I have no knowledge to contribute here. I just wonder if you would get bouquets or brickbats from other violin makers and musicians.

@ Ruchira: In my late 20s, I was at my first wine tasting party at the home of friends. He and she were wine enthusiasts and would take vacations through 'wine country' and visit and sample from many vineyards. The host set up a single-blind procedure, handed out prepared rating forms, and introduced 'neutral' crackers between samples to wipe clean each discerning palate. The results: The cheap wine was just that, cheap. The two expensive French wines were middling. The moderately priced California wines were the best. To my younger palate, the expensive French wines were like good flavored vinegar.

[Dean C. Rowan]

A wannabe audiophile, I've had my fill of double-blind tests and whatnot. Like Sujatha's son and his violin, I've learned how to spend fairly prudently on equipment (every piece nevertheless depriving my children of a meal or two), yet without settling for merely decent budget gear. The last centimeter of high fidelity quality would cost more than I'll ever be capable of spending. Besides, the most important component in a stereo system is the space, and I won't be building a listening room anytime soon. Still, isn't it just weird to try to rig a facade of objectivity about a circumstance that is inherently subjective? What's debunked here? Pretentiousness? That required a double-blind test?

There are of course experts who can distinguish wines and discern terroir. I'm not one of them, but I do keep my distance from domestic grog, including California wines, which taste to me of bubble gum.

[prasad]

Ruchira, you *have* a wine rack? Next you'll be saying you know a foreign language.

Pete, lol. Don't forget the delicate interplay between the violin and the violinist, his body, psyche and temperament, irreducible even in principle. The violin chooses the player, not vice versa.

Dean, re point of debunking, I'd say the usual things about the value of skepticism, evidence-based reasoned argument etc. In addition, introspection makes clear that my taste for this sort of thing is not entirely distinct from that for, say, Sarah Palin bashing or India beating Pakistan or PCs against macs. Rah-rah my team etc. I don't think pretension is the point - I don't think Yehudi Menuhin was faking it. I'd even cop to a certain marginal dewy-eyed romanticism about how unusually rotten people tend not to make the best musical performers. But self-deception can be rife. I don't see quite so much instrument fetish in Indian music, but there's something similar about the alleged microtonality of ragas, which may be subject for another post. You'll find utterly down-to-earth and unpretentious ustads, wonderful people, saying hopelessy incorrect (and confused) things about the tonal character of pitches they voice between two notes. It's plausible that they're magnifying music, not themselves.

[Dean C. Rowan]

Prasad, re: microtones in Indian music performance -- Are you saying the musicians don't know a microtone when they hear one, or that microtones play no part in the music? Or is there a besotted cult of microtonal music akin to old violin worship or expensive bad wine consumption? Why are microtones in ragas only alleged? There's nothing especially mysterious of implausible about the concept...is there?

[prasad]

Dean, I'll point you to the wiki page in lieu of a long explanation. Basically, while pitches between (or oscillating around) ordinary notes (well, modulo temperament etc) play a significant role in Indian music, there's a also musicological theory about there being 22 actual microtones (including the 12 notes) and how musicians actually use these in singing. Tests (as well as simply the experience of learning and listening to classical indian music) make quite clear that the notion of ten extra discrete pitches in an octave has NO relevance to the music as either performed or taught...

[Dean C. Rowan]

Judging from the wiki page on srutis, related pages, such as the one pertaining to microtonal music and the one on the value of a cent in music, and the first couple pages of Krishnaswamy's latest piece cited on the wiki page, I'd say this is a complicated issue, tied inextricably to the traditions of Indian classical music performance. Perhaps ten extra discrete pitches is excessive, but even the article suggests that "the number of perceptible intermediate tones may be less or even much more than 22."

[Ruchira]

Forget about the intricacies of violin sounds Indian music, Dean. Start the work on Indian ghosts:-)

[prasad]

Dean, if you feel like reading up, it's quite a fascinating topic. I've seen good review articles by Harold Powers as well as by Nazir Jairazbhoy. Most convincing perhaps is the empirical work, and Arvind Krishnaswamy has done very nice stuff, as have others. I don't think it's disputed that Indian musicians use non-note sounds - it's quite clear from just listening that they do. The question is whether such sounds have any connection to a certain historical musical tuning system involving 22 (or more) *discrete* pitches. There's every reason to think this is false, that musicians when they perform aren't implicitly using/approximating *any* set of discrete extra notes besides the usual ones.

[Elatia Harris]

I'm going to send this link to Keith Hill, a brilliant instrument maker who has studied violin acoustics like no one else I know of. If we are very lucky, we'll see what he has to say.

[Dean C. Rowan]

Can't wait to hear from Hill. He built instruments played on the late Joseph Spencer's Wildboar label recordings, some of which feature harpsichords tuned to meantone temperament. The effect is relevant to this discussion. The tuning produces a sour quality to the performance; it's not a function of the introduction of different pitches, but of microtonal adjustments. Also, Ruchira, the effect can be eerily ghostly.

Sad news: Nikolaus Harnoncourt has died.

[Elatia Harris]

I posted on behalf of Keith Hill last night and it disappeared! Stay tuned...

[Elatia Harris]

Dean and interested others, worst case scenario, I can send you Keith's comment via email. Please contact me at [email protected] if Ruchira can't wrestle down the house gremlin and make a post for Keith that sticks!

[Ruchira]

Elatia, I have published Mr. Hill's response as an independent post. Couldn't do it as a comment. It could be because Dean and I mentioned ghosts :-)

[Elatia Harris]

If we don't talk about religious painting or discarnate entities, will Typepad let us back in their good books? Jesus, Mary and Joseph, I hope so!

[Ruchira]

I think one point that seems to be getting lost here is what Prasad points out in the penultimate paragraph. It is clear that the "new" violins are not cheap and shoddy articles - they are very good and I am assuming that the instrument makers took care to capture all the qualities of a good violin that Keith Hill lists in his comment. So, the comparison is indeed between apples and apples, except for age and reputation. What does the test prove? Exactly what Dean suspects - demolition of pretentiousness and audio-mystique. Is it necessary? I don't know. But the truth never hurt anything except delicate sensibilities. Will the musicians change their minds? Depends on the musician.

It will be interesting if a blind study is done in a concert hall with the right kind of audience and acoustics. Let the same musician play a set of old and new violins in a way that the experts, musicians and music lovers, cannot see the instruments but can only hear them without knowing which ones they are listening to. No need for welders mask or dabs of perfume under the nose.

[Sujatha]

I thought that the dabs of perfume and welders mask was meant to blind the musician as to which violin they played. That is an important component of how well their subjective perception of the violin's sound could be altered. If I were a violinist playing what I thought was an antique Stradivarius, I might be conceivably inspired to play better than if I thought that I was playing a modern albeit top-notch violin.

[Ruchira]

Sujatha, you are right. I meant the audience don't have to wear masks or perfume. The violinist will be treated the same way. Then qualities like flexibility and intensity can be judged from a distance - a double-double blind test.

[Norman Costa]

I tried to comment here, yesterday, but it disappeared into the Grand Canyon. After submitting a post, and then typing in the correct pass phrase, it appeared to accept and display my comment. Then I clicked on 'post another comment' and poof, it disappeared.

[Ruchira]

Norman, it happened to me too and not just when I was trying to publish Keith Hill's comment. We have been suspecting that TypePad may be averse to supernatural / religious discussions. I now feel it may be art!

[prasad]

Ah, the joys of typepad! In this case it's perhaps worked out for the best. I'll migrate over there.

Re welder masks, the kind of test you suggest has been done - that's the more practically useful sort of test and easier to do. Those tests show roughly that listeners can't make much of violin differences in concert settings (well, but not all audiences or members are equal etc). A persistent *criticism* of those tests has been that the musician hasn't been blinded. This goes both ways - from detractors (who say the violinist is the best judge, that from close by and thru intimate contact with the violin they hear and feel subtleties that are lost on a mere audience) and defenders of testing (who say the violinist plays better on what he regards as a good/old instrument). The welding perfume stuff is a clever twist that someone tried to shed light on those issues, done after many iterations of the more standard listening tests. Of course, NOW some people say the violinist is ill-placed to judge what the music sounds like because he's too close to it!

[Elatia Harris]

The more I think about it, the more the test as written up in the article Prasad references sounds rather like a Marx Brothers affair. Or like a blind tasting of wines with the tasters required to wear dental dams to significantly disguise the taste of the wine. Everyone here but me is a scientist by training -- what WOULD a good experiment to tease out the most meaningful and distinguishing qualities of assorted violins consist of?

Prasad, you were being facetious, but you hit on it. The player cannot from his unique point of view tell exactly the sound of his violin to the house. How could he? He can know about its playability however. One reason opera singers stick close to their coaches is to have -- right at hand -- the ear of the person who can not only help them stay on top of voice production techniques that will enable their voices to last, but to allow them to know how they sound to someone other than themselves. Visual artists who work in 2-D have an imperfectly analogous trick to help them SEE their paintings when they are all too familiar with what's there -- they turn the canvas upside down, or they behold it in a mirror. It's a jolt that almost substitutes for another point of view...

[prasad]

Elatia, (thanks for forwarding the post to Keith!) I'm not trying to make the violinist into some sort of unique arbiter. It's quite likely that a teacher, the conductor, the recording engineer, the person way back in the stands, the fellow who's practically sitting on the stage etc etc all have something to say. There may be complementary value, or even tradeoffs among such perspectives.

My facetious comment was about how suddenly it seems violinists are sub-optimal judges now that they seem to be incapable of making certain kinds of cherished distinctions. If you look at comments and responses to previous studies (where typically the audience, not the player, is tested) you'll frequent claims that the violinist is not just a better judge but in a sense the best one. Now audience is king. It feels rather like whack-a-mole!

[Ruchira]

I meant that the violinist WILL wear mask and perfume as in the study described - both the musician and the audience will be unaware of the provenance of the instrument.

[Elatia Harris]

Got it, Prasad -- thanks! I am not terribly bright at 3 am but I am alas awake.

[Elatia Harris]

Latest comment swallowed.

[prasad]

Hmm. I was just rooting around in the typepad area for the blog, and actually found Norman's lost comment in the spam area. I de-spammed it, so it's on the first page now. Elatia, I can't find your latest, so there seems to be more than one problem with the comments here.

[Ruchira]

I found both Elatia's and my attempts at posting Keith Hill's comment in the spam filter. I won't publish them because it has already appeared as a post. I can't find Elatia's more recent comments. Very frustrating especially because TypePad's help desk don't seem to know either.

I have deleted Elatia's comment from Keith Hill that was marked as spam so that TypePad won't tag her address. But she is able to post most of the time. So there is something else going on.

[Elatia Harris]

If this comment is not swallowed, it will be illuminating about Typepad's vagaries. I did not see there was a second page to the comment thread at the time I last commented, and my comment appeared to have disappeared, but did not. It was at the top of what was at the time the second page, but now that Norman's comment has been restored, my supposedly eaten-for-breakfast comment appears in a different place in the whole array. So it may be that only the spam filter is operating against us here, and not something harder to pin down.

[Sujatha]

Ha, I was just waiting for someone to jump in with a reference to vocal music. Thanks, Elatia, for giving me the chance to pontificate.

When I sing, I have a general sense of how I sound, but I cannot precisely tell what the listener is hearing.
One method to hear how I sound to others is to cup my hand around the ear while I'm singing, but that would be a public singing no-no, typically. Some Indian singers on the concert stage don't hesitate to use that technique, as I've noticed.
Another method would be to record it and listen to it, after the fact, assuming that the recording is reasonably faithful to the original.