Blind test in which Strads and del Gesus came out on top

[Bruce Tai]

There have been many blind tests in which Strads or del Gesus did not better modern violins. 

Fritz et al. published theirs in a series of three papers in Proceedings of the National Academy of Sciences, USA. Many take them as scientific proof that Strads do not possess superlative tonal qualities. 

I am delighted to see that some blind tests do show favorable results for Strads and del Gesus:

(1) One organized by Strad Magazine

https://www.thestrad.com/stradivarius-violin-tops-the-strads-blind-test-of-old-and-modern-instruments/5129.article

 

(2) One conducted at the Joachim competition

YouTube video below  (go to 48:00)

 

There are two major problems with blind tests, such as the ones mentioned above and those by Fritz et al. 

 First, working memory for timbre only lasts from a few seconds to tens of seconds in humans, shorter than the time interval required for the player to switch instruments and play the same passage. The timbre memory of the previous instrument may have already decayed when the next violin is being played.

Secondly, Fritz and coworkers did not measure or report the loudness of individual instruments, meaning that subjective evaluations about timbre and preference could have been confounded by differences in loudness. Even if loudness were measured, there would have been no simple method to normalize for inter-instrument differences during live listening tests. Louder violin tones usually sound fuller and more preferable in side-by-side comparisons. Hence, without loudness equalization, it would be difficult to properly assess timbre.

Scientifically, timbre is defined as the character or quality of a musical sound distinct from its loudness, pitch, duration, and spaciousness. What kind of blind listening test is sufficient for differentiating violin timbre remains unclear. Do we need to pre-screen the listeners for those with string auditory working memory? How about pre-screening the Strads  involved and only admit the truly great ones? 

 

 

[Quadibloc]

This is encouraging news. Of course, it is only a beginning; it isn't science to do repeated experiments, and then stop as soon as you get the results you want. But if it is possible, as a result of these tests, to improve the technique of conducting blind tests, so that they produce more consistent and conclusive results in the future, that will be very helpful.

And, indeed, I'm well aware from comparison tests of audio reproduction equipment that a small increase in volume can given an appearance of better quality.

[martin swan]

4 hours ago, Bruce Tai said:

 

 

 First, working memory for timbre only lasts from a few seconds to tens of seconds in humans, shorter than the time interval required for the player to switch instruments and play the same passage. The timbre memory of the previous instrument may have already decayed when the next violin is being played.

 

 

Absolutely the opposite of my experience and that of everyone I have watched trying violins over the years. Where do you get this from?

My observation is that every violin is judged largely in comparison to the violin heard just before it. 

Gross judgments such as loud, sweet, dark, bright etc are made almost entirely with reference to the previous instrument, and very very few people can carry a tonal "ideal" in their heads independently from what they've just played. I would say it takes more like a minute of playing to expunge the imprint of the previous instrument.

[Marty Kasprzyk]

7 hours ago, Bruce Tai said:

There have been many blind tests in which Strads or del Gesus did not better modern violins. 

Fritz et al. published theirs in a series of three papers in Proceedings of the National Academy of Sciences, USA. Many take them as scientific proof that Strads do not possess superlative tonal qualities. 

I am delighted to see that some blind tests do show favorable results for Strads and del Gesus:

(1) One organized by Strad Magazine

https://www.thestrad.com/stradivarius-violin-tops-the-strads-blind-test-of-old-and-modern-instruments/5129.article

 

(2) One conducted at the Joachim competition

YouTube video below  (go to 48:00)

 

There are two major problems with blind tests, such as the ones mentioned above and those by Fritz et al. 

 First, working memory for timbre only lasts from a few seconds to tens of seconds in humans, shorter than the time interval required for the player to switch instruments and play the same passage. The timbre memory of the previous instrument may have already decayed when the next violin is being played.

Secondly, Fritz and coworkers did not measure or report the loudness of individual instruments, meaning that subjective evaluations about timbre and preference could have been confounded by differences in loudness. Even if loudness were measured, there would have been no simple method to normalize for inter-instrument differences during live listening tests. Louder violin tones usually sound fuller and more preferable in side-by-side comparisons. Hence, without loudness equalization, it would be difficult to properly assess timbre.

Scientifically, timbre is defined as the character or quality of a musical sound distinct from its loudness, pitch, duration, and spaciousness. What kind of blind listening test is sufficient for differentiating violin timbre remains unclear. Do we need to pre-screen the listeners for those with string auditory working memory? How about pre-screening the Strads  involved and only admit the truly great ones? 

 

 

These weren't double blind tests (where the player can't see out of both eyes).

[Guy Harrison]

6 hours ago, Bruce Tai said:

 First, working memory for timbre only lasts from a few seconds to tens of seconds in humans, shorter than the time interval required for the player to switch instruments and play the same passage. 

Hi Bruce-  can you point to study to back this claim up?   I don't recall if this has been proven anywhere. 

6 hours ago, Bruce Tai said:

I am delighted to see that some blind tests do show favorable results for Strads and del Gesus:

(1) One organized by Strad Magazine

https://www.thestrad.com/stradivarius-violin-tops-the-strads-blind-test-of-old-and-modern-instruments/5129.article

 

With this test there were two obvious problems.  It wasn't a double blind test. ( the audience weren't aware of what they listen to, but the musician was aware what he was playing)  Plus one of the instruments included was the violinists own regular violin (the 1709 Strad). It would have been better not to include this Strad, so each of the instruments would have been equally unfamiliar to the player.  So I wouldn't call this a scientific test,  rather more like mildly interesting promotional entertainment. 

[Andreas Preuss]

A few thoughts on the sound testing conducted by  the STRAD magazine.

1. The Strad played in comparison with the other instruments was the players own instrument and apparently he knew what instrument he is playing during the test. As far as I know in the Fritz test players didn't even know what they were playing. This makes a difference.
2. Sometimes I wished that the description of the played instruments would be a little preciser. We all know that  a del Gesu model MUST sound different to a Strad model and can make a difference in the testing. In particular I would have liked to know what was the model for the Vuillaume and the ASP Bernardel. Both makers made Strad as well as del Gesu.
3. In general I wished that in the case of the compared Strads rhat they would give an approximate outline of the  condition especially of the top. If we really take an over restored Strad and expect it to be better than a healthy new instrument by one of the best makers today, does this really show which instrument is better?
4. I wished as well that the audience would be tested. For those 6 instruments they could have made a second round in a different order and ask who was able to recognize all the instruments by their number. In the final (3rd round) I would include only people in the audience who were able to identify the instruments by their number, which means that they had a sort of memory for the sound.

We can well ask ourselves what would be the outcome of this experiment in the order 

1-5-2 and 3-4-6?

Edited May 6, 2018 by Andreas Preuss
additional comment

[Torbjörn Zethelius]

Post moved from another thread: I'd like to see a blind test between old vs new where some of the best old ones –by any maker, not necessarily Strad/del Gesu– are compared to some of the best new ones. Then repeat the test with some other violins. That could settle the debate of old vs new, or at the least bring it forward.

I would also like to see an audience of professionals with experience of listening to great instruments.

[Andreas Preuss]

Here is how I usually try to identify the sound of a Strad (and I think it is not horribly difficult)

I mostly listen to the G string alone. There is a profound depth in the sound which is matched by only a very few more modern instruments. This depth can go to the point that is sounds almost a bit hollow.

To distinguish from more modern instruments with a deep profound G string I listen to how rough the G string sounds. New instruments have almost always a kind of crispier or one could say rougher sound.

The last round in the Strad test makes this pretty clear. Both had the same depth but the modern instrument was a bit on the rough side for the G string, while the G string of the Strad was beautifully round and mellow.

I don't say this a perfect recipe but works probably for 80 percent of cases to pick the Strad from other instruments. Check it out yourself.

[GeorgeH]

45 minutes ago, Andreas Preuss said:

I mostly listen to the G string alone.

They did all use the same brand of strings on all the instruments broken in for the same amount of time, didn't they?

This so-called "blind test" was hardly scientific.

Quote

Speaking at the event, Simovic complimented the tone of Ihle’s violin, but expressed his preference, in general, for older instruments, saying he believed one can hear ‘the passage of 300 years in the sound’.

Do you think they served drinks? 

Like you, Andreas, I go to the G-string first - I love a deep G you can sink deeply into. But many fine violins made since the 18th century through today have this quality, not just Strads.

 

[Marty Kasprzyk]

8 hours ago, Bruce Tai said:

There have been many blind tests in which Strads or del Gesus did not better modern violins. 

Fritz et al. published theirs in a series of three papers in Proceedings of the National Academy of Sciences, USA. Many take them as scientific proof that Strads do not possess superlative tonal qualities. 

I am delighted to see that some blind tests do show favorable results for Strads and del Gesus:

(1) One organized by Strad Magazine

https://www.thestrad.com/stradivarius-violin-tops-the-strads-blind-test-of-old-and-modern-instruments/5129.article

 

(2) One conducted at the Joachim competition

YouTube video below  (go to 48:00)

 

There are two major problems with blind tests, such as the ones mentioned above and those by Fritz et al. 

 First, working memory for timbre only lasts from a few seconds to tens of seconds in humans, shorter than the time interval required for the player to switch instruments and play the same passage. The timbre memory of the previous instrument may have already decayed when the next violin is being played.

Secondly, Fritz and coworkers did not measure or report the loudness of individual instruments, meaning that subjective evaluations about timbre and preference could have been confounded by differences in loudness. Even if loudness were measured, there would have been no simple method to normalize for inter-instrument differences during live listening tests. Louder violin tones usually sound fuller and more preferable in side-by-side comparisons. Hence, without loudness equalization, it would be difficult to properly assess timbre.

Scientifically, timbre is defined as the character or quality of a musical sound distinct from its loudness, pitch, duration, and spaciousness. What kind of blind listening test is sufficient for differentiating violin timbre remains unclear. Do we need to pre-screen the listeners for those with string auditory working memory? How about pre-screening the Strads  involved and only admit the truly great ones? 

 

 

What should we would expect from a magazine named "The Strad"?

[Bill Merkel]

The fact that these tests are done at all says there's no obvious difference 

[Don Noon]

1 hour ago, Torbjörn Zethelius said:

That could settle the debate of old vs new, or at the least bring it forward.

22 minutes ago, Bill Merkel said:

The fact that these tests are done at all says there's no obvious difference 

The debate can never be settled, as it depends on human judgement of instruments that are all different.  In every test, every instrument will have some who think it's the best and some who think it's the worst, and if the test is repeated with the same instruments and people, the results will not come out the same.

There can be a difference, perhaps not obvious, and some may prefer one character, and others prefer a different character.  And in testing whether people can differentiate old vs new sound, you have to consider that people might not necessarily know how to hear the difference, assuming there is one.  This is (or should be) a distinctly different test objective than just determining "preference".

In "The Strad" test of 6 instruments, I thought the Strad sounded the worst of the group... but the recording was awful, and my tonal preferences are what they are.

 

[Bruce Tai]

4 hours ago, Guy Harrison said:

Hi Bruce-  can you point to study to back this claim up?   I don't recall if this has been proven anywhere. 

 

 

There are many papers that show working memory for timbre only lasts seconds. Here is a list of the ones I consulted:   

McKeown D & Wellsted D (2009) Auditory memory for timbre. J. Exp. Psychol. Hum. Percept. Perform. 35:855-875.

Mercer T & McKeown D (2014) Decay uncovered in nonverbal short-term memory. Psychon. Bull. Rev. 21:128-135.

Cowan N (1984) On short and long auditory stores. Psychol. Bull. 96:341-370.

[Quadibloc]

In the YouTube video which included the test at the Joachim competition, just after that was an experimental setup using digital signal processing to simulate the resonances of a Stradivarius for an electric violin.

I had seen a paper from Inria on this sort of thing, but now seeing the compact set-up, I was prompted to do a bit of web searching.

The Saelig (or Signal Wizard Systems) vSound is a commercial product embodying this notion.

Obviously, since there are so many ways to play a note on a violin, sampling a Stradivarius for a Mellotron or one of today's digital sampling keyboards is not a way to make a cheap quasi-Stradivarius, but this approach, with an electric violin (not a MIDI violin!) as input device might work. (Of course it misses the complicated directionality of a violin's sound, but one can't have everything, and so this might be "good enough".)

Edited May 6, 2018 by Quadibloc
Including name of manufacturer, not just distributor

[Guy Harrison]

1 hour ago, Bruce Tai said:

 

There are many papers that show working memory for timbre only lasts seconds. Here is a list of the ones I consulted:   

McKeown D & Wellsted D (2009) Auditory memory for timbre. J. Exp. Psychol. Hum. Percept. Perform. 35:855-875.

Mercer T & McKeown D (2014) Decay uncovered in nonverbal short-term memory. Psychon. Bull. Rev. 21:128-135.

Cowan N (1984) On short and long auditory stores. Psychol. Bull. 96:341-370.

The first two studies are a long way from the issue of listening to violins.  I'm not sure how relevant they are to the question at hand.  

[Argon55]

This study by the Strad magazine has been effectively critiqued by others here and it clearly adds nothing to the debate because of its limitation in design.

But it's interesting that some here find the fact that The Strad study was encouraging or somehow a good thing. In other words, that something made in the 18th Century was better than it's modern equivalent. I've been trying to think of things, any things, made in the 18th century that are better than today's, and can't think of a single object. This isn't surprising given the increase in knowledge since then, together with improvements in materials and technology.

So why should Strads be anything different? There are infinitely better tools today than in Stradivarius time, centuries of research on violin-making, greater knowledge of accoustics etc etc. I don't doubt that he was one of the greatest makers ever and made breathtakingly beautiful instruments, but I'd bet that there are some modern makers (perhaps including some posters on here) that are better. And this seems to be borne out by the comparative acoustic studies such as they are. Surely that's good news and not problematic?

[Quadibloc]

46 minutes ago, Argon55 said:

In other words, that something made in the 18th Century was better than it's modern equivalent. I've been trying to think of things, any things, made in the 18th century that are better than today's, and can't think of a single object. This isn't surprising given the increase in knowledge since then, together with improvements in materials and technology.

So why should Strads be anything different?

Today, I'm not so sure about. But that neither Vuillaume or anyone in the Victorian era quite equalled Stradivarius, or even Bergonzi or Guadagnini, seems to be reasonable to believe.

Of course, luthiers are in an awkward position. It is uncomfortable to always be in the shadow of Cremona, especially if one believes it to be undeserved.

But a possible alternative is that exploding the Stradivarius mystique will lead to the conclusion that there are no real differences in sound quality between violins (at least above the very cheapest ones made carelessly or out of inadequate materials like plywood).

That conclusion won't bring more customers to the door of the quality violin makers of today!

[palousian]

1 hour ago, Argon55 said:

I've been trying to think of things, any things, made in the 18th century that are better than today's, and can't think of a single object. This isn't surprising given the increase in knowledge since then, together with improvements in materials and technology.

Well, I thought of one right off the bat and I'm sure Violadamore would agree with me-- katana (Japanese "samurai swords").  To be honest, better ones would be from the 17th century, or even the 13th century, but you get the idea.  Also, carpets--absolutely.  Classical music, too, honestly, though I'm sure that will get some folks worked up.  Still, the argument can be made that it all went downhill after Napoleon.

[Argon55]

2 minutes ago, palousian said:

Well, I thought of one right off the bat and I'm sure Violadamore would agree with me-- katana (Japanese "samurai swords").  To be honest, better ones would be from the 17th century, or even the 13th century, but you get the idea.  Also, carpets--absolutely.  Classical music, too, honestly, though I'm sure that will get some folks worked up.  Still, the argument can be made that it all went downhill after Napoleon.

Ah, thank you for that. I'd heard that old swords were good but it's not an interest of mine. But.....are they better aesthetically (which I could believe as I wasn't really thinking about aesthetic or artistic differences but rather material objects which have specific functions) or is the steel better (which would surprise me) and if so, is there any objective evidence for that? Same would apply to carpets.

[Marty Kasprzyk]

48 minutes ago, Quadibloc said:

>

>

But a possible alternative is that exploding the Stradivarius mystique will lead to the conclusion that there are no real differences in sound quality between violins (at least above the very cheapest ones made carelessly or out of inadequate materials like plywood).

That conclusion won't bring more customers to the door of the quality violin makers of today!

 I use 3 ply 0.8mm thick birch model aircraft plywood for my ribs.  

[palousian]

I suspect that objects made by hereditary craftspeople with a centuries-long tradition of doing the work with the highest-quality materials, and an extremely low labor cost--generally servant-class individuals working for aristocracy or a similar arrangement, whose work has a purpose beyond simply getting paid--produce better quality work than that produced now.  I will await Violadamore's comments on katana, but on carpets... even though the material (mainly wool, also cotton and silk) is mostly the same stuff (as with wood in violins) being used today, the mastery of vegetal dyeing of wool has gradually declined over the last two centuries.  Sure, modern dyes can replicate any color, but... there is nothing like an 18th-c. Salor Turkmen carpet or earlier Safavid "classical" court carpets from Iran (go to the V&A museum in London and get a look at the legendary 16th-c. "Ardebil carpet"), in weaving technique, texture, dyes, and design.  In most respects, handmade oriental carpet weaving was over by the 20th century.  In many ways, modern violin- and bow-making is an exception, since such effort has been made to reproduce the quality of work and materials in an entirely different economic and cultural context.  That's why I'm not sure this argument actually applies to violin- and bow-making--luthiers generally care quite a bit how things were done two centuries ago.  But I can understand why questions still linger about old vs. new.

[martin swan]

Same deal with furniture, metalwork, glass, pottery ...

Industrialisation has offered a very different kind of object, and many skills have died out completely. A technical college training results in very different tool skills from an apprenticeship which starts at the age of 10.

[Danube Fiddler]

6 hours ago, Bruce Tai said:

 

There are many papers that show working memory for timbre only lasts seconds. Here is a list of the ones I consulted:   

McKeown D & Wellsted D (2009) Auditory memory for timbre. J. Exp. Psychol. Hum. Percept. Perform. 35:855-875.

Mercer T & McKeown D (2014) Decay uncovered in nonverbal short-term memory. Psychon. Bull. Rev. 21:128-135.

Cowan N (1984) On short and long auditory stores. Psychol. Bull. 96:341-370.

If we believe in any relevance of these findings ( not read by me), then any comparisons of violins don´t make any more sense.

However to look only at working-memory based soundreception should be misleading, since listeners always have the possibility to compare with sound-images of their longtime-memory. This memory recognizes voices with big easiness after very long times.

So possibly it is impossible for us to compare the sounds of 2 violins directly. However we can easily do indirect comparisons on the way to compare/relate each violin with the nearly infinite number of sounds, stored in our longtime-memory.

 

We should not make it too complicated : A price-related difference of 1 : 100 between a good modern-mean-violin and an even not so very good Strad should easily be able to show even in a not optimal designed (double) blind - study. 

I would ask you : Please design an improved study of comparison - first publish the design - and only then do the study with some very good Strads/Guarneri e.g. of the Chi-Mei-Collection, to which you have connections. 

 

[PASEWICZ]

18 hours ago, Bruce Tai said:

 

 First, working memory for timbre only lasts from a few seconds to tens of seconds in humans, shorter than the time interval required for the player to switch instruments and play the same passage. The timbre memory of the previous instrument may have already decayed when the next violin is being played.

 

 

This is absolutely not true in any situation I have ever been involved in or seen.  Certainly musicians remember timbre and build their careers around being able to reproduce the timbre they are seeking.  Have you never heard a great player switch instruments and sound the same?  I am afraid such pronouncements throw a whole lot of doubt on anything that follows.

[GeorgeH]

Just now, Jerry Pasewicz said:

I am afraid such pronouncements through a whole lot of doubt on anything that follows.

Exactly!