Strad vs Modern - The results


Recommended Posts

  • Replies 278
  • Created
  • Last Reply

Top Posters In This Topic

Quite so.

As I understand it, the experimenter chose the violins behind a screen and handed them to an assistant in the room, who placed them on the bed. So for this to be anywhere near approaching blind we have to assume that the assistant was unaware of the identity of the violins. Was that so? Were they wearing goggles as well? Were they totally ignorant about violins?

Also, the fact that the experimenter knew the identity of the violins that were handed to the assistant will introduce experimenter effect. The purpose of having a double-blind arrangement is to eliminate experimenter effect, and this does not seem to have been achieved here.

Another point that has been mentioned is the lack of controls. Attempts were made to disguise the violins' identities - welder's goggles and scent under the chinrests - but did anyone test to see if these measures were effective?

One of the first steps I would have taken, had I been conducting the experiment, would have been to have the players attempt to guess the identity simply by holding and feeling each violin.

If there were no statistical significance in the 'hit rate' then one might assume that the disguising techniques had been successful. But as far as I know this was not done.

If any of the modern violins had a full varnish and nice sharp corners, identification could have been pretty easy.

But these faults pale into insignificance with my main objection, which is the fact that Joseph Curtin was one of the authors. In the paper it clearly states that the authors have no conflict of interest. But as a living violin maker it is surely in Curtin's interest to have modern violins perform well in the experiment. How is this not a conflict of interest?

Therefore the entire experiment falls at the first hurdle and all further discussion is pointless.

Andrew

I don't know all the details of the experiment. To eliminate "experimenter's effect" it would be necessary to eliminate any interaction between the experimenter and the subjects. This could have been done simply by having the experimenter put the violins on the bed when the subject was out of the room and has no contact, visual or aural, with the experimenter. However, completely flawless double-blind experiments are extremely difficult to perform. Even if you decide that this was actually only an observational study rather than a true controlled experiment the results are quite interesting. I discount the possible unconscious bias of Joseph Curtin. His instruments are good enough that he has no need to deliberately bias the results in favor of newer instruments. Besides, the great old instruments are effectively unavailable to 99+% of players. I would think that the results of this "test" and others, slightly flawed though they may be, would be reassuring to players saying, in effect, that they don't have to have a Strad or del Gesu to get a job in a good orchestra or chamber group. But we knew that. There is certainly a publicity aspect to this situation, witness the excitement in the media.

As for my personal opinion, I think that before too long the great old Cremonese instruments will effectively leave the musical arena and become items for extremely wealthy collectors only. I think players can serve the music just as well with less famous instruments.

Link to post
Share on other sites

over and over before, people have said not to trust youre own under ear assesement of a violin, as projection can never be gauged by the player if theyre playing it, in this case the players chose modern about 2/1 but when the same instruments were tested before the assembled audience the strad won 3/1 or 75 % of the time, obviously the audience is in a better position to judge what a violin sounds like in a concert hall than the player in a hotel room.

i dont think any serious soloist ready to plunk down thousands or hundreds of thousand of dollars would be making their final judgement on under ear sound only, obviously they would want to go to the back of the concert hall and hear someone else play the violin,

this is not an example of how much better modern violins are getting, but rather how much stupider the public gets when someone tells them what they want to hear, all you need is 30,000usd modern violin and youre as good as stradivarius, which is rubbish.

these new vs old studies go all the way back to vuillaume, i think, they all have one thing in common, the new violin winners are never considered winners 50 or 100 yrs after the study......

Link to post
Share on other sites

Adding Chinese violins into the experiment has been done by Saitis and collaborators at the McGill university along with Fritz: http://www.music.mcgill.ca/~harry/pdfs/000246.pdf

Dear Anders,

How do you find these type of violin research related articles? I miss the old Cat Gut Society Journal which used to list the new publications.

Thanks, Marty

Link to post
Share on other sites

The Dunning & Kruger paper I cited earlier gives a good template for conducting experiments of this sort. To minimize instrument selection bias, you start with 2 groups of people.

Group A consists of career soloists. You have them rank instruments (new, old, Cremonese, Chinese, whatever). You then present this selection to Group B (which doesn't have anyone from group A), and and have them go through the ranking process. If there is a big discrepancy in rankings between group A and group B, go figure out the reasons for that discrepancy.

Link to post
Share on other sites

But these faults pale into insignificance with my main objection, which is the fact that Joseph Curtin was one of the authors. In the paper it clearly states that the authors have no conflict of interest. But as a living violin maker it is surely in Curtin's interest to have modern violins perform well in the experiment. How is this not a conflict of interest?

Andrew,

I don't know whether Joseph Curtin had a bias here, one way or another, about which results he might prefer. While we can each speculate, none of us can actually know.

I realise this is your point. But surely the material matter of concern is: was he in a position to sway the results one way or another? If not, isn't the point academic?

I haven't read anything so far that indicates loaded dice to me. And I can't immediately see how Curtin would, even if he wanted to. If this is your feeling, can you suggest how this might be possible?

Cheers,

Ed.

Link to post
Share on other sites

In general terms (and without questioning for a moment the integrity of the particular journalist involved) it could be reasonably argued that the journalist might be biased towards the more newsworthy outcome. This would lead to problems if the "blinding" procedures were not totally effective (and I personally have doubts that they were).

Being editor of The Strad hardly makes me Carl Bernstein, does it? In fact, I was made to sign a confidentiality agreement when the tests were done in September 2010 that meant I couldn’t even mention the experiment to anyone. I was only released from that when the research came out and I didn’t even know that was going to happen at the time: I only asked when I saw John Soloninka had written it up in his blog. In that respect, my experience was unsullied by what might be perceived as journalistic motivation. I think it’s fair game to question my participation, as I wrote in my blog, as it is that of any of the players, but not because I edit The Strad!

Ariane

Link to post
Share on other sites

Being editor of The Strad hardly makes me Carl Bernstein, does it? In fact, I was made to sign a confidentiality agreement when the tests were done in September 2010 that meant I couldn’t even mention the experiment to anyone. I was only released from that when the research came out and I didn’t even know that was going to happen at the time: I only asked when I saw John Soloninka had written it up in his blog. In that respect, my experience was unsullied by what might be perceived as journalistic motivation. I think it’s fair game to question my participation, as I wrote in my blog, as it is that of any of the players, but not because I edit The Strad!

Ariane

I'm sorry, but it IS fair to question your participation (as well as Laurie Niles, so don't feel singled out) precisely because you're the editor of "The Strad."

Your editorial space in "The Strad" is not unlimited. Presumably the Strad would also like to publish articles that are timely. You also have to craft and choose appropriate headlines.

Embargoes are the norm in academic publishing, so we are well aware that you all had to sign a non-disclosure agreement. But the values of scholarly research are different from that of journalists and bloggers.

Link to post
Share on other sites

I'm sorry, but it IS fair to question your participation (as well as Laurie Niles, so don't feel singled out) precisely because you're the editor of "The Strad."

Your editorial space in "The Strad" is not unlimited. Presumably the Strad would also like to publish articles that are timely. You also have to craft and appropriate headlines.

Embargoes are the norm in academic publishing, so we are well aware that you all had to sign a non-disclosure agreement. But the values of scholarly research are different from that of journalists and bloggers.

And I'm sorry, because I still genuinely don't understand what I'm specifically being accused of!

Link to post
Share on other sites

Dear Anders,

How do you find these type of violin research related articles? I miss the old Cat Gut Society Journal which used to list the new publications.

Thanks, Marty

Dear Marty,

By googling for it. :-) I miss the CAS Journal too! We have a possibility to use the VSA Papers, but everything is slowed down quite much there now of several reasons. The volumes are much larger than the CASJ. A possibility is to use the online Savart Journal: http://savartjournal.org/index.php/sj

Best regards,

Anders

Link to post
Share on other sites

Being editor of The Strad hardly makes me Carl Bernstein, does it? In fact, I was made to sign a confidentiality agreement when the tests were done in September 2010 that meant I couldn’t even mention the experiment to anyone. I was only released from that when the research came out and I didn’t even know that was going to happen at the time: I only asked when I saw John Soloninka had written it up in his blog. In that respect, my experience was unsullied by what might be perceived as journalistic motivation. I think it’s fair game to question my participation, as I wrote in my blog, as it is that of any of the players, but not because I edit The Strad!

Ariane

Ariane, I thought I went to great pains to make it clear that my post was not directed at you but at the general principle that "having a journalist on the panel couldn't possibly be relevant" (I paraphrase).

Link to post
Share on other sites

In that respect, my experience was unsullied by what might be perceived as journalistic motivation. I think it’s fair game to question my participation, as I wrote in my blog, as it is that of any of the players, but not because I edit The Strad!

Ariane

In the SHORT TERM, yes. Long term, we can't possibly know.

Link to post
Share on other sites

Some of you are now arguing that the test is now flawed because the subjects could have identified the instruments by sight, smell, or feel, in spite of all the precautions. Some of you are even arguing that the subjects could have cheated to defeat the blind.

OK, let's assume they did. Let's assume they broke all the rules, took off their goggles, and examined the instruments carefully. Make any assumptions you want about how they could have cheated to identify the instruments.

Then what do you make of the fact that they still couldn't tell the difference?

Link to post
Share on other sites

And I'm sorry, because I still genuinely don't understand what I'm specifically being accused of!

You might not feel you have done anything wrong. And I want to make clear that I'm not accusing of doing anything wrong in this specific instance.

But your participation immediately raises questions of Fritz' scholarly independence and "The Strad's" journalistic independence as far as your readership is concerned.

Suppose you're the editor of a technology periodical. Manufacturers send you products (for free) so you'll review them. Are then you then functioning as advertiser for those manufacturers or independent reviewer?

Link to post
Share on other sites

But your participation immediately raises questions of Fritz' scholarly independence and "The Strad's" journalistic independence as far as your readership is concerned.

I've read and re-read this statement a few times now. In all honesty and all humility, I have no idea what you're driving at. Could you clarify please?

Ed.

Link to post
Share on other sites

Some of you are now arguing that the test is now flawed because the subjects could have identified the instruments by sight, smell, or feel, in spite of all the precautions. Some of you are even arguing that the subjects could have cheated to defeat the blind.

OK, let's assume they did. Let's assume they broke all the rules, took off their goggles, and examined the instruments carefully. Make any assumptions you want about how they could have cheated to identify the instruments.

How then do you explain the fact that they still couldn't tell the difference?

That's the subject of another study...

Link to post
Share on other sites

I did read the FAQ twice before my first post. I see no listing of anything about what music, etude, scale, or anything else was used by any player during his/her time with the violins.

I don't see the relevance of your objection. But in any case, the players were instructed to play for one minute, until a bell rang. In the second part of the test they were allowed to play any or all of the violins in any order, for 20 minutes. In other words, they played whatever they wanted. You will find the instructions to participants in the supporting information, which is attached to the paper. In case you missed the paper, it's here: http://www.lam.jussieu.fr/Membres/Fritz/HomePage/Indianapolis_paper.html.

Link to post
Share on other sites

That's the subject of another study...

I meant to write "What do you make of the fact that they still couldn't tell the difference?".

The argument made was that they may have been able to detect the old instruments by touch, or even by cheating, and this knowledge may have affected the results. This is a hard argument to make since a tally of the comments indicates that they actually could not tell the difference between old and new.

Link to post
Share on other sites

Adding Chinese violins into the experiment has been done by Saitis and collaborators at the McGill university along with Fritz: http://www.music.mcg...pdfs/000246.pdf But the issue here is the consistency of the players, not the violins per se. But they do have the data if they would like to share it.

I do not think we will see Fan Tao or Joe Curtin including a chinese fiddle in such a test.

I think there might be some answers to certain questions modern makers do not want to hear or see.

Origin country is not a particlularly good description of violin sound nor playability to what I know. Wood is wood.

Although the experiments may have already been done, what I have noticed is a dramatically marked improvement in quality among well made Chinese violins in the past year alone, and I'm not talking about VSA winners. So I'd be curious to see double blind experiments done within another year or two and see a rather unsettling truth revealed and mythology de-bunked. Let's face it: if Strad and Guarneri are the emperor's clothes, modern makers are too. At least Strad and Guarneri have antique and art value.

Inexpensive Chinese fiddles are sounding and playing fantastic and are made as well as anything out there, they are good enough player tools to be used in orchestra and in some cases solo. You're right absolutely: wood is wood. When I say "Chinese" violins it is merely because that is where the bulk of mass-produced fiddles happens to be at present. Perhaps "India" or "Congo" or "Tasmania" will catch on next, it is totally irrelevant. Point is - the secret (which benefits mostly the player) is that you simply don't have to shell out $10,000 or $20,000 or upwards of $50,000 and be on a waiting list for a really fine modern violin anymore. Not for sound, playability, or investment "potential" (which contemporaries clearly don't have, based on auction results where they go for a fraction of the waiting list price; not talking about a couple freak exceptions based on who played them). Both sides - dealers of old and makers of new are not science-driven; they are profit-driven, which is understandable. We all have to eat. Some prefer to eat in a castle near Vienna. But don't get comfortable; a paradigm shift is in process. The Fritz thing will be seen as a part of it.

You bet, Anders, there are some answers to certain questions modern makers do not want to hear or see - like the fact that their house of cards is about to collapse.

It's called globalization.

Link to post
Share on other sites

I don't know whether Joseph Curtin had a bias here, one way or another, about which results he might prefer. While we can each speculate, none of us can actually know.

He may not *consciously* know, either.

But surely the material matter of concern is: was he in a position to sway the results one way or another? If not, isn't the point academic?

He is in a position to do that because he was one of the authors. If it were not possible for an author to sway the write-up, conclusions, etc, why does the journal have the standard clause stating that the authors have stated that there is no conflict of interest? Note that I am not saying that he swayed the results (as you wrote) - that would imply fabrication.

Was Curtin also partly responsible for the poor design of this experiment? An experiment of this nature could easily be designed - consciously or otherwise - to provide the results that are wanted.

Andrew

Link to post
Share on other sites

But your participation immediately raises questions of Fritz' scholarly independence and "The Strad's" journalistic independence as far as your readership is concerned.

Suppose you're the editor of a technology periodical. Manufacturers send you products (for free) so you'll review them. Are then you then functioning as advertiser for those manufacturers or independent reviewer?

.

Not for me it doesn't.

I just pursued my copy of the Plowden Strad...It seems to have a pretty equal representation of new VS old violin advertizements.Most the pictures are of old instruments.With 10 pages devoted specifically to old work,not including the Sacconii varnish article, VS six pages focusing on new work ( three of those involved the proper cutting of wood)thanks for that one..this suggests to me an unbiased editorial,Or if any thing, a bias toward the old....now IF I saw full page adds for Claudia Fritz and J. Curtain , on the cover sheets,I might think something fishy,but really the Strad has done SO much work to positively highlight old works,never demeaning of them.I don't understand the assertion of a positive bias toward modern work.

If any of the testers( or editors) would have received anything more than a "thank you" for participating I could understand more, but I think that's all they got of the deal....and to play old and fine violins ,that would be enough to get me there.....even if I couldn't tell the difference.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    No registered users viewing this page.