Sign in to follow this  
not telling

Notes on VSA winners

Recommended Posts

I bet some of them are real cigar boxes.  Here's something you can do.  Say you have as scores 2,8 and 0,10 and 5,5.  You might decide to penalize variance because it means the bow is crazy or something.  Especially the 0,10 bow.  You want to reward high marks and consistent marks.  So you take the average of the scores which in the example is 5.  To the average you add 1 divided by 1 plus the standard deviation of the scores.  The first score will then be 5.191 and the second one 5.124  and the third one 6. 

Share this post


Link to post
Share on other sites

Interesting that in the bowmaking competition, it's the bow-maker judges that grade the playing qualities, not players as in the tone competition.

 

Let me make sure I understand this, because I was under the impression that  bows were not judged for playing at all, at least in the past.  Am I misinformed?  I'm happy to stand corrected for my mistaken post earlier if I was wrong.  

 

Even if it's only the makers judging, I think that's better than nothing.

Share this post


Link to post
Share on other sites

I got my number and checked my score. The violin I entered was my own model, the experimental I just finished. I like the tone of this fiddle a lot. A lot more than two of the judges. One judge scored it good enough for the second round, but not the other two. Probably too dark. The workmanship was never going to be a consideration as it was just a experiment. Anyway the fiddle was great fun to make and enter. Always next time. I know I'm going in the right direction with this model, I just need to work backwards with some of the exaggerations.

Share this post


Link to post
Share on other sites

Let me make sure I understand this, because I was under the impression that bows were not judged for playing at all, at least in the past. Am I misinformed? I'm happy to stand corrected for my mistaken post earlier if I was wrong.

Even if it's only the makers judging, I think that's better than nothing.

Please click the link to the rules above, they will explain more.

Share this post


Link to post
Share on other sites

The VSA worksmanship judging puts numerical values on a subjective impressions of several features: Setup, Varnish, Modeling, Scroll, Construction, and Overall impression.  The combined numerical scores of the three judges are used for ranking the entrees and for picking award winners.  Participants can learn something to improve their violin making.

 

It would be also be helpful if a similar evaluation method was used for the tone judging.  Numerical values could also be given to subjective impressions of several features such as Projection, Note evenness, Playability, Whatever and Overall impression.

 

In both cases all instruments should be evaluated. Finding out you didn't get past the first cut-off round isn't really very informative for many people.

Share this post


Link to post
Share on other sites

Player judging of bow performance has been tried, but as I recall, consistency and repeatability of evaluations was so poor, that it was abandoned. What does a competition do if a player judge evaluates a bow twice, giving it an eight the first time, and a two the second time, and patterns like that seem to repeat over many judges and bows?

 

Very interesting! I guess then that this kind of inconsistency hasn't shown up in the instrument testing process. I wonder how much of that is due to the relative facility of visually recognizing an instrument one has already tried, whereas a player who isn't a serious bow nut might be totally lost in a sea of bows. I wonder if in the end the "all over the place" bow results weren't somehow more objective  :)

 

Sorry, I don't want to belittle the hard work and dedication of all those involved in this endeavor! For many years, my main professional fiddle was a modern from a maker who won a gold medal for tone at the VSA competition, so  at some level I must have been on the same wavelength as the judges at that competition! I'm just very sceptical about players' judgements of instruments or bows. I have yet to see one colleague (and I fear I'm not immune myself) who can judge a violin or bow independently of appearance, authorship or value. 

Share this post


Link to post
Share on other sites

Since there has been much talk on the result of tone award of this year’s VSA competition, I decided to do a simple analysis based on the jurors’ scores. The Violin Tone score sheet obtained here contains the three jurors’ scores in the first round (preliminary run) and the second run (evaluation run). No scores are available for the third round (medal run); I guess no one entered the medal run.

 

I want to show two things in the analysis: (1) The extent of score similarity between the jurors (using results of round 2). (2) The juror’s scoring consistency between the two rounds.

 

Instead of naming names, I’ll use Juror1, Juror2, and Juror3 instead. I dropped observations to which the status is not marked as “CLEAR”, and dropped one additional observation for which one of the jurors did not give mark (recused him/herself, perhaps). As a result, there are 192 observations/violins on the first round, and 51 on the 2nd run.

 

Part I: To show the extent of score similarity between the jurors, I produce tables of frequency counts between pairs of jurors, using results of the 2nd run evaluation. Note that only scores of 1, 2, and 3 are given in the 2nd run. In the first panel of the graph below, the figure “17” indicates that 17 violins received score=1 from both Juror1 and Juror2. The figure “5” indicates that 5 violins received score=3 from Juror1 and score=2 from Juror2.

post-24616-0-73198300-1480095190_thumb.png

 

Some observations: (1) Juror1 and Juror2 each pick 8 and 9 violins, respectively, to confer their highest mark (score=3). Juror3, however, only offered 2 violins the credit. Could the stinginess of Juror3 contribute to the low number of tone award this year? (2) The similarity between Juror1’s and Juror2’s scores appears to be the lowest among the pairs. Note also that Juror2 gave score=3 to 9 violins, but 8 of them received only score=1 from Juror1. The difference in taste is not so drastic between other pairs of jurors.

 

Part II: To show the juror’s consistency between the two rounds, I counted the number of cases in which a juror gave score=0 to a violin in the 1st round (not recommended to advance) but then gave the same violin score=2 or score=3 in the 2nd round. These are the cases of inconsistency because, had the juror been consistent, the violin s/he didn’t like in the first round (score=0) should have received from him/her the lowest possible score (score=1) in the 2nd round.

 

Here is the result: Among the 51 violins that entered the 2nd round, Juror1 has only 1 case of inconsistency (1.96%), Juror2 has 7 cases (13.73%), and Juror3 has 2 cases (3.92%) of inconsistency.

 

----- Edit: The following is an improved version of calculating the inconsistency rate. ----------------

  • For the 51 violins that entered the 2nd round, Juror1 gave 6 of them score=0 in the first round and subsequently gave 1 of them score>1 in the 2nd round. The inconsistency rate is 1/6 = 16%.
  • Juror2 gave 19 out of the 51 violins score=0 in the first round and then gave 7 of them score>1 in the 2nd round. The inconsistency rate is 7/19=37%.
  • Juror3 gave 9 out of the 51 violins score=0 in the first round and then gave 2 of them score>1 in the 2nd round. The inconsistency rate is 2/9=22%.

-----------------------------------------------------------------------------------------

 

What's your take?

Edited by HJW

Share this post


Link to post
Share on other sites

What's your take?

 

Two judges play in the same quartet and the third is a highly rated soloist.

 

Perhaps you should redo the analyses with this in mind.

Share this post


Link to post
Share on other sites

Two judges play in the same quartet and the third is a highly rated soloist.

 

Perhaps you should redo the analyses with this in mind.

 

Yes, I am aware of their status. However, their opinions/scores are equally weighted in the competition, so I could not see how the analysis can or should be done differently. Suggestions are welcome. On the other hand, could readers identify the soloist from the analysis (or the soloist vs. the quartet players)? I think this is an interesting question. From the hindsight, I think this is not difficult.

Share this post


Link to post
Share on other sites

 I guess quartet vs. soloist would make it 2:1. Perhaps it would have been more sensible to have 2 soloists + 2 quartet members (from different quartets) or 3 soloists. 

 

 Generally, I would think that any first-round inconsistencies would turn into an orderly hierarchy during the second round. For soloists, it might be sensible to have even 3 rounds. Or tell all jurors to rate the work in absolute terms instead of 'against each other'. A lot of bows will get bad scores, though.

 

 I think someone pointed out earlier, but quartet is more about blending in whereas soloist is more about standing out. So that skews orientations quite a bit. And someone who has not had serious thoughts about what makes a good bow and had commensurate experiences will not make a good judge.

Share this post


Link to post
Share on other sites

What's your take?

I think it's possible to make a violin that will please everybody and therefore win all competitions, and customers (which are a subset of everybody).

It's got to be utterly smooth, balanced from string to string, amplitude of course, and color as well.  Got to be brilliant but with depth.  Cannot be overly bright but cannot be hollow.  Easy!  How do you do it?  No idea, but it's not my profession.  I would think it has to start with the right sound in mind.  It took a long time as a player to figure out what a good sound was.  Can't conceive how a maker could be better than his playing, really.  But players and makers are different personalities...so...  Or maybe it is as simple as following the long established rules and plans. without the need to hear anything, and the one who does that best takes home the gold.  That's probably it.  Uncovering all the rules and then following them to a T. 

Share this post


Link to post
Share on other sites

I'm curious, did anyone play the double gold winner as well as other violins in the competition. If so what was your thoughts. Also any thoughts on the new makers room fiddles. I played those several times, some of those were pretty amazing.

Share this post


Link to post
Share on other sites

Yes, I am aware of their status. However, their opinions/scores are equally weighted in the competition, so I could not see how the analysis can or should be done differently.

 

The 2 players in the quartet may have developed similar tastes in the 'sound' they are looking for, so 'quartet membership' would be a covariate that needs to be adjusted for, using for example ANCOVA. 

Share this post


Link to post
Share on other sites

I'm curious, did anyone play the double gold winner as well as other violins in the competition. If so what was your thoughts. Also any thoughts on the new makers room fiddles.

 

I played the double gold winner as well as many others, including the contemporary makers' exhibit room.

 

I think I'm not a good enough violinist to really judge what is a desirable violin... at the top levels.  I can tell clunkers from decent by myself, but beyond that I need to hear a good player put them through the wringer to bring out the full capabilities (or lack of them).  Even then, it's not always clear; they all sound a bit different.

Share this post


Link to post
Share on other sites

The 2 players in the quartet may have developed similar tastes in the 'sound' they are looking for...

 

That doesn't seem to be reflected in the scoring. If you look at judge #3, the only two violins she gave 3 points to were scored low by the other judges (1-1 and 1-2). If you look at judge #2, four of the nine violins that she gave 3 points to were given 1's by the other judges. It seems to me that the judges had individual tastes that didn't really overlap much. Goes to show how subjective this all is.

Share this post


Link to post
Share on other sites

 Goes to show how subjective this all is.

 

Yep and full of potential hidden confounders.

 

Here's another: Ms Fullard plays a Cannone copy made by Vuillaume, while Ms Sato plays a Politi (I think that's what was said).  Anyway from where I was sitting at the VSA recital, the Cannone copy sounded like a cannon, while the Politi was hardly audible when carrying the melody.

Share this post


Link to post
Share on other sites

The 2 players in the quartet may have developed similar tastes in the 'sound' they are looking for, so 'quartet membership' would be a covariate that needs to be adjusted for, using for example ANCOVA. 

Thank you, I see what you meant. The effect of quartet-taste confounder may not be correctly identified in this small setting though (only three jurors). For instance, the quartet membership cannot be separately identified from, say, tone judge experience, with Oliviera being a much more experienced tone judge than the two quartet players. So the differences between the soloist's and the quartet players' scores may be due to experience rather than taste. To some extent, gender is also confounded with the membership in this case. More interesting analysis can be done if we have data on multiple contests where different sets of judges were employed.

If we want to have a serious analysis, I am not sure what kind of model we could use. What is the outcome/dependent variable? It can't be the total score of a violin since it is just a sum of the jurors' scores. Ideally, if we have data on the characteristics of the violin tone: warm, bright, or even the violin's various modes, then we can see how a certain judge prefers certain types of tones. (BTW, ANCOVA cannot be helpful here because of the required distribution assumption.)

 

It seems to me that the judges had individual tastes that didn't really overlap much. Goes to show how subjective this all is.

I agree. No matter how I attempted to generalize the choice/taste between the soloist vs. the quarter players using the data, I could always find angles to contradict.

Share this post


Link to post
Share on other sites

No matter how I attempted to generalize the choice/taste between the soloist vs. the quarter players using the data, I could always find angles to contradict.

 

Many times I'll pick up one of my violins and be somewhat surprised at how good I think it is.  And there are other times I'll pick up the same violin, and say "blecch".  Similarly, in the VMAAI tone competition, I'll have quite different opinions on the same instrument from one round to the next... and the same thing can be seen in the judges' scores between rounds on the same instrument.  There's just a lot of random variation in tone judging that can't be eliminated, and we need to accept that.

Share this post


Link to post
Share on other sites

I have seen some large fiddle contests where they have five judges. They take the high score and the low score and through them out. They use the middle three. Weather that sort of score keeping would help here, I don't know. I just think something was wrong this year.

Share this post


Link to post
Share on other sites

Many times I'll pick up one of my violins and be somewhat surprised at how good I think it is.  And there are other times I'll pick up the same violin, and say "blecch".  Similarly, in the VMAAI tone competition, I'll have quite different opinions on the same instrument from one round to the next... and the same thing can be seen in the judges' scores between rounds on the same instrument.  There's just a lot of random variation in tone judging that can't be eliminated, and we need to accept that.

I don't think becoming a fine player first is the most efficient track toward becoming a fine violin maker.  Same with electronic testing.  The two are similar ideas actually, in that you evolve ideals and then your job is to somehow impose that on the violin you make.   There is a thread here somewhere where somebody makes a post that tells what the most efficient track IS, I think.  It is being an apprentice.  It is having the best maker standing over you simply saying do this and do that.  Pretty damn simple.  That's what the workshops and etc. are about, but I sense there is a lot of holding back of information, "jealously guarded secrets" if you will.  One time I made a thread titled something like "Mysterious secrets of modern makers" and nothing came of it.  But then not being a violin maker, I even wonder if special information even exists -- for all I know all you need to do is get hold of a good plan and follow it really well.  Just an exercise in fine cabinet making.  I haven't even learned truth or falsehood of that particular secret here yet :)

Share this post


Link to post
Share on other sites

 I just think something was wrong this year.

 

 

As far as I can tell, there was not a thing wrong this year in the least.  A competition occurs in a brief slice in time; at this time, and under these conditions, these were the results.

 

Ultimately the certificate and medal recommendations have to be unanimous.  The fact that the judges have varying opinions only enhances the success of the gold medal winner.

Share this post


Link to post
Share on other sites

I don't think becoming a fine player first is the most efficient track toward becoming a fine violin maker.  Same with electronic testing.  The two are similar ideas actually, in that you evolve ideals and then your job is to somehow impose that on the violin you make.   There is a thread here somewhere where somebody makes a post that tells what the most efficient track IS, I think.  It is being an apprentice.  It is having the best maker standing over you simply saying do this and do that.  Pretty damn simple.  That's what the workshops and etc. are about, but I sense there is a lot of holding back of information, "jealously guarded secrets" if you will.  One time I made a thread titled something like "Mysterious secrets of modern makers" and nothing came of it.  But then not being a violin maker, I even wonder if special information even exists -- for all I know all you need to do is get hold of a good plan and follow it really well.  

 There is a book titled 'The "Secrets" of Stradivari'.  Along with many of my colleagues, I have always found this title amusing.  It is making reference to there being no real "Secrets", and in fact the only things people view as "Secrets" are learning from those that came before, getting really good, and working your butt off for 80 years.

Share this post


Link to post
Share on other sites

^I meant secrets in the same sense, I think.  Just information that isn't readily available.  Possibly but not necessarily guarded.  Which a master would drill into an apprentice as part of daily life.  And which could best or only be learned that way.

It's either that or there is no secret at all -- just get a good plan and follow it to the letter and you're Stradivari.

Share this post


Link to post
Share on other sites

-- just get a good plan and follow it to the letter and you're Stradivari.

 

-- just play all the notes written on the page, and you're Paganini.

 

It has been my observation that VSA tone winners (and even moreso in the workmanship winners) have been at it for many years, and I am not aware of any meticulous beginners that just "followed the plan" and came out a winner.   To me, this indicates that there are subtle details that are not in any plans and make the difference between good and the best.

 

Let me see... what does "the plan" say for varnishing?  :rolleyes:

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.