Machine Learning for making better violins?

[szuper_bojler]

Basically title.

I'm a graduate student researcher at a major university interested in the application of Machine Learning and AI in music. However, although there has been a lot of research into music generation, composition etc., not a lot of work has been done with regards to musical instruments. Having played violin for half of my life, I've immediately thought of the intricacies of violin making, but this is also a field that I'm not that I'm not that familiar with, so wanted to gather some opinions here

Is there anything you think Machine Learning/AI can help with making violins? Or any music related application of machine learning that you think would be interesting to explore?

The more specific the better, lemme know! Thanks!

[HoGo]

In not so disytant past there were several discussions on MN where this was mentioned, mostly those relating to CNC machining of parts. Use google search (instead of built in one) to find them. I personally added some contents there.

The main problem with AI/ML is need of LOTs OF DATA. That may be biggest problem if you want to apply that in violin making as most makers' lifetime production is not enough for that. And most makers are not systematic enough to collect measurements and even thos who do likely change the methodology and/or number of taken measurements. For this you'd better get in touch with BIG manufacturer and ask if they were willing to allow you spend time in their workshops doing measurements and evaluating each instrument. This may slow down their production so they may not accept that (unless you persuade them that your research will help them improve their product). Count with months/ years spent in dusty chinese workshops to get the data.

I think I haven't seen an AI tools used for music transcription, yet. (but I admit I haven't looked around too much). In classical music virtually all music is already written but transcribing correctly jazz, blues, bluegrass etc solos and chord progressions can be tedious job that I think would not be hard for AI once trained on large pool of GOOD data.

[Jonathan B]

There is certainly a question about violin making that ML might be able to solve, although as @HoGo says acquiring the necessary data sets would be a daunting task.

Experts seem to have a reasonable consensus about which violins are very good or outstanding, in a world where the vast majority are just good or worse. But they are much less agreed on what physical parameters make a violin excellent, certainly not enough to give current violin makers a template that guarantees high success.

If there were a sufficiently large data set then I am sure ML could identify the most important features that raise a violin from good to outstanding. But it would have to be an exhaustive data set that doesn't exist, one that for a huge number of existing violins contains detailed physical and tonal measurements as well as how experts rate them. I guess the nature of ML means it could work from photographs for at least some measurements - though those don't show plate thicknesses for example - and from the audio files included by some retail listings. However the ratings couldn't come from those same retail sites who for good business reasons only find positive things to say about all the instruments on sale, they would have to be independently assessed. The data set should include generic production violins (Markneukirchen and Mirecourt for example, or modern Chinese and Romanian) with those few of them that happen to be much better than others identified.

The information most easily accessible on the internet wouldn't be that useful, there is not a lot of point putting a lot of work in to find out that violins that were made 300 years ago in Cremona are often considered excellent.

[HoGo]

1 hour ago, Jonathan B said:

The data set should include generic production violins (Markneukirchen and Mirecourt for example, or modern Chinese and Romanian) with those few of them that happen to be much better than others identified.

I don't think that would be possible as you cannot measure properties of wood of already built instruments easily. CT-ing instruments is very expensive and not as precise as direct measurement.

I can see this done in medium sized workshop that produces higher quality instrument (Stentor had separate workshops for different grades) where each piece of wood would be measured for all possibly important parameters using reliable method and all parts would be measured again during production and assembly steps and final product will be tested after stringing and classified it's tone/playability using as simple objective rules as possible.

[Don Noon]

5 hours ago, HoGo said:

IThe main problem with AI/ML is need of LOTs OF DATA.

That's the basic problem, and most of the data is nearly impossible to quantify.  Detailed wood properties (including damping), varnish effects, arching data, thickness maps, etc., and then there's the issue of evaluating the acoustic performance in a meaningful way.  Even for ONE instrument, it's basically hopeless.

[GeorgeH]

AI can learn how to synthesize the voice of a violin from recordings, say sampling Perlman's Soil Stradivarius. With human testing, it could also learn bridge responses the bow strokes that generate specific sound qualities.

From there, it is a matter of measuring the bridge response and using that to synthesize specific tones of the violin the user has selected. The bridge would be made from a synthetic material to avoid variability. The sound would be emitted from speakers in the top of the "violin" or top and back.

So you could dial in whatever violin you wanted to play or dial in your own custom sound.

There are violins that already play synthesized violins from speakers mounted in the instrument, but I don't know if or how AI is being used yet in the sound generation.

Or perhaps AI will be able to design fully synthetic violins. The future is plastics. 

 

 

[Nicholas Lewis]

I did my master's in chemical engineering with data science applications. Lots of cool ideas, and lots of challenges...the lack of data is the big one. Few-shot models might be helpful, maybe in something like varnish making? There's probably lots of models out there about chemical properties that could maybe be used to then extend to small size data samples about violin varnish to get some type of result. The things about varnish we typically think about in violin making are more subjective, but you could try to predict rheological properties or hardness, drying time, etc--more objective criteria that we care about when applying a varnish. 

Another cool idea would be in string manufacturing. There are charts out there like this one that give subjective categorization of how different strings are perceived tonally. I don't know much about string manufacturing, and the different compositions of strings is likely proprietary, but if you could somehow collaborate with a string manufacturer and get some of the manufacturing procedures, you could develop some pretty cool applications. I remember seeing a Youtube video once from a string manufacturer about how they wish they could take a customer request on how they wanted a string to sound, and be able to just go in and tweak the materials and manufacturing parameters to get that exact custom string. So the application would be train a model on existing strings, then see if we could develop a new string for a very specific sound based on a customer request. 

I think the main issue with most things in violin making is the widely variable starting materials--we don't use synthetics for very much, so raw materials are highly variable. So just try to control as much as possible, and then when you publish make sure to explain all the assumptions and caveats, lest anyone think you're trying to make a general rule about all violin making everywhere! Scientific studies are interesting and informative, but my take is that there are so many nuances and subjective opinions that the best way to make a good violin is still a lifetime of intuition. 

Feel free to PM me to bounce around more ideas or details! 

[Shunyata]

1 hour ago, GeorgeH said:

 

AI can learn how to synthesize…

 

Current AI cannot “learn” anything… it can only generalize from prior examples.  

The examples don’t exist for AI to provide violin construction guidance.  No luthier has recorded the data at a detailed enough level, let alone enough examples to support generalization.  

And each violin construction is unique enough that generalization is of limited value anyway (beyond the generalizations that are already incorporated in formal luthier education).

[szuper_bojler]

9 hours ago, HoGo said:

I think I haven't seen an AI tools used for music transcription, yet. (but I admit I haven't looked around too much). In classical music virtually all music is already written but transcribing correctly jazz, blues, bluegrass etc solos and chord progressions can be tedious job that I think would not be hard for AI once trained on large pool of GOOD data.

That's something I've worked on before and there have been a lot of research into this (a major player there is TikTok, believe it or not). This rather technically challenging since it's hard to 1. separate the voices, and 2. get the notes out of the music. You also have to train a model to first identify the tempo, rhythm, key of the piece before you can even try to classify the notes, but you also kinda have to know the notes before you can infer the tempo and rhythm... So far, the state of the art there is limited basically to piano music at this point, even that doesn't work that well. 

10 hours ago, HoGo said:

The main problem with AI/ML is need of LOTs OF DATA. That may be biggest problem if you want to apply that in violin making as most makers' lifetime production is not enough for that. And most makers are not systematic enough to collect measurements and even thos who do likely change the methodology and/or number of taken measurements. For this you'd better get in touch with BIG manufacturer and ask if they were willing to allow you spend time in their workshops doing measurements and evaluating each instrument. This may slow down their production so they may not accept that (unless you persuade them that your research will help them improve their product). Count with months/ years spent in dusty chinese workshops to get the data.

Hmmmm, that might sound like an idea actually. By including their name in a reputable journal, perhaps even the VSA conference, they'll get their fair share of recognition, so that might be reason enough. Most of these workshops have a huge stock as well, so I don't think it'll interrupt their production schedule that much.

In terms of actually measuring an instrument, it'll definitely be challenging. We'll need to develop a systematic way to gather the needed data, thickness maps, bridge response, etc.. Definitely a large time investment though... 

 

[Urban Luthier]

In my day job, I work with workstation hardware that houses the big GPUs to drive AI projects...

One example I can think of that would be practical to execute and useful, is to build an AI model with all the dendrochronology data has been captured by experts. 

If there are any experts on the board who are using ML for dendro-based ID (or even thinking about doing it), please let me know. I may be in a position to help direct you to resources that could help.

[Riccardo964]

18 hours ago, HoGo said:

LOTs OF DATA

good luck dealing with organisations like tarisio and the likes where they protect their data like hell on a bend

[HoGo]

9 hours ago, Riccardo964 said:

good luck dealing with organisations like tarisio and the likes where they protect their data like hell on a bend

I don't think Tarisio has any useful data for AI. Zygmuntowicz, Curtin or Schleske may be much better sources, though probably not for the quantity needed.

I now wonder what parameters (both on input, output side) would seasoned makers call for?

Something like density, speed of sound/stiffness along/across wood of plates. Density and stiffness along for neck wood and side wood. weigth of rib assembly, thickness of ribs, height of ribs, arch heights, thickness of plates at least at upper bout, lower bout, center, basic tap tone frequencies. "Style" and height of arch. Density,weigth and size of fingerboard for the input...

[Don Noon]

There are certainly makers (like Sam Z.) who try to record lots of data on the violins they make.  I do that too.  As yet, nobody seems to have come up with much of anything useful, other than perhaps a tonal character associated with top arch height.  

At the extremes of the data, you can certainly find things correlated with "bad" (95 gram tops, for example), but the broad middle range seems frustratingly murky.

[Deo Lawson]

AI will replace violinists before it gets used to make better violins.

[David Burgess]

2 hours ago, Deo Lawson said:

AI will replace violinists before it gets used to make better violins.

I suspect so.

[Dr. Mark]

If you want your own AI you can pick up something like MATLAB and get packaged AI software, or build your own - that's kind of the easy part.

[szuper_bojler]

17 minutes ago, Dr. Mark said:

If you want your own AI you can pick up something like MATLAB and get packaged AI software, or build your own - that's kind of the easy part.

That's the plan, and is in fact the easy part  I'm familiar with the tools but I'm trying to find interesting ideas to apply them to, which is why I made this post.

[Chris Anderson, PhD]

On 12/5/2024 at 2:28 PM, Urban Luthier said:

In my day job, I work with workstation hardware that houses the big GPUs to drive AI projects...

One example I can think of that would be practical to execute and useful, is to build an AI model with all the dendrochronology data has been captured by experts. 

If there are any experts on the board who are using ML for dendro-based ID (or even thinking about doing it), please let me know. I may be in a position to help direct you to resources that could help.

I thought of doing this myself (as a fellow database person as a day job) - and eventually found that many of the 'critical' datasets for particular tree ring data is actually in the hands of private collectors!  (who want to sell you their personal services for dating the wood of violins... since I can't think of any other reason why ONLY northern Italian Spruce trees would have privatized and proprietary tree ring data...).  

My comments are not meant to be a discouragement - but merely setting expectations for the challenges I found when I first looked into it.  I agree that the tree ring application would be particularly good for a AI/Neural network kind of solution.  There is free software to perform a least squares analysis that I found - but the user must provide the datasets and the measurements.  The other fun challenge is 'how do you know where that spruce tree came from'... because tree ring data varies by species and location.

-Chris 

[GeorgeH]

7 minutes ago, Chris Anderson, PhD said:

I agree that the tree ring application would be particularly good for a AI/Neural network kind of solution. 

How would tree ring data possibly help train AI to reproducibly make "good" violins?

What would you try to correlate it to (even if you could)?

[Urban Luthier]

On 12/4/2024 at 9:54 PM, szuper_bojler said:

...Or any music related application of machine learning that you think would be interesting to explore?

falls under this category form the original post.

As someone who is bombarded AI stuff in my day job, MHO is that there is very little that AI can do to help makers make a better violin. However there are many use cases for violin connoisseurship. Dendro is one, Image and pattern recognition for instrument identification is another. Challenge here (as noted above) is that much of the data is in private hands and I believe the associated meta data is neither consistency captured, or accurate enough to create an AI model to work from. Measurements and images captured from CT scans would be a great place to start but I doubt there is enough to work with and we still have the same issue of the data sitting in private hands.

Lots of good stuff out there like Nvidia's ai workbench. Challenge is one needs to create the model first that means two things 1) good data 2) a lot of compute horsepower

[Jonathan B]

Exactly. The concept of dendrochronology is very simple, and lends itself to computer analysis. Those who sell dendrochronology services are selling access to their private database as much as their analytical ability.

That is the thing with "AI" which of course isn't actually intelligent. There was a lot of cleverness in designing it, the way it can analyse databases without being provided with a pre-written algorithm as long as it has a huge number of examples of actual intelligent human responses to similar problems to copy (there were Nobel prizes won for the way it does that, this year). But that having been done the requirement is for a huge database to train it. Which is why there is the issue of people whose business is the data capable of analysis (e.g. news websites, books which have been digitised without paying royalties) wanting to block the automated data collection done by companies hoping to sell their AI products.

So if an AI system was given Peter Ratcliff's database* including all his considered opinions to date, given a new piece of data it would have a high success rate with violins in coming to the same conclusion Peter Ratcliff would. Without that proprietary data all it could do is what open access data (not focussed on violins) provides - which I imagine would be reasonable on dating something, probably to the nearest decade. What it wouldn't be able to do is relate one violin to another previously analysed, with the probability the wood came from the same small geographical region (so probably same tonewood supplier) or even the same tree, which judging by some of the results quoted on Ratcliff's website is the professional service he is charging for.

(Apologies to Mr Ratcliff if I have misrepresented him).

Back to the OP, to use AI to design a "better" violin it would need both a large database of actual violins which have been measured in meticulous detail, and evaluation by intelligent humans of each of those violins' plying qualities. No such database already exists available to exploit, and if such an AI system was created its value would not be in its use of the now well-established "AI" process of utlising data but in the database which would have to be assembled for the purpose at considerable expense.

[*Other dendrochronologists are available].

[Don Noon]

28 minutes ago, Jonathan B said:

Back to the OP, to use AI to design a "better" violin it would need both a large database of actual violins which have been measured in meticulous detail, and evaluation by intelligent humans of each of those violins' playing qualities.

That last bit about player evaluations introduces a huge pile of variables into the problem, which is necessary if you want to make a "better" violin.

Perhaps it would be more feasible (or slightly less impossible) to have AI help create violins that "sound a particular way".  Using CT scans for dimensional and density data, and then a repeatable method of obtaining frequency response, at least you could get rid of the pesky humans from the data.  Then you might get a recipe for the sound of the Cannone, most of which actually we have now:

• Shape and arching like the Cannone
• Graduations like the Cannone
• Wood densities like the Cannone
• Assemble and allow to age for 300 years

Oh, well... AI might tell you what you can and can't get.  Or it might need more data about the wood acoustical properties, which are nearly impossible to get on some instruments.  Or it might just say that the answer is 42.

[Andreas Preuss]

On 12/5/2024 at 3:54 AM, szuper_bojler said:

Is there anything you think Machine Learning/AI can help with making violins? Or any music related application of machine learning that you think would be interesting to explore?

If I had knowledge in this field I’d probably analyze the effectiveness of working procedures rather than trying to use it for making better instruments.  

[HoGo]

I don't think there is much new AI can do on Dendrochronology. The statistical methods are straight and simple without any need for AI. The good database of dated tree ring spacings is critical and building it is a lifelong tour so noone will give it out for free.

AI is good at crunching different problems.

[uguntde]

I see more and more AI used in science. But it only works where there was lots of data to train AI systems. In the case of violins this is not the case, in fact, there is close to nothing publicly available. On the other hand, if violin makers gathered to collect spectrograms and other data on lots of violins together with plate thickness etc and resulting sound quality, good models could be trained. You would need such data for at least a few thousand violins.