Realtime GPU Audio 157
CowboyRobot writes "Two researchers at San Francisco State University has successfully implemented hardware acceleration for realtime audio using graphics processing units (GPUs). 'Suppose you are simulating a metallic plate to generate gong or cymbal-like sounds. By changing the surface area for the same object, you can generate sound corresponding to cymbals or gongs of different sizes. Using the same model, you may also vary the way in which you excite the metallic plate — to generate sounds that result from hitting the plate with a soft mallet, a hard drumstick, or from bowing. By changing these parameters, you may even simulate nonexistent materials or physically impossible geometries or excitation methods. There are various approaches to physical modeling sound synthesis. One such approach, studied extensively by Stefan Bilbao, uses the finite difference approximation to simulate the vibrations of plates and membranes. The finite difference simulation produces realistic and dynamic sounds (examples can be found here). Realtime finite difference-based simulations of large models are typically too computationally-intensive to run on CPUs. In our work, we have implemented finite difference simulations in realtime on GPUs.'"
Oh the possibilities! (Score:5, Interesting)
Re:Oh the possibilities! (Score:5, Funny)
You had to wait for a computer model to find this out? I guess the music scene in your town is pretty boring.
Re: (Score:1)
Yes. I'm not allowed around cats anymore.
Re: (Score:2)
Austin is about as far from redneck as it gets.
Go into ANY Mississippi town or city.
Re: (Score:2)
You can only be from one place. And having been to New York and Currently living in California, I'd have to say you're probably way off-base, because I've seen plenty of Appalachian rednecks from the northeast, and plenty of Rocky Mountain rednecks from out here in the west.
Re: (Score:2)
I can tell you that if the cat wasn't neutered, it would sound like a trip to the hospital.
Since it is on a GPU... (Score:2)
...perhaps it would be easier to visualize it? The frame buffer is just one bit-throw away...
Re: (Score:3)
I had a pair of earrings made out of my cat's cojones and gave them to my wife for her birthday.
I'm hoping to distract her from doing the same with mine.
Finally (Score:4, Insightful)
Something to do with all those GPUs when ASIC mining of Bitcoin takes over. It's going to get noisy.
Re: (Score:1)
They've already got GPU accelerated noise makers, but all they do is repeat "litecoin litecoin litecoin"!
impossible! (Score:5, Funny)
" simulate nonexistent materials or physically impossible geometries"
the sound of one hand clapping
Re: (Score:2)
Actually, I can make a clap with only one hand (and nothing else but that hand.), it's just kinda faint.
Certainly not impossible.
Re: (Score:2)
Re: (Score:2)
you sir? are the winrar of TGIF
Mac Only? (Score:2)
Re: (Score:2, Funny)
This just in: paper author commits suicide now that slashdot poster not interested in his life's work.
Re: (Score:3)
You do realize this is research and not a product, don't you? As in, hey look what we discovered we can do!
If you want it ported to Linux using AMD GPUs, request the source code (since that's the only way it's provided) and port it yourself.
Re: (Score:2)
Yawn (Score:2, Insightful)
Yeah, you can do computationally heavy things in a GPU. We've done that for years. All this is saying is that some audio signal processing tasks are computationally heavy.
Re:Yawn (Score:5, Insightful)
I think what's most important is now we have the mathematical models in place that allow us to simulate convincing sounds rather than "sample and include". For the creative types, this will save a ton of effort and money. It also has implications for games, e.g. with the given environment model, be able to produce convincing sounds in real-time rather than taking sound samples mixing them with reverb, attenuation, positioning, etc.
Re:Yawn (Score:4, Insightful)
I think what's most important is now we have the mathematical models in place that allow us to simulate convincing sounds rather than "sample and include". For the creative types, this will save a ton of effort and money. It also has implications for games, e.g. with the given environment model, be able to produce convincing sounds in real-time rather than taking sound samples mixing them with reverb, attenuation, positioning, etc.
Yes, absolutely! I see it as analogous to vector graphics vs bitmapped graphics. Vector audio is THE holy grail of accurate sound reproduction.
If these guys can pull this off, it will be the literal (digital) equivalent of having your own live performance - every time! You will have software based models of various instruments that will play music for you by playing their respective instruments for you real-time. The possibilities of this are actually astounding. You would record or store music not as digital samples (lossy, lossless, notwithstanding) but in terms of *how* each instrument is played. You have now turned the problem on its head - you are constrained by the accuracy of your software/mathematical model of each instrument, and by how well you are able to control it to become more nuanced. At a hardware level, if you assume infinite processing power, the challenge would be to accurately play these software instruments. You could again take a completely different approach - you could for example have an array of speakers where each speaker is dedicated to playing a specific instrument, and all the speakers are fed separate audio signals.
Contrast this to the currently audio setup - which would be a 2.0 or 2.1 or 5.1 or 7.1 stereo/HT setup - where each speaker tries (and fails) to accurately reproduce the entire audible frequency spectrum, or you have a mish-mash setup where different speakers divvy up the frequency spectrum between themselves (think sub-woofer and satellite speakers) so they can do a marginally better half-assed job.
If you look at the entire chain in a traditional setup, you have the speaker driver's mechanicals, the speaker crossover electronics, the speaker wire, the power amp, the pre-amp, the DAC, the player, the source audio signal (mp3, flac, redbook CD etc.), the recording mike, and the recording room - all of these links in the chain distort the music in their own way.
What I mentioned above is only my interpretation of how this technique can be used -there are a huge number of other possibilities - software defined objects, such as in games, can now have their own (genuine) sound, and that will sound different depending on how you interact with them. You could also have virtual instruments, unconstrained by the laws of physics, define their own physics and their own unique sound. You could even program room acoustics and have the instruments play sounds as if it was being played in open space, a large hall, a studio, on a beach etc.
Sigh.
Re: (Score:2)
The novelty is doing it on a GPU which means
Exciting, but still a dream (Score:2)
The difficulty in synthesizing sound is getting the models right. You can't simulate each atom so you need a simplifying model that allows you to reduce the work. And that model has to be accurate in the areas where it matters.
While moving stuff to a GPU gives more computing power (but in a more constrained fashion than a CPU) and certainly helps, the models aren't there yet.
The people researching physical modelling continue to make progress, but I think that if you put state of the art in a game, you'd per
Re: (Score:2)
They could make a bundle consulting for Hollywood space opera movies or pro sound designers maybe.
And how about Google... what will the clanging feel like when they bump into and drill into that asteroid? Will it drive the miners or their robots insane??
Re: (Score:2)
Wasn't Aureal starting in on this kind of thing before Creative bought them and killed the product? I seem to recall their sound chips doing some things to calculate real-time echos and other changes to the sound based on materials and room geometry.
I guess it's good that it can be done on the GPU; it might make for one less chipset to go into a system especially given the move toward DisplayPort.
Re:Yawn (Score:5, Interesting)
Actually I think this is pretty cool. It's always bothered me how repetitive sounds can get in games, it would be a neat trick if you could model object's for sound the way you model them for graphics. Each door, window, rock, etc, could have a subtly different sound from the one next to it. I'm sure they're not to that point now, but they are spelling out the possibilities.
Re: (Score:2)
Re:Yawn (Score:5, Insightful)
Surround sound (Score:2)
I was thinking that it would be good for mapping out real "surround sound" similar to how complex reflection and/or ray-tracing is done.
Even if the initial sounds themselves are canned, the sound through a wooden hallway, a hallway with a carpet, or a large open room would be different. Combine that with digital surround and it could be quite useful.
Re: (Score:2)
Re: (Score:2)
Or the game creators could just stop being cheap, and record/licence 100 different smashy-glass sounds instead of 3. And don't get me started on that damn squeaky-door noise in movies!
What next? Are you going to bitch about the Wilhelm Scream?
Re: (Score:2)
There's already some sound randomization in better games.
Re: (Score:2)
No exactly.
If you used a generalized human vocal tract model, with parameters for noteworthy features like chord thickness, length, trachea diameter, et al, then used a natural language generator to generate textual dialog, and finally, combined these into a text to speach engine, you could have a very wide variety of NPC dialog.
The natural language part can run on the cpu, and the realtime sound rendering TTS can run on the GPU.
Re: (Score:2, Interesting)
It's standard fare for science press releases. If the actual advance you're making is boring (using different hardware to speed up processing), then tell them about the part that's been done all along and take credit for it. (Alternatively: take credit for being "just about there" from some far off future goal, to which you've just made a non-trivial but still minor advancement.)
Most "science" journalists eat it up, slightly rewriting it and passing it over to their editors so that they can knock off early
Re: (Score:2)
Re: (Score:2)
The *video* is not real time either - it is delayed by 1/60th (or 1/72nd, etc) of a second. So, whether you process your audio in the CPU, in the GPU, or elsewhere, you need to have it line up with the delayed video.
Re: (Score:1)
Just have the video processed by the CPU. It's not like it's very busy at that moment :-)
It'd be like an IT version of Freaky friday.
Re: (Score:2)
Actually, the GPU is much faster, by far, than any* DSP setup. The GPU consists of hundreds or thousands of DSP building blocks, although more limited in what they can perform and in which order.
* Naturally you can build out a DSP system to match the speed of a GPU, as you can build a rack of CPUs to outperform a single GPU. That's not my point.
Re: (Score:2)
"Real time DSP on a GPU, what? you'd never need that much juice."
Ever try pitch shifting? Even the best hardware in a computer tends to make the sound warble like mad once you go up or down a whole step.
This would significantly reduce that sort of artifacting, and trust me, you need a TON of power to do it.
Re: (Score:2, Funny)
You just need to parse the sentence correctly --- perhaps the name of the university is "Two Researchers At San Francisco" State University.
take that old zen koan! (Score:1)
then we'll finally have the answer to "what is the sound of one hand clapping"
This is nothing. (Score:1)
Oscillofun (Score:2)
You should see 3d graphics done with my audio card.
Are you referring to Oscillofun [youtube.com]? I thought that was interesting.
Impossible geometries? (Score:4, Interesting)
What do they mean by "physically impossible geometries"? Are they talking about things that have a higher or lower number of physical dimensions (eg: a 4 dimensional object or a 2 dimensional object)? A weird combination of Euclidean and non-Euclidean geometry?
Re: Impossible geometries? (Score:5, Insightful)
Imagine a metal cymbal shaped as a sphere with no holes in it floating free in the air. Now hit that cymbal with a mallet that is longer than the diameter than the cymbal. But hit the cymbal on the inside of the sphere. Oh and the interior of the sphere is a vacuum.
There you go, there are a few impossible geometries (and other things) in that scenario.
Re: (Score:1)
If the mallet is inside a vacuum inside the sphere, its dimensions are irrelevant because it won't be producing any sound waves.
It will create vibrations that will travel and make sounds on the outside
And you can suspend a sphere with a electromagnetic field.
which would act forces upon the sphere, possibly constricting/altering its resonance
Heck, it wouldn't be that different from a brass ball hanging on a string getting hit with a heavy stick.
More physical dimensions might be interesting though.
solid vs. hollow. less/more mass on the object
Re: (Score:2)
If the mallet is inside a vacuum inside the sphere, its dimensions are irrelevant because it won't be producing any sound waves.
Yes it will, but they'll be confined to the surface of the sphere (and the mallet).
Your impossible geometry wouldn't be very different from a completely possible thing.
You missed the part about the mallet being simultaneously longer than, and entirely within, the sphere.
Re: (Score:2)
Not impossible. It is entirey possible to have an arclength far longer than the diametric distance of the sphere's interior.
Just coil up the hammer's handle. Boom. Longer than the sphere, still inside.
Re: (Score:2)
No, he said that the hammer has a length that is greater than the diameter of the sphere, is inside the sphere, does not touch the sphere, is evacuated. At no point did the GP asster that the handle of the hammer is a linear vector, and not a curve. He only said the length was greater than the spherical diameter. Next time he should be more careful.
I assert that this can exist:
1) the hammer's arm is a coil that does not touch itself, nor the wall of the sphere. Its length is greater than the sphere diameter
Re: (Score:2)
Just to point out, what would be geometrically impossible:
A sphere with no thickness to its skin.
A hammer with a volume greater than the volume of the sphere, being completely inside said sphere. (Very different from length, which can occupy many spacial dimensions.)
Superluminal oscillations interacting with normal matter (can't be done realtime though.)
A closed solid with a greater interior volume than its exterior one.
A material comprised of antimassed particles.
A sonically superconducting material
(Others
Re: (Score:2)
Re: (Score:2)
A cymbal shaped like a Klein bottle could be simulated by this, for example.
Re: (Score:2)
the sound of vibrating klein bottle
People use GPUs for numerical simulation (Score:4, Insightful)
News at 11.
Re: (Score:1)
News at 11.
Will them news be GPU rendered in Realtime ?
It's pretty great actually (Score:2, Insightful)
One of the fundamental problems with computer based music production is that we're still, unless we're working with synthesized music, limited to pre-recorded samples.
Vienna Symphonic Library, for example, is well over several hundred gigabytes in size, many of those samples covering various articulations (playing techniques) of the same instrument.
One set of violins playing legato. One set of violins playing pizzicato. Marcato samples etc. etc. With virtual instruments that is no longer necessary. We can j
GPU is not that useful for audio (Score:2)
The second is that most audio processing usually relies on complex directed graphs consisting on nodes that each process a different task, and that kind of interaction is too complex for the simpler, massively parallel GPU architecture.
It would be fanastic for us that work in the audio industry to have some sort of DSP acceleration coprocessors for audio, but there's not enough demand to make that affordable so we can only wait for GPUs to becom
Re: (Score:1)
With HDMI output, the graphics card is the last place in the computer to touch audio before it goes to a TV. It might have other advantages.
Re: (Score:3)
They're not talking about processing in the sense of DSP, they're talking about synthesis of sound waveforms simulating physical models of the instruments. Any DSP would come after that.
Re: (Score:2)
It would be fanastic for us that work in the audio industry to have some sort of DSP acceleration coprocessors for audio, but there's not enough demand to make that affordable so we can only wait for GPUs to become more flexible and realtime friendly, or CPUs to become more parallel.
You can very easily buy audio DSP co-processor quite cheaply. Connect via Firewire or USB, PCIe, or as a standalone unit.
Pro Tools HD being one of the most widely used DSP co-processors. Write your own RTAS or AAX plugin, you can use the DSPs.
If you really want to get down to prototyping, use Matlab and buy a SHARC dev board.
DSP is being used in almost all audio hardware these days. Consoles, compressors, EQs, it's all going digital. The demand is huge.
Re: (Score:2)
Digital isn't really what everyone is into but it has it's purposes.
Re: (Score:2)
Re: (Score:2)
DSP is better than GPU but GPU can do stuff a DSP cannot. Physical modeling is a perfect example because to get certain instrument sounds right, like strings based instruments, steel drums, gongs, etc, you need hardware acceleration. Software does a crappy job at it and sampling cannot do it very well period.
If it's possible to use some of the GPU power for audio then we should.
In other news (Score:2)
Hopefully this means my old college buddy Marc can finally graduate. :p
Hoping for realtime voice (Score:2)
I can't wait until real-time synthesized voices escape the uncanny valley. Neal Stephenson was pretty prophetic in 'The Diamond Age' of having live voice actors behind dynamically scripted content; not that we have that, but that we still don't have good voice generators.
Voice 'acted' games without requiring actors to pre-record every possible phrase would be great.
Re: (Score:2)
Because of expenses like that, I sometimes wish the dialogues were un-voiced (as in Fallout 1/2); however, a TTS engine would be a good alternative t
GPU/DSP (Score:2)
Re: (Score:2)
DSPs have done sound modeling for years. So is the GPU the new DSP? Or is it simply cheaper because your desktop machine already has a GPU, whereas it may not have a DSP?
Cheaper and possibly more accurate.
SuperNatural (Score:2)
When will it be in FLStudio? (Score:2)
Because if it's not, why should we care?
Re:RESONANCE FREQUENCY (Score:4, Informative)
When will you people LEARRRNNNN???????
Maybe when there is something to learn? Both forms are in common use, including physics textbooks and papers.
resonant frequency n. Physics a frequency at which resonance (of any kind) takes place.
1897 L. Bell Electric Power Transmission x. 393 When the [electrical] oscillations are strongly damped by the presence of iron, the total resonant rise is considerably diminished, but it varies less rapidly as the resonant frequency is departed from.
1934 J. P. Den Hartog Mech. Vibrations ii. 52 The forced frequency coincides exactly with the natural frequency... This important phenomenon is known as ‘resonance’, and the natural frequency is sometimes called also the ‘resonant frequency’.
2001 S. Hawking Universe in Nutshell ii. 52 (caption) Just like the strings on a violin, the strings in string theory support certain vibrational patterns, or resonant frequencies, whose wavelengths fit precisely between the two ends.
Re: (Score:1)
Re: (Score:2)
Do you often drink a glass or a bottle, litterally? Saying "a frequency" rather than "a pressure wave with a given frequency" is a metonymy.
Re: (Score:3)
Dear anonymous moron,
Established use of a term is what gives it definition. That's how language works. if we decided that "cow" is a small bird that's delicious, then that's what a cow would be.
Re: (Score:2)
To be fair though. I don't care how many people tell me it is legitimate.
If you try to "Aks" me a question I judge them an idiot. Period.
Re: (Score:2)
That's a mispronunciation, not a misused word.
Re:RESONANCE FREQUENCY (Score:4, Funny)
I could care less how many people tell me it is legitimate.
FTFY.
Re: (Score:2)
FTFTFY
Re: (Score:3)
"I WHOOOOOSH how many people tell me it is legitimate."
FTFTFTFY
Re: (Score:2)
Re: (Score:2)
Dear Nobel Laureate,
I'm guessing you got your "Nobel" in Economics? Because you're clearly completely delusional about how the real world works, and consider your "correct" definitions to establish the basis for all truth. Here's a clue: language is created by the people who use it. When a huge number of published physicists use a phrase in countless papers and textbooks, it is by virtue of that widespread use "correct" regardless of any other syntactical, etymological, or semantic arguments.
Re: (Score:2)
Yep. Pretty much any linguist worth his salt will tell you that unofficial and incorrect useage of language is largely how language evolves.
Old german didn't evolve into old english via committee, nor did old english evolve into new english via committee. It came from peasants abusing language , speaking in slang, fucking up grammar and generally speaking however felt comfortable. And here we are centuries later, with the language of academia, commerce and international relations (yeah yeah, french guys, i
Re: (Score:2)
Dear ball profferer,
I'm glad to hear your GF enjoyed my performance enough to want me back for more. By the way, you should spend more time listening to her and learning how her body works --- you might find out that "ovaries" aren't the part you lick.
Anyway, I'm perfectly happy with my education (public elementary and high school, followed by private college and gradschool), which puts me on the side of this terminological issue with Feynman, Hawking, and the Oxford English Dictionary. Perhaps with a more
Re: (Score:2)
"Frequency is an abstract measurement."
Nobel failure. Frequency can be quantized and measured non-abstractly.
I work with frequencies all day long, in the THz range. Try again when you actually understand something.
Re:RESONANCE FREQUENCY (Score:5, Informative)
I'm an Engineer, and this is the first time in my life I've heard "Resonance" and "Frequency" in that order.
I studied that stuff for a while. Had some tests on it. Read some books on it. Look into it at work.
I'm also a bit of a musician.
I'm not saying you're incorrect -- I've just never heard that phrase before.
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
It can be used either way. In Pro-audio speakers, each has a Resonant Frequency. It's tested as "Fs" or "Free air Resonance" as a result the community ended up mixing the two words ages ago and now everyone calls it Resonance Frequency, Resonant Frequency, Fs or RF. It all means the same thing. Language is a living breathing thing and this guy is only wrong in trying to get someone else to stick to rules that don't exist. His use of the term isn't incorrect, he's just being a dick.
Re: (Score:2)
That's funny. I'd say you got cheated, then.
Because at the merely-public huge university engineering school I attended, one of the earliest things we learned about was the "resonant frequency" of a bridge could cause it to be destroyed by wind or marching soldiers. Yeah, professors with 30 years tenure used that term, as did all of the physics professors at that same institution. And that people and buildings have resonant frequencies as well (as Tesla showed when he scared his neighbors badly.)
Re: (Score:2)
We agree, so you may be a response level too deep.
Resonant frequency = yes
Resonance frequency = no
Re: (Score:2)
In English, at least, there's only one s in resonant & resonance.
Re: (Score:2)
It is not RESONANT frequency. It is RESONANCE frequency. When will you people LEARRRNNNN???????///slash
I've never seen that in a text. The CRC/IEEE Electrical Engineering Handbook uses the term "resonant frequency". Some math texts refer to it as "natural frequency," particularly when a mechanical system is being modeled -- like the traditional mass-on-a-spring thing. Musicians use "fundamental" (or "fundamental frequency"), but what's a couple "pi"s between friends?
Now if you'd go crack down on "damp" versus "dampen," I could totally get behind that.
Re: (Score:3)
[Damp vs dampen (vs moisten)]
Look, I LIKE my sonic disturbances to come with a little added moisture, ok? Is that so wrong?!
Same with my inertia! Dry inertia is just erosive as hell, and very uncomfortable!
I don't *care* that those are both things that fudamentally cannot be made moist. I want to dampen them anyway!
(Lol!)
Re: (Score:2)
It is not RESONANT frequency. It is RESONANCE frequency. When will you people LEARRRNNNN???????///slash
So, when will you youngsters stop with this nonsense of misapplying the word "energy" to science-y stuff? As any theology student knows it's always meant the activity of God in the world [wikipedia.org]. Gee, silly 16th'ers and their "mechanics" distorting the plain meaning of the language for us good folk!
And get off (the ancient ruins of) my lawn!
Re: (Score:3, Funny)
Looks like you need some GPU acceleration to handle realtime first posting.
Re: (Score:1)
(re-)Choose your specialization :
- Audio Processing Unit (APU)
- Stocks Processing Unit (SPU)
- First Post Unit (FPU)
Re: (Score:2)
It's clear you know nothing about music making.
Developing virtual instruments brings down the cost of music production, but controlling, say, a virtual violin to get the same kind of articulation as an actual violinist would, in real time, requires - essentially - a violin as a controller, with all the skills necessary to play it.
We already have sample libraries that fit the bill to make demos and a wide variety of music. We have hybrid synths/samplers. Doing it on the GPU won't revolutionize music.
Wh