Digital signal processing through speech, hearing, and Python

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Digital Signal Processing through
Speech, Hearing, and Python
Mel Chua
PyCon 20...

Agenda
● Introduction
● Fourier transforms, spectrums, and spectrograms
● Playtime!
● SANITY BREAK
● Nyquist, sa...

What's signal processing?
● Usually an upper-level undergraduate engineering
class
● Prerequisites: circuit theory...

Therefore
We'll skip a lot of stuff.

We will not...
● Do circuit theory, differential equations,
MATLAB programming, etc, etc...
● Work with images
● ...

We will...
● Play with audio
● Visualize audio
● Generate and record audio
● In general, play with audio
● Do a ...

Side notes
● This is based on a graduate class teaching
signal processing to audiology majors
● We've had half a s...

Introduction: Trig In One Slide

Sampling

Let's write some code.
Open up the terminal and follow along.
We assume you have a file called 'flute.wav' in
the direct...

Import the libraries we need...
from numpy import *

...and create some data.
Here we're making a signal
consisting of 2 sine waves
(1250Hz and 625Hz)
sampled...

What does this look like?
Let's plot it and find out.
import matplotlib.pyplot as pyplot
pyplot.plot(sig)
pyplot.sa...

While we're at it, let's define a
graphing function so we don't need
to do this all again.
def makegraph(data, fi...

Our first plot showed the signal in
the time domain. We want to see it
in the frequency domain.
A numpy function th...

That's a start.
We had 2 frequencies in the signal,
and we're seeing 2 spikes here, so
that seems reasonable.
But ...

That's because the fourier transform
gave us a complex output – so we
need to take the magnitude of the
complex...

But this is displaying raw power
output, and we usually think of audio
volume in terms of decibels.
Wikipedia tells u...

We see our 2 pure tones showing
up as 2 peaks – this is great. The
jaggedness of the rest of the signal
is quantizati...

Answer: the numpy fft function goes
from 0-5000Hz by default.
This means the x-axis markers
correspond to values of ...

The two peaks are at 16 and 32.
(5000/128)*16 = close to 625Hz
(5000/128)*32 = close to 1250Hz
...which are our 2 origi...

Another visualization: spectrogram

Generate and plot a spectrogram...
from pylab import specgram
pyplot.clf()
sgram = specgram(sig)
pyplot.savefig('sgram....

Do you see how the spectrogram is
sort of like our last plot, extruded
forward out of the screen, and
looked down upon...

Now let's do this with a more
complex sound. We'll need to use a
library to read/write .wav files.
import scipy
fro...

Let's define a function to get the
data from the .wav file, and use it.
def getwavdata(file):
return scipy.io.wavfile...

Hang on! How do we make sure
we've got the right data? We could
write it back to a .wav file and make
sure they sound...

Now let's see what this looks like in
the time domain. We've got a lot of
data points, so we'll only plot the
beginnin...

What does this look like in the
frequency domain?
audiofft = fft.rfft(audio)
audiofft = abs(audiofft)
audiofft = 1...

This is much more complex. We can
see harmonics on the left side.
Perhaps this will be clearer if we plot
it as...

You can see the base note of the
flute (a 494Hz B) in dark red at the
bottom, and lighter red harmonics
abov...

Your Turn: Challenge
● That first signal we made? Make a wav of it.
● Hint: you may need to generate more samples.
● ...

Your turn: Challenge 2
● Record some sounds on your computer
● Do an FFT on it
● Plot the spectrum
● Plot the spec...

Sanity break?
Come back in 20 minutes, OR: stay for a demo
of the wave library (aka “why we're using scipy”)
note: wa...

Things people found during break
Problem #1: When trying to generate a pure-tone
(sine wave) .wav file, the sound is not ...

More things people found
Problem #2: The sine wave is audible in the
.wav file, but sounds like white noise rather
than a...

Coercing to int16
# option 1: rewrite the makewav function
# so it includes type coercion
def savewav(data, outfile, sampl...

Post-break: Agenda
● Introduction
● Fourier transforms, spectrums, and spectrograms
● Playtime!
● SANITY BREAK
● ...

Nyquist: sampling and aliasing
● The sample rate matters.
● Higher is better.
● There is a tradeoff.

Sampling

Aliasing

Nyquist-Shannon sampling theorem
(Shannon's version)
If a function x(t) contains no frequencies higher
tha...

Nyquist-Shannon sampling theorem
(haiku version)
lowest sample rate
for sound with highest ...

Let's explore the effects of sample
rate. When you listen to these .wav
files, note that doubling/halfing the
sample...

Your turn
● Take some of your signals from earlier
● Try out different sample rates and see what
happens
● H...

What do aliases alias at?
● They reflect around the sampling frequency
● Example: 40kHz sampling frequency
● Implies...

Agenda
● Introduction
● Fourier transforms, spectrums, and spectrograms
● Playtime!
● SANITY BREAK
● Nyquist, sa...

Remember this?

Well, these are filters.

Noise and filtering it
● High pass
● Low pass
● Band pass
● Band stop
● Notch
● (there are many more, but thes...

Notice that all these filters work in
the frequency domain.
We went from the time to the
frequency domain using an...

We can go back from the frequency
to the time domain using an inverse
FFT (IFFT).
reflute.wav should sound i...

Let's look at flute.wav in the
frequency domain again...
# plot on decibel (dB) scale
makegraph(10*log10(abs(fluteff...

What if we wanted to cut off all the
frequencies higher than the 5000th
index? (low-pass filter)

Implement and plot the low-pass
filter in the frequency domain...
# zero out all frequencies above
# the 5000th index
# ...

Going from frequency back to time
domain so we can listen
reflute = fft.irfft(flutefft,
len(audio))
reflute_coerced ...

What does the spectrogram of the
low-passed flute sound look like?
pyplot.clf()
sgram = specgram(audio)
pyplot.savefig(...

Compare to flutespectro.png

Your turn
● Take some of your .wav files from earlier, and
try making...
● Low-pass or high-pass filters
●...

Agenda
● Introduction
● Fourier transforms, spectrums, and spectrograms
● Playtime!
● SANITY BREAK
● Nyquist, sa...

Formants

Formants
f1 f2
a 1000 Hz 1400 Hz
i 320 Hz 2500 Hz
u 320 Hz 800 Hz
e 500 Hz 2300 Hz
o 500 Hz ...

Vocoding
http://en.wikipedia.org/wiki/Vocoder

Bode plot (high pass)

Another Bode plot...

Credits and Resources
● http://onlamp.com/pub/a/python/2001/01/31/numerically.html
● http://jeremykun.com/2012/07/18/t...

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

LinkedIn
Facebook
Twitter
Email

Replay

Top clipped slide

1 of 76 Ad

1 of 76 Ad