Mailing List Archive: MythTV: Dev

New Music Visualization : Need to process signal between frames

Index | Next | Previous | View Threaded

mythtv-dev at platformedia

Dec 15, 2011, 10:24 PM

Post #1 of 3 (434 views)
Permalink

New Music Visualization : Need to process signal between frames

I'm building a 'Piano' spectrum visualization, where the pitches detected
directly correspond to piano key pitches.

At the moment, the standard fps=20 means that I'm only seeing a fraction of
the data in my .process(), since (at the moment) the
MainVisual::timeout<http://code.mythtv.org/doxygen/classMainVisual.html#a0f7532480cfb1706438af7c1bf02f11c>
is
discarding 'between the frame' nodes of audio data, choosing to only call
.process() when it wants to .draw() the visualization.

However, for low notes on the piano, I need longer periods of audio data to
get good recognition, so I'd like to run some kind of .process() even for
non-displayed chunks of audio.

Since this feature doesn't seem to be present, what is the MythTV preferred
way to implement this :

(a) Add an additional parameter to .process(node) : i.e. .process(node,
will_be_displayed=true). But the downside of this is that other
visualizations will have an extra 'ignore this one' choice to make; OR

(b) Add an additional virtual function to
VisualBase<http://code.mythtv.org/doxygen/classVisualBase.html> :
i.e. .process_between_displayed_frames(node), which no-one else will be
hooking into, but I'll be able to use to analyze the whole audio stream.
Downside is VisualBase<http://code.mythtv.org/doxygen/classVisualBase.html>
gets
a little hairier; OR

(c) Some method I haven't thought of.

Each will need a little something added into the loop of
MainVisual::timeout<http://code.mythtv.org/doxygen/classMainVisual.html#a0f7532480cfb1706438af7c1bf02f11c>()
which is discarding node data.

An alternative (funky) workaround is to set fps high enough that I catch
all the data. But that's pretty inefficient, since there's no need to
refresh the screen at ~90fps (44100/512 looks like it would be chunk
frequency).

Could someone with an opinion please weigh in? I'll submit a patch vs SVN
once it's looking classy...

Thanks
Martin
:-)

mark.kendall at gmail

Dec 16, 2011, 11:52 AM

Post #2 of 3 (405 views)
Permalink

Re: New Music Visualization : Need to process signal between frames [In reply to]

On 16 December 2011 06:24, Martin Andrews <mythtv-dev [at] platformedia> wrote:
> I'm building a 'Piano' spectrum visualization, where the pitches detected
> directly correspond to piano key pitches.
>
> At the moment, the standard fps=20 means that I'm only seeing a fraction of
> the data in my .process(), since (at the moment) the MainVisual::timeout is
> discarding 'between the frame' nodes of audio data, choosing to only call
> .process() when it wants to .draw() the visualization.
>
> However, for low notes on the piano, I need longer periods of audio data to
> get good recognition, so I'd like to run some kind of .process() even for
> non-displayed chunks of audio.
>
> Since this feature doesn't seem to be present, what is the MythTV preferred
> way to implement this :
>
> (a) Add an additional parameter to .process(node) : i.e. .process(node,
> will_be_displayed=true). But the downside of this is that other
> visualizations will have an extra 'ignore this one' choice to make; OR
>
> (b) Add an additional virtual function to VisualBase : i.e.
> .process_between_displayed_frames(node), which no-one else will be hooking
> into, but I'll be able to use to analyze the whole audio stream. Downside
> is VisualBase gets a little hairier; OR
>
> (c) Some method I haven't thought of.
>
> Each will need a little something added into the loop
> of MainVisual::timeout() which is discarding node data.
>
> An alternative (funky) workaround is to set fps high enough that I catch all
> the data. But that's pretty inefficient, since there's no need to refresh
> the screen at ~90fps (44100/512 looks like it would be chunk frequency).
>
> Could someone with an opinion please weigh in? I'll submit a patch vs SVN
> once it's looking classy...

Martin,

This looks interesting. I don't honestly know the most appropriate
response to your query, other than to say you should really contact
Paul Harrison about his plans for MythMusic. It is currently
undergoing a re-write. He has contacted me about using the fledgling
visualiser support that was added to libmythtv but I'm not sure if
that's the route he's going to take.

regards

Mark
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev

lists at wildgooses

Jan 22, 2012, 7:48 AM

Post #3 of 3 (262 views)
Permalink

Re: New Music Visualization : Need to process signal between frames [In reply to]

I think the original code was trying to do a crude resampling to reduce
data processing requirements. Arguably it should actually *resample*
down to some smaller number of samples, but instead it effectively just
samples X bytes every so often and truncates the rest. This isn't so
bad in practice, but appears not to suit your needs

I would suggest you look at libsamplerate resampler on lowest quality or
even go lower quality than that and just literally skip samples to
achieve the resampling effect (equivalent to resampling with no
windowing function, so expect aliasing junk to appear)

Good luck

Ed W

On 16/12/2011 06:24, Martin Andrews wrote:
> I'm building a 'Piano' spectrum visualization, where the pitches
> detected directly correspond to piano key pitches.
>
> At the moment, the standard fps=20 means that I'm only seeing a
> fraction of the data in my .process(), since (at the moment) the
> MainVisual::timeout
> <http://code.mythtv.org/doxygen/classMainVisual.html#a0f7532480cfb1706438af7c1bf02f11c> is
> discarding 'between the frame' nodes of audio data, choosing to only
> call .process() when it wants to .draw() the visualization.
>
> However, for low notes on the piano, I need longer periods of audio
> data to get good recognition, so I'd like to run some kind of
> .process() even for non-displayed chunks of audio.
>
> Since this feature doesn't seem to be present, what is the MythTV
> preferred way to implement this :
>
> (a) Add an additional parameter to .process(node) : i.e.
> .process(node, will_be_displayed=true). But the downside of this is
> that other visualizations will have an extra 'ignore this one' choice
> to make; OR
>
> (b) Add an additional virtual function to VisualBase
> <http://code.mythtv.org/doxygen/classVisualBase.html> : i.e.
> .process_between_displayed_frames(node), which no-one else will be
> hooking into, but I'll be able to use to analyze the whole audio
> stream. Downside is VisualBase
> <http://code.mythtv.org/doxygen/classVisualBase.html> gets a little
> hairier; OR
>
> (c) Some method I haven't thought of.
>
> Each will need a little something added into the loop of
> MainVisual::timeout
> <http://code.mythtv.org/doxygen/classMainVisual.html#a0f7532480cfb1706438af7c1bf02f11c>()
> which is discarding node data.
>
> An alternative (funky) workaround is to set fps high enough that I
> catch all the data. But that's pretty inefficient, since there's no
> need to refresh the screen at ~90fps (44100/512 looks like it would be
> chunk frequency).
>
> Could someone with an opinion please weigh in? I'll submit a patch vs
> SVN once it's looking classy...
>
> Thanks
> Martin
> :-)
>
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-dev

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads