arsclist Thoughts on a "dream" manual declicker/deglitcher for 78 restoration (Windows-based software)

[Jon Noring <jon@xxxxxxxxxxx>]

I've previously posted on this topic to the Usenet newsgroup
rec.audio.pro, but thought I'd repost it here a few months later, with
some editing. I apologize for its length -- consider it as a sort of
"white" paper.

It is entirely possible that somebody has built the tool described
here: a manual declicker using frequency-space for visual location
of clicks/glitches, and a robust removal and reconstruction
algorithm to rebuild the wave form at the point the glitch was
removed.

*****

In the digital restoration of transfers from older, pre-1950 shellac
recordings (commonly referred to as 78's since that is the typical
speed they were recorded at), the removal of clicks and crackle is
almost always necessary in the first steps of the restoration. There
are excellent, professional-level yet relatively inexpensive automatic
declickers and decracklers on the market which will remove most clicks
and crackle without perceptably distorting the sound (as an example,
Waves X-Crackle is a very well-regarded yet reasonably-priced
decrackler, considered by some to be on par with the astronomically-
expensive CEDAR and NoNoise decracklers), but such tools cannot remove
them all (well, they can be set to remove 100%, but then they audibly,
adversely and unacceptably distort the wanted signal itself, creating
annoying artifacts.)

Thus, to remove the rest of the audible glitches (most of which are
very short, less than 1 millisecond in duration, oftentimes only a few
samples wide), and leave only the uniform background "78 hiss",
requires a manual "search and destroy" process.

(Note that many afficionados of old recordings do not mind the "78
hiss" so long as it is completely *uniform* and the clicks and crackle
are removed -- oftentimes no more noise reduction is necessary. But
even if further noise reduction is to be done, it is best to first
have all the audible clicks and crackle fully removed.)

Some 78 restorers have become quite proficient at searching and
finding audible glitches simply by scanning the waveform around the
time frame they hear an audio glitch (George Morrow is one such
person -- he uses NoNoise as will be further described here.) To do
this requires a lot of practice, a very good ear and eye, and it is a
*lot* of work. Once a glitch is found, then there's still the need to
fix it properly, which I'll talk about in a moment.

Exacerbating this problem is that it is not unusual to fix anywhere
from one up to ten glitches per second of audio! For a typical three
minute 78 recording we are talking about *hundreds* to well over a
*thousand* glitches to manually remove, thus it is important any tool
doing a manual "search and destroy" be easy to use, quick, and
effective.

--> Any tool requiring "painting", manual interpolation by moving
samples up and down, anything involving human guesswork, etc., etc.
--> is unacceptable <-- it just takes too damn long, it is fatiguing
for production use, and prone to errors which replace one audible
glitch with another. It is really unnecessary, too, as I intend to
explain. The solution must be point, click, and move on to the next
glitch -- if more than 10 seconds *on average* is spent to manually
and perfectly remove one glitch -- that is unacceptable. (As I'll note
next, it is fortunate that glitches are usually very narrow in time
and simply slicing them out and properly reconstructing the sliced out
samples is all that is needed -- in the "good old 'analog' tape days",
Scott Dorsey talked about getting good results using a typewriter
eraser on 30 ips tape to "erase" out the glitches! Of course, it was
common practice to slice out clicks using a razor -- it takes someone
with the patience of Job to do this.)

Returning to the repair process, I strongly believe, from a lot of
experimentation with various products doing aspects of what I am
looking for, that the only acceptable repair is to remove the samples
from the waveform where the glitch occurs -- just wide enough to
encompass the glitch and *no more* -- and then replace those dozen to
a few dozen samples with a machine-calculated interpolation/
reconstruction based on high-order frequency analysis on both sides of
the glitch, and of course in phase and continuous with both sides. All
samples right outside of this narrow time slice should not be touched
whatsoever. (It is unacceptable to replace the glitch with either
silence, or to connect the end-samples of the glitch with a "straight"
line, as most wave "repair" tools do. It is also unacceptable to
require the user to "re-paint" the waveform since that takes too long,
is prone to errors, and various other problems.)

(Note: Obviously a fairly robust waveform reconstruction algorithm is
needed. I do know from a recent RAP response by Monte McGuire that the
NoNoise option in the classic Sonic Solutions (for the Mac) has an
excellent waveform reconstruction tool ("Type D"), but unfortunately
Sonic Solutions is breathtakingly ultra-expensive to the point that if
you have to ask how much it costs, then you probably can't afford it
-- supposedly the NoNoise module has been licensed out to Pro Tools
and will be available for an "afforable" $2000 (pocket change for most
of us, of course), but I have not been able to locate a reference to
it at DigiDesign, so it probably hasn't been released yet. Sonic
Solutions is also Mac-based only, and I am looking for a Windows-
environment, *** software-only *** tool (these days having hardware-
based digital signal processors is no longer necessary except maybe
for real-time processing, which has no bearing on 78 digital
restoration.)

Regarding an algorithm, I would assume the excellent NoNoise
algorithms are proprietary, and have not yet been published anywhere.
However, Francois recently mentioned on RAP that Robert Maher authored
an intriguing paper on this topic a few years ago in the Journal of
the Audio Engineering Society that outlines a robust algorithm to
reconstruct a short damaged portion of a waveform. Having not yet
studied his algorithm, I assume that implementing his algorithm may be
all that is required. It would not surprise me if other refereed
journal papers have been published on this same topic.)

Now, to summarize my requirements for a manual declicker/deglitcher:

1) Allows one to quickly locate glitches using some sort of *sure-fire*
   visual approach verified with a listening test, and

2) The repair should be done as previously noted: easily and rapidly
   remove and replace only the samples within the narrow glitch
   without affecting the samples outside of the glitch. The
   reconstruction is done using a high-order interpolation based on
   the frequency characteristics found on both sides of the glitch,
   and make sure the reconstructed section is reasonably continuous
   and in phase with the surrounding signal.


One product on the market, which can be used as a manual declicker
when adjusted right, Pristine Sounds 2000, acceptably, albeit not
admirably, fulfills the first requirement. It presents to the restorer
a three second 'sonograph', a frequency versus time map (using
wavelets to calculate, I surmise). The glitches are readily seen as
thin vertical line segments (of course, verification they are glitches
and not part of the music is still necessary -- PS 2000 allows one to
play the three second snippet to confirm the glitches.) PS 2000 allows
the user to actually edit within this frequency map, and using its
declicker mode, one can remove the thin vertical lines representing
glitches. For isolated glitches it works well.

However, PS 2000 has several problems, one of which is fatal based on
my experimentation with it (one can download a demo version of PS 2000
-- that's what I am currently using for testing.)

The first noted problem is that the sonograph window is too small, and
one cannot change its size nor other settings such as time/frequency
spans -- thus trying to "hit" the one pixel wide glitches is oftentimes
difficult. The second noted problem is fatal -- PS 2000 does the actual
editing in frequency space, not on the waveform itself, thus when a
click is removed, it affects the waveform 10 milliseconds and more (!)
on each side of the glitch (I've verified this.) This is not so bad for
isolated glitches, but (as oftentimes needed for many older recordings)
I've observed that removing several glitches close to each other in a
quieter section results in an audible and annoying distortion
("pumping") of the background noise, which destroys the background
noise uniformity, and thus is totally unacceptable (there may also be
other artifacts created as well.) The goal is to maintain ** absolute
uniformity ** of the background 78 hiss -- I'd rather hear the glitch
than the "pumping" of the background hiss. Thus, in essence, PS 2000
violates rule 2 of not confining its repair to the quite narrow glitch
itself *in amplitude-time space* -- it's overkill, and one which
sometimes leaves unwanted audible artifacts.

The important thing to note about PS 2000, however, is that its use of
a sonograph to identify and precisely locate glitches is *exactly* what
is needed to visually and easily locate glitches for the manual
deglitcher I have in mind.

Another interesting frequency space editor is now being developed by
Herbert Griebel: Jaco Visual Signal. I'm honored to be on Mr. Griebel's
beta testing list for version 2 and am impressed with Jaco in being a
powerful and general frequency space editor. In addition, it allows for
the end-user to magnify and customize the editing window, and to easily
move to anywhere in the wave file (while PS 2000 has a fixed window
size, and additionally requires you to start at the beginning and
manually move through the windows to get to any spot within the
recording.) On the downside, however, Jaco does not yet provide a
dedicated deglitcher optimized for ** rapid and sure ** waveform
reconstruction in amplitude-time space.

(Currently, Jaco provides a "paint brush", which requires some
tweaking to setup, and then a lot of skill to use properly -- this
tool is great for complex alteration of two-dimensional areas within
the frequency-time map, which has definite use in 78 restoration (for
those occasional very tough-to-fix problems), but wholly impractical
(and unacceptable in my opinion) for simple (essentially one-
dimensional) glitch removal on old 78 recordings as previously noted.
And undoubtedly the "paint brush" works in the frequency domain so it
will "cast its shadow" well beyond the boundaries of the narrow glitch
itself, again unacceptable for simple glitch removal.)

I'm hoping Mr. Griebel will add a deglitcher which works on removing
glitches in amplitude-time space: interpolating and replacing the
waveform samples only within the glitch itself (say a 0.5 to 1
millisecond time frame -- one can imagine being able to quickly select
a custom time-slice to reconstruct.) In essence, the frequency map
(wavelet mode appears the best) would be used *only* to visualize and
precisely locate a glitch, and not used as a frequency-space editor.
(Jaco is superb at visualizing frequency-space for a recording -- I've
seen nothing else out there which comes remotely close.)


Conceptual Design of a Deglitching Tool
=======================================

O.k., let me summarize all this verbosity above and present a
preliminary conceptual design for a deglitching tool (from the
end-user's perspective.)

A. Visualization
----------------
A wavelet-generated frequency-time map (a 'sonograph') of a portion of
the recording to restore is presented to the user. The user can adjust
the time and frequency spans as needed, and optionally adjust the data
visualization settings (color/gray, dynamic range, etc., etc.) The user
can aurally audition the displayed time span as needed. Of course, the
user must be able to directly and immediately move to any part of the
recording they wish.

B. Glitch Removal
-----------------
The user, upon seeing a glitch in frequency-time space (a very thin
vertical line which can either be limited between certain frequencies
or which extends most of the way between 0 and 1/2 the sample rate),
and after verifying it is unwanted by aural listening, or simply
knowing it is bad from working experience, will move his/her mouse
pointer atop the glitch and click once to select. At this stage, one
of two things will happen next:

1. The waveform at the selected point is instantly reconstructed.
   Obviously, there needs to be some assumed default settings
   depending on deglitching algoritm, such as glitch width (e.g., 10
   samples, 20 samples, etc.) -- settings which the end-user can
   adjust in some fashion as needed.

   Or,

2. A separate waveform view window of suitable magnification is brought
   up (or is next to the frequency-time map) which shows the waveform
   at that selected point. The user then selects the portion of the
   wave deemed "bad" and runs the reconstruction algorithm on the
   selected portion.

Once the waveform is reconstructed at the selected glitch, the
frequency-time window is instantly updated to verify if the glitch was
properly removed in a visual sense. The user can aurally audition the
displayed window as well to assure proper results. Of course, being
able to undo the fix is necessary (such as to change the default
settings for a particularly wide glitch), and it may even be necessary
to provide a means to undo a whole window's worth of fixes for whatever
reason.

*****


Hopefully this "white paper" will encourage audio tools developers to
develop the right tool needed by the 78 restoration community to
manually remove glitches as outlined above. Of course, if someone
has already built this tool (and not part of some astronomically-
priced software or hardware), let us know.

Your comments, and criticisms, are welcome.

Thanks.

Jon Noring

-
For subscription instructions, see the ARSC home page
http://www.arsc-audio.org/arsclist.html
Copyright of individual posting is owned by the author of the posting and
permission to re-transmit or publish a post must be secured
from the author of the post.