Join Us

Empowering the Next Generation of Women in Audio

Join Us

When it stops running like clockwork – troubleshooting the basics

Live Digital Audio Part 4

What can you do when it goes wrong? Each live set up will vary, but there are a few fundamentals you can check before having to call tech support.

Snap, crackle, and pop – how to recognise digital issues

What does a sampling or word clock issue even sound like? Unfortunately, they can sound like all sorts of weird noises. As mentioned in previous posts, if your word clock starts to slip out of sync it will just sound a bit off. You may notice the stereo image doesn’t feel as big, or it sounds a little dull, or phasey. More disconcerting problems manifest themselves as strange periodical pops or clicks, or even like someone twanging a rubber band. Think of the 1s and 0s – if the information isn’t transmitted correctly there isn’t any intermediate value, there is no 0.5 in binary. The bit is either there, or it isn’t. Devices then try to read the empty (or partially empty, in the case of slight mistiming) slot and it may result in random noise. If we go back to our video analogy if you’re watching an analogue transmission and the reception goes bad, it will go fuzzy around the edges, but it’ll still be watchable. If a digital transmission does the same, the entire image tends to freeze, become pixelated, or fill with weird patterns and colours.

Don’t overcomplicate it

So what can you do about it? The number one step should be to make sure it is actually a digital issue. Don’t hear hooves and presume zebras. If something is crackling, make sure it isn’t a bad stage cable or piece of backline, etc. If there’s a weird noise, is it over the entire system or is it just on one or two channels? If it isn’t global, it’s more likely to be something like a broken input or analogue transmission line. If it’s over several channels, it could be crosstalk in a multipin cable or something in one area of the room that several mics are picking up.

If you’re certain it’s a sampling or word clock problem, think through the signal path methodically, like you would with any other fault. Make sure everything is still set to the correct sample rate, master clock, and IP address. Check your cables are seated correctly – this is especially important for BNC connectors. Due to the central pin being so small, if they aren’t in the socket completely straight, or are taking a lot of strain, they can lose too much of their connecting surface area and stop passing signal. Optimally (I always try to secure cables in such a way that the weight is taken off the connections for this reason. It also keeps the cable in good condition for longer). Double-check that you haven’t connected cables of differing impedance. Work from one end of the signal path to the other, swapping out cables or bypassing devices one at a time. Take it down to the simplest setup you can, removing things like effects units or even the desk. If your amps have the capability, playing music or pink noise directly into them can take a big chunk of your setup out of the equation. If you are still experiencing problems, check that everything’s firmware is up to date, or at least the same version as other devices of the same type (e.g., all the amps, or desk stage boxes).

Have you tried turning it off and on again?

If this is all in order, it’s time to turn things off and on again! I know it sounds like a cop-out, but power cycling devices (using that term in front of clients makes you sound more professional than saying you’re turning it off and on again) can force them to reset to the right settings, or do their boot-up checklist, where they do a sweep of the network and see other devices that they might have lost contact with for whatever reason. It might not be enough to just switch it off; completely unplugging the device from its power source and leaving it for half a minute can sometimes make it forget any bad information it had stored and start again. Of course, be sure to save anything you need, like your show file, before trying this step!

Make the call

If none of this helps, I’m sorry to say it’s time to call tech support. But here’s the secret to most tech’s tech support: it’s other techs. Nearly everyone in this industry has friends who know more than them about certain topics, which they can phone when they’re stuck.  If nothing else, they might know the solution just because they’ve had the same thing happen to them before. Yet another reason to be (genuinely) nice to everyone; we’re all part of a hive mind of knowledge, and nearly all of us are more than happy to help a colleague out of a bind. On top of this, audio equipment manufacturers are invested in making sure you don’t have bad experiences with their gear, and their support people are often ex-techs themselves, so they will definitely want to sort you out! There’s no shame in tapping into that knowledge if it gets the gig back up and running, and soon enough you’ll be the one who receives the panicked phone call half an hour before doors. Just remember that the more problems you experience, the more you can help others in the future.

Live digital audio in plain English part 2

Never mind the bit clocks… it’s a word clock primer

My last blog dealt with translating audio into a digital signal. The next step is keeping that signal in time when it’s being captured, processed and sent to different parts of the system. This is where the fabled word clock comes in. If anything weird ever happens with a digital set up, like odd clicks or pops over the PA, you can seem wise beyond your years by nodding sagely, saying “Hmm, it sounds like a clocking issue”, then making your excuses and leaving before any further questions can be asked. However, you can become a rare and very valuable member of your audio team by actually learning what word clocks are, how they work and how to fix the most common problems they can cause. They might seem strange and complicated, but they are of course not black magic. It’s all about crystals.

So… what is a word clock?

Any device receiving audio sees a string of 1s and 0s. How does it know whether 0000011100001011 is two samples, reading 00000111 (= 7) and 00001011 (=11), or the second half of a sample, a full sample (01110000 = 112), and the first half of the next one? As you can see, the resulting values can be very different, so it’s essential to get it right to the exact bit.
A word clock is a signal that is sent at a very accurate frequency of one square wave per sample (the bits in each sample make up a ‘word’). This signal is produced by passing an electrical current through a small crystal inside a word clock generator. The rising edge of the resulting wave means 1; the falling edge means 0. The clock runs alongside the audio signal, with 1 usually meaning “this is the start of the sample” and 0 meaning “this is the end.” Different shapes and sizes of crystal resonate at different frequencies, then more subtle changes are controlled by variations in the voltage running through the circuit and temperature. Some clock generators even keep their crystals in tiny ‘ovens’ to keep the temperature constant.

What the clock?

Clocks are necessary for a few different stages in the signal path. AD convertors might take a fixed number of samples per second, but they still need to make sure those samples are evenly spaced. If they aren’t, the waveform will end up deformed when reproduced by something that is in time. Thinking back to the video analogy from the last post, it’s like film taken on old hand-cranked cameras: uneven capturing of the signal leads to weird inconsistencies when it’s played back. In audio, it’s referred to as jitter. This can also happen when an accurately-captured signal gets reconverted with an unreliable clock, like a film being played on a clunky projector (see figure 1). Clocks used to trigger the capturing of the signal are often called sample clocks. There are also bit clocks, which produce one cycle per bit. These days they are only used for signal transport within devices, for example from one PCB to another. You’re very unlikely to encounter a problem with a bit clock, and if you do there isn’t much you can do except send it back for repair. You might also hear people referring to word clock as sync clock, signal clock or simply clock.

An AD converter with a stable word clock (represented by the square wave at the top) captures an accurate waveform (left), but if it’s converted back to analogue through a DA converter with an unstable clock, the waveform will become deformed (right). Source: Apogee Knowledge Base http://www.apogeedigital.com/knowledgebase/fundamentals-of-digital-audio/what-is-jitter/”

 

One clock to rule them all

A stable clock compared to a jittery one, compared to one whose frequency has drifted. Jitter is caused by a varying clock frequency, whereas a clock that has drifted has a pretty stable frequency. It’s just the wrong one. Source: Apogee Knowledge Base http://www.apogeedigital.com/knowledgebase/fundamentals-of-digital-audio/word-clock-whats-the-difference-between-jitter-and-frequency-stability/”

What we are really interested in for live audio is using word clock to keep multiple devices, e.g., the front of house desk, monitor desk, and system processors, in sync. Think of it like keeping a band in time: most digital devices on the market have their own internal clock, so it’s like each member of the band having their own click (or metronome if you’re that way inclined). If it’s a solo artist, there’s no problem. Even if the click wanders a bit, it probably won’t be noticeable, because there’s nothing to compare it to. However, when there are several members, they need to stay in tempo. Neither clocks nor clicks are perfect, and even if everyone starts off together, they will eventually fall out of sync (known as frequency drift. See figure 2). It makes sense to choose one person to keep the beat for everyone else, like the drummer. Much in the same way, you need to designate one device in your system to be the master clock, and the other devices are slaves who sync their clocks to the master. Sometimes, it can be even better to get a separate device whose only job is to keep time, i.e., an external word clock generator. This is like hiring a professional conductor for the band. Much like a conductor though, they can be very expensive and for the most part aren’t necessary as long as you have a good enough band/set up.

“Mirga Gražinytė-Tyla conducting the City of Birmingham Symphony Orchestra at the Snape Maltings Concert Hall during the Aldeburgh Festival, 2017, by Matt Jolly. https://en.m.wikipedia.org/wiki/File:Mirga_Gra-inyt–Tyla_conducts_the_CBSO,_Aldeburgh_Voices_and_Aldeburgh_Music_Club_at_Aldeburgh_Festival-crop.jpg”

Each device still uses its own clock when following the master. They constantly monitor at what phase in the cycle the incoming clock signal is and compare it to their own. If the two fall out of time, the device can adjust its clock (usually by varying the voltage running through the crystal) until it’s locked in sync. The circuit that does this is called a phase-locked-loop. It’s like a band member nudging the speed of their click or metronome until it matches the conductor. However, some common sense is needed. You don’t want to constantly adjust for every tiny discrepancy, nor do you want everyone to follow when the conductor is obviously wrong, like if he sneezes or falls over. A phase-locked loop’s sensitivity can be adjusted, so it ignores fleeting differences and remains locked to the last signal it received if the master clock outputs major errors or drops out of the system. The device will then continue at that speed until the master gets reinstated or replaced, but will slowly drift if this doesn’t happen. The sensitivity can also be adjusted depending on how good the device is compared to the master. If your conductor isn’t the best, it might be better to listen to your own click when in doubt (or invest in a better conductor). In the next post, I’ll discuss how all this relates to our real life setups.

X