Provisional Graph

I’ve now located the minimum data sources needed to start putting together the neural network for this system. I now need to consider how to sample & shape this data. To this end I’ve roughed out a graph – it’s short on details and will undoubtedly change, but should be enough to decide on how to handle the inputs.

To reiterate the aim, I want to take ELF/VLF (and historical seismic) signals and use them to predict future seismic events.

As an overall development strategy, I’m starting with a target of the simplest thing that could possibly work, and iteratively moving towards something with a better chance of working.

Data Sources

I’ve not yet had a proper look at what’s available as archived data, but I’m pretty sure what’s needed will be available.  The kind of anomalies that precede earthquakes will be relatively rare, so special case signals will be important in training the network. However, the bulk of training data and runtime data will come come from live online sources.

Seismic Data

Prior work (eg OPERA) suggests that clear radio precursors are usually only associated with fairly extreme events, and even those are only detectable using traditional means for geographically close earthquakes. The main hypothesis of this project is that Deep Learning techniques may pick up more subtle indicators, but all the same it makes sense to focus initially on more local, more significant events.

The Istituto Nazionale di Geofisica e Vulcanologia (INGV) provides heaps of data, local to Italy and worldwide. A recent event list can be found here. Of what they offer I found it easiest to code against their Atom feed which gives weekly event summaries. (No surprise I found it easiest, I had a hand in the development of RFC4287 🙂

I’ve put together some basic code for GETting and parsing this feed.

Radio Data

The go-to site for natural ELF/VLF radio information is vlf.it and it’s maintainer Renato Romero has a station located in northern Italy. The audio from this is streamed online (along with other channels) by Paul Nicholson. Reception, logging and some processing of this data is possible using Paul’s VLF Receiver Software Toolkit. I found it straightforward to get a simple spectrogram from Renato’s transmissions using these tools. I’ve not set up a script for logging yet, but I’ll probably get that done later today.

It will be desirable to visualise the VLF signal to look for interesting patterns and the best way of doing this is through spectrograms. Conveniently, this makes the problem of recognising anomalies essentially a visual recognition task – the kind of thing the Deep Learning literature is full of.

The Provisional Graph

Here we go –

provisional-nn-2017-07-03

CNN – convolutional neural network subsystem
RNN – recurrent neural network subsystem (probably LSTMs)
FCN – fully connected network (old-school backprop ANN)

This is what I’m picturing for the full training/runtime system. But I’m planning to set up pre-training sessions. Imagine RNN 3 and its connections removed. On the left will be a VLF subsystem and on the right a seismic subsystem.

Pre-Training

In this phase, data from VLF logs will be presented as a set of labeled spectrograms to a multi-layer convolutional network CNN. VLF signals contain a variety of known patterns, which include:

  • Man-made noise – the big one is 50Hz mains hum (and its harmonics), but other sources include things like industrial machinery, submarine radio transmissions.
  • Sferics – atmospherics, the radio waves caused by lightning strikes in a direct path to the receiver. These appear as a random crackle of impulses.
  • Tweeks – these again are caused by lightning strikes but the impulses are stretched out through bouncing between the earth and the ionosphere. They sound like brief high-pitched pings.
  • Whistlers – the impulse of a lightning strike can find its way into the magnetosphere and follow a path to opposite side of the planet, possibly bouncing back repeatedly. These sound like descending slide whistles.
  • Choruses – these are caused by the solar wind hitting the magnetosphere and sound like a chorus of birds or frogs.
  • Other anomalous patterns – planet Earth and it’s environs are a very complex system and there are many other sources of signals. Amongst these (it is assumed here) will be earthquake precursors caused by geoelectric activity.

Sample audio recordings of the various signals can be found at vlf.it and Natural Radio Lab. They can be quite bizarre. The key reference on these is Renato Romero’s book Radio Nature – strongly recommended to anyone with any interest in this field. It’s available in English and Italian (I got my copy from Amazon).

So…with the RNN 3 path out of the picture, it should be feasible to set up the VLF subsystem as a straightforward image classifier.

On the right hand side, the seismic section, I imagine the pre-training phase being a series of stages, at least with: seismic data->RNN 1; seismic data->RNN 1->RNN 2. If you’ve read The Unreasonable Effectiveness of Recurrent Neural Networks (better still, played with the code – I got it to write a Semantic Web “specification”) you will be aware of how good LSTMs can be at picking up patterns in series. But it’s pretty clear that the underlying system behind geological events will be a lot more complex than the rules of English grammar & syntax. But I’m (reasonably) assuming that sequences of events, ie predictable patterns do occur in geological systems. While I’m pretty certain that this alone won’t allow useful prediction with today’s technology, it should add information to the system as a whole in the form of probabilistic ‘shapes’. Work already done elsewhere would seem to bear this out (eg see A Deep Neural Network to identify foreshocks in real time).

Training & Prediction

Once the two subsystems have been pre-trained for what seems a reasonable length of time, I’ll glue them together, retaining the learnt weights. The VLF spectrograms will now be presented as a temporal sequence, and I strongly suspect the time dimension will have significance in this data, hence the insertion of extra memory in the form of RNN 3.

At this point I currently envisage training the system in real time using live data feeds.  (So the seismic sequence on the right will be time now, and the inputs on the left will be now-n). I’m not entirely sure yet how best to flip between training and predicting, worst case periodically cloning the whole system and copying weights across.

A more difficult unknown for me right now is how best to handle the latency between (assumed) precursors and events.  The precursors may appear hours, days, weeks or more before the earthquakes. While I’m working on the input sections I think I need to read up a lot more on Deep Learning & cross-correlation.

Advertisements

Reading online VLF

For the core of the VLF handling section of the neural nets, my current idea couldn’t be much more straightforward. Take periodic spectrograms of the signal(s) and use them as input to a CNN-based visual recognition system. There are loads of setups for these available online. The ‘labeling’ part will (somehow) come from the seismic data handling section (probably based around an RNN). This is the kind of pattern that hopefully the network will be able to recognise (the blobby bits around 5kHz):

Screenshot from 2017-07-01 18-07-52

“Spectrogramme of the signal recorded on September 10, 2003 and concerning the earthquake with magnitude 5.2 that occurred in the Tosco Emiliano Apennines, at a distance of about 270 km from the station, on September 14, 2003.” . From Nardi & Caputo, A perspective electric earthquake precursor observed in the Apennines

It’ll be a while yet before I’ll have my own VLF receiver set up, but in the meantime various VLF receiver stations have live data online, available through vlf.it. This can be listened to in a browser, e.g. Renato Romero’s feed from near Turin at http://78.46.38.217:80/vlf15 (have a listen!).

So how to receive the data and generate spectrograms? Like a fool I jumped right in without reading around enough. I wasted a lot of time taking the data over HTTP from the link above into Python and trying to get it into a usable form from there. That data is transmitted using Icecast, specifically using an Ogg Vorbis stream. But the docs are thin on the ground so decoding the stream became an issue. It appears that an Ogg header is sent once, then a continuous stream. But there I got stuck, couldn’t make sense of the encoding, leading me to look back at the docs around how the transmission was done. Ouch! I really had made a rod for my own back.

Reading around Paul Nicholson’s pages on the server setup, it turns out that the data is much more readily available with the aid of Paul’s VLF Receiver Software Toolkit. This is a bunch of Unixy modules. I’ve still a way to go in putting together suitable shell scripts, definitely not my forte. But it shouldn’t be too difficult, within half an hour I was able to get the following image:

img

First I installed vlfrx-tools, (a straightforward source configure/make install, though note that in latest Ubuntu in the prerequisites it’s libpng-dev not libpng12-dev). Then ran the following:

vtvorbis -dn 78.46.38.217,4415 @vlf15

– this takes Renato’s stream and decodes it into buffer @vlf15.

With that running, in another terminal ran:

vtcat -E30 @vlf15 | vtsgram -p200 -b300 -s '-z60 -Z-30' > img.png

– which pulls out 30 seconds from the buffer and pipes it to a script wrapping the Sox audio utility to generate the spectrogram.

 

 

 

A Quick and Dirty Audio Signal Generator

It was pretty clear that my ELFQuake setup would need fairly major mains hum filtering, especially since this location is very close to overhead power lines. This was emphasized the other day when I tried a little AM radio, all but two stations on MW were totally drowned out by hum.

When I was last experimenting with the filter circuitry I found the limited equipment I have rather frustrating. When it came to a signal generator, I didn’t have much joy using either a tablet app or the BitScope waveform generator. What I really wanted was a knob to twiddle for easy tweaking of the frequency. So a couple of days ago I spent a few hours knocking together a simple analog circuit. My requirements were essentially the twiddly knob and something approximating a sine wave. The constraints were the components I had at hand, and no desire to spend too much time over it.

What I came up with is as follows. The circuit starts with a simple triangle/square wave generator using standard analog computational elements based around op amps:

tri-osc

I’m using a TL084 quad op amp with a +/- 12v supply.

When the output of the left-hand op amp is negative, that will ramp up the integrator on the right until it flips the switch on the left (note positive feedback on that op amp). Then the integrator will ramp down until it flips the switch the other way. The frequency is simply determined by the resistors in the middle and capacitor C, as f = 1/2piRC. In practice I’ve got 6 values of C between 4u7 and 1n, producing a frequency range (measured) from about 5Hz to 200kHz. In the middle, with 100n, varying the pot in the middle gave a range from about 275Hz to 11kHz with what looked on the scope like a clean triangle wave. Switching to the top range gave a significant warping of the triangle, but I thought it might still be useful.

Next is a more interesting circuit, which modifies the triangle wave into something approximating a sine:

shaper

The transistors are just a couple of regular BC109s I happened to have a lot of. The transistors act as differential voltage to current converters with a nonlinear transfer function, with the op amp converting the currents back to voltages.

Here’s the cunning bit – the transfer function of the transistors is theoretically proportional to tanh, which looks like this:

TanhReal

Given the right scaling (and bias) this will have the effect of rounding over the upper and lower corners of the triangle waveform. This isn’t an entirely arbitrary choice of function. The first few terms of the Taylor Series for tanh are roughly the same as those for sine.

The actual component values were chosen pretty much by trial and error, adjusting values until the harmonic components appeared to be at a minimum on a frequency response display. Ideally the transistors should have been a matched pair in thermal contact and maybe some tweaking of the levels/balance/bias of this block might have made for a purer sine wave, but I was only after quick & dirty…

This is what the waveforms/freq plots look like (measured on a BitScope), first the square & sine from the first circuit block:

square

triangle

Here’s the shaped output:

sine

Though the 2nd harmonic is still about at the same level as the triangle, but the higher harmonics do seem significantly attenuated.

According to Wikipedia, the Total Harmonic Distortion (THD) of a square wave is around 48.3% – the high harmonic levels are clear on the trace above. For a triangle wave the value is around 12.1%. I’ve not done the sums to figure out the theoretical best achievable using the tanh shaping (homework anyone?), but there is a visible improvement in the displays above. I don’t know, maybe something around 5%..?

(Incidentally, the triangle wave generator can be easily modified to generate a ramp by slipping a diode in the feedback loop).

You may have noticed I’m using a quad op amp, but only 3 of them are in use. I contemplated using the 4th as a buffer or even as the heart of a switched filter. But more useful for me I reckon, and definitely more fun, was to deploy this in a white noise generator:

noise

The noise source is the reverse-biased emitter-base junction of another BC109 transistor. This is simply followed by a DC-blocking capacitor and the op amp set up to give a gain of 100. I think the capacitor I used is a 100n. There was a little bleeding through of the oscillator’s signal visible, but not enough to be troublesome. The output did look remarkably wideband, only really starting to drop off gradually around 50kHz.

noise

I then transferred everything to a piece of stripboard, and in true quick & dirty style mounted it on a scrap of wood:

vero

(Sorry about the blur). Incredibly, even though I put it onto the stripboard without any real planning, I only made one wiring mistake which took about 2 minutes to discover. But I was bitten by hubris when I wired up the capacitor switch – it’s a 2-pole, 6-way, and it took me ages to get the wiring right.

When I tested the sine wave output again there was a visible asymmetry – pointier on top, more rounded on the bottom. I think most likely I got the transistors the other way around. Rather than desolder/resolder them I tweaked the value of the shaper’s input resistor, adding a 1k in series with the 18k in the schematic above, which restored the balance.

So now I have another little addition to my little electronics workbench (big coffee table)…

desk

 

 

 

Data Sources

Two data sources are key to this project: natural VLF/ULF radio signals and seismic vibration. Other environmental signals have been examined in relation to earthquake prediction, several are described on Wikipedia. Three are of note here:

Prior seismic events – in the long term, patterns are recognisable in the progress of earthquake systems. In the short term, major events are often associated with numerous lower magnitude events, after the fact these get classified into foreshocks, mainshocks or aftershocks. Given that seismic signals will be captured by the system proposed here, no extra effort is needed to include these factors into analysis.

Radon gas emissions – while still the subject of ongoing research, spikes in geological radon emissions have been noted preceding earthquakes. Measuring the typical level of radon at a given location is relatively straightforward, by accumulating particulate matter at the location over weeks or months and then measuring its radioactivity, exposing the contribution of radon decay products. But continuous monitoring is much more involved given the low levels of radioactivity involved. Given this difficulty, the radon approach isn’t a high priority in this project. Maybe later.

Animal Behaviour – there is anecdotal evidence that animals exhibit unusual activity prior to an earthquake. This will be left out of scope here due to the difficulty in exploiting such a phenomenon (if it exists). Also, I have two dogs and they displayed absolutely nothing out of the ordinary prior to the last tremor we felt here, they remained asleep. During and after the tremor, they were perturbed…

PS. See Biological Anomalies around the 2009 L’Aquila Earthquake – a lot were reported.

There are additional environmental signals that may have little or no correlation with seismic events but might still improve prediction.

Earth’s magnetic field – the geomagnetic field changes over time and is linked to variations in geology, especially in the Earth’s core. The proposed VLF/ELF receivers will pick up rapid variations in this field, additionally given that it is relatively straightforward to detect ultra-low frequency changes, it will probably be useful to gather this data.

Static electrical field – similar to the magnetic field, the ‘static’ field at a given location can be seen as the ultra-low frequency aspect of signals that the VLF/ELF receivers will pick up. ‘Spherics’, the radio signals generated by lightning strikes are a major feature of VLF/ELF radio signals, and nearby thunderstorms strongly modify the local static electrical field. Again, such data may well be a useful contribution to the system, and measuring this field is relatively straightforward.

Solar and lunar cycles – these bodies exert significant gravitational forces over the Earth and hence are likely to be a factor in geological changes. It should be straightforward to include such data in analysis.

Sunspot activity – whether or not solar discharges have an influence on the occurance of earthquakes (it seems unlikely), they have a major impact on radio signals. It should be feasible to find an online source of related data and include it in the analysis.

Ambient temperature – the reception of radio waves is influenced by the weather and temperature is a key aspect. Also any other measurement circuitry will, to some extent, suffer from temperature drift. Being able to effectively subtract this factor from other signals may help improve results. Temperature sensors are relatively easy to set up, so measuring this seems worthwhile.

Acoustic noise – ie. ambient audio. As with temperature, changes here are is likely to be picked up to some extent by other sensors, especially the seismic subsystem. For example, tractors quite often pass by here, causing a deep rumble that is likely to be detected as a seismic signal. It will be useful to have the corresponding sound signal to help discount such artificial signals. Recording audio is simple so again this seems worthwhile.

[[TODO – move this paragraph to an overview page]] It should be noted that as no completely reliable prediction method has yet been discovered, the general attitude surrounding any approach is that of scepticism. This mood is amplified by the political aspects of prediction – action on a prediction (eg. mass evacuation) is liable to be expensive, so false positives will be costly. Additionally the lack of prediction advice when a major event has occurred can have significant fallout. A notable example being the charging of six Italian scientists with manslaughter following the devastating 2009 l’Aquila earthquake. They were subsequently acquitted (though a government official’s charges were upheld). That such charges were even considered shows the sensitivity in this field.

Loops, Oops!

While I’m very familiar with handling audio-frequency signals in electronics and have a basic understanding of how radio circuits work, there are huge gaps in my knowledge around radio waves, their propagation and reception, and rather a key part of radio system design: aerials.

My planned aerial design for VLF/ELF reception was three air-cored coils positioned in orthogonal directions, like this :

xyz-circles

(see also Hardware Issues)

Unlike, say FM aerials, these small loop antennas will pick up the magnetic component of the electromagnetic signal (as do the coils in typical AM radios). I assumed that the signal received on the N-S axis would be different from E-W and Up-Down. This seemed like the way to capture directional information, assuming the Up-Down direction would also be useful, ie. overall collecting as much information as possible.

Well, I’d had the Wikipedia page on Loop Antennas bookmarked for weeks, yesterday I finally read it, and got a surprise.  As Wikipedia puts it:

Small loop antennas are much less than a wavelength in size, and are mainly (but not always) used as receiving antennas at lower frequencies…

…Surprisingly, the radiation and receiving pattern of a small loop is quite opposite that of a large loop (whose circumference is close to one wavelength). Since the loop is much smaller than a wavelength, the current at any one moment is nearly constant round the circumference. By symmetry it can be seen that the voltages induced along the flat sides of the loop will cancel each other when a signal arrives along the loop axis. Therefore, there is a null in that direction. Instead, the radiation pattern peaks in directions lying in the plane of the loop, because signals received from sources in that plane do not quite cancel owing to the phase difference between the arrival of the wave at the near side and far side of the loop. Increasing that phase difference by increasing the size of the loop has a large impact in increasing the radiation resistance and the resulting antenna efficiency.

So my planned N-S coil would actually have nulls in those directions, similarly for E-W, and the Up-Down coil would in effect be omnidirectional NSEW. D’oh!

There is a lot to electromagnetic radiation! (eg. near vs. far field reception is something I need to read up on). It’s weird stuff.

220px-Electromagneticwave3D

Reading around the topic a little more, the null positions are the key to radio direction finding (RDF). A coil will have two nulls, 180° apart, which RDF gets around by adding a sense antenna, which may be a simple vertical whip aerial. This will be omnidirectional and (if I understand correctly) when summed at the right levels with the coil signal will effectively remove one of the nulls.

Luckily, hardware-wise I’m still at the planning/experimentation stage, so haven’t wasted too much time winding coils. Looks like I’ll have to change the design, provisionally having N-S and E-W coils plus a sense antenna, a la RDF. There are lots of designs for all kinds of aerials and receivers at VLF.it (I’ve just started assembling a list of reference links, that site is at the top of the list).

 

 

Hardware Issues

[work in progress 2017-04-15]

Hardware Overview

(I initially made a significant mistake in my radio reception plans, see Loops, Oops!)

I had hoped by now to have prototypes for a radio receiver and seismograph ready in hardware. What I have got on that front is an increased awareness of how tricky they are likely to be. I still think I’ve got a reasonable design for the sensor part of the seismograph.

Noise Annoys

On the radio side, the experiments I’ve done so far have only really shown that the signal/noise ratio I need to deal with is even worse than I imagined. This is largely due to my location. Although this is a very small village in the hills, everywhere you look are overhead mains power lines. To get a signal that won’t be swamped with noise I anticipate having to filter out at least mains hum at 50Hz and 100Hz (possibly even higher harmonics) in analog signal conditioning. A standard circuit that can do this job is the twin-T notch filter. By wrapping it around a couple of op-amps it can be bootstrapped, boosting the Q, hence the depth & narrowness of the notch can be increased. The big problem here is component tolerance. Metal film resistors are pretty good, having 1% accuracy, but capacitors are another story – anything better than 10% is quite specialist (and that’s not even taking into account effects of thermal variation). Some level of manual tweaking seems unavoidable, worst case 4 trim pots per filter (3 in the T, one for the feedback = Q).

My plan to make a receiver consisting of 3 orthogonal coils will have to remain on hold until I have some funds available (it sounds crazy, but wire looks like it’s going to be the greatest expense). I do have the bits to play with a single channel. And that’s also simpler.

Data Acquisition

I haven’t written it up yet, but once I’ve got usable analog signals from the radio, seismo and other* sensors, I want to use dedicated hardware to get them into the digital domain. Not only that, ideally I want this part of the system to make the raw data available on the Web. I think it makes sense to use a subsystem to collect the data independently of processing, to keep things modular. That way the subsystems can be developed in isolation and additionally the processing workload can be more distributed. So essentially I will have analog-to-digital converters (ADCs) attached to one or more small computers. The physical location of sensors is a consideration – away from sources of interference for the radio receivers, solidly attached to the ground for the seismic sensors. It is necessary in each case to have some of the signal conditioning circuitry local to the sensor (at least pre-amplification).

I then need to get the preconditioned signal from the sensors through the ADCs into the dedicated processing subsystems and on again to the part that will do the processing. The most elegant approach I can think of is also having the ADCs & dedicated computers local to the sensors, with the data being delivered to subsequent processing over WiFi. (Care will be needed in shielding the digital electronics to avoid interference).  An existing standard transfer protocol will be desirable (probably TCP or UDP), and from there it’s only a small step to proxying it onto the Web.

Given this setup, and the considerations of location, modularity and performance, it will make sense to use one ADC+computer subsystem for the radio receivers and one for the seismic sensors.

*  for reasons described below, I plan to eventually detect signals in 3D, ie. both the radio and seismic units having a N-S, E-W and vertical element. But most ADC boards have their number of channels as multiples of 2. Hence I intend to use 2 x 4 channel ADCs, and use the 2 spare channels for acquiring other environmental signals – provisionally acoustic noise (audio) and ambient temperature (again, there’s rationale below).

Two small species of small, inexpensive, general-purpose computers stand out from the marketplace: Raspberry Pi and Arduino. Now I’ve spent a lot of time around digital audio and rather naively assumed that suitable ADCs would be ten a penny. Not so.

For the radio signals, ideally I want to capture something roughly equivalent to fair quality audio, ie. say a bandwidth of 20kHz with 16 bit precision. ADCs that can do this are available of-the-shelf in the recording part of regular soundcards. Unfortunately, there doesn’t appear to be anything available that’s inexpensive and interfaces easily with either computer board (and I have read of issues with data transfer bottlenecks when using USB devices). However I have found a card that appears to fit the bill, for the Arduino : the Mayhew Labs Extended ADC Shield.

Progress (of sorts)

[work in progress 2017-04-15 – material was getting long so splitting off to separate pages]

General Status

In the last month or so I’ve done a little hardware experimentation on the radio side of things, along with quite a lot of research all around the subjects. I also had a small setback. About 3 months ago I ordered a pile of components, they never arrived, a screw-up by the distributor (I wasn’t charged). Unfortunately I don’t have the funds right now to re-order things, so although I do have the resources for some limited experimentation, I can’t go full speed on the hardware. (Once I’ve got a little further along, I reckon I’ll add a ‘Donate’ button to this site – worth a try!)

This might be a blessing in disguise. Being constrained in what I can do has forced me to re-evaluate the plans. In short, in the near future I’m going to have to rely mostly on existing online data sources and simplify wherever possible. This is really following the motto of Keep It Simple, Stupid, something I should maybe have been considering from the start.

 

Next Steps