Back to it

Boring Personal STUFF

I’ve neglected this project badly. Aside from being a first-class procrastinator, I am also prone to getting overwhelmed by things. The latter is what happened here. I was at peak enthusiasm when the Kaggle Earthquake Challenge came along, coincidentally my computers all decided to fail at the same time. Not really a big deal, just needed to get things fixed, didn’t take very long. But it knocked the wind out of my sails. Just couldn’t face it at the time.

Fast forward. I’ve just had a couple of weeks knocked out by Covid, clear now. I do need to chase a contract for $$$s like yesterday, but I’m not quite up to working on someone else’s project just yet. I’ve probably got about 100 unfinished projects I could get back to, software and various lumps of electronics sitting on my shelves. But ($$$s aside), this one stands out a mile as being the most worthwhile. So now I’m ready to get back onto the horse/bicycle/crag.

The Proposition

I’m sure I’ve got something similar in this blog’s description, but the basic idea is to use machine learning to identify patterns of correlation between natural radio signals and seismic events and then attempt to make useful earthquake predictions from radio precursors. I have no illusions about this. I reckon, in the best case, very approximate predictions for a very small proportion of events is possible. It won’t be easy and it will take a lot of time. But given how cataclysmic such events can be, it’s worth a try.

The Plan

There are a handful of separate components needed, at the core: data acquisition, a model, a notification system. I think a reasonable 1000 ft view is that of a control system – inputs, processing, outputs, (validation/)feedback. All of which will need creating and tuning.

I really like fiddling with electronics hardware, have put in many hours work looking at the sensor/data acquisition parts of the system. Very poor use of my very limited cognitive resources. After a long break from this, I can shout at myself :

The novel part of this system is around the model.

I think it makes sense to narrow the geographic scope as much as possible, and ‘near me’ is an obvious choice. I live in northern Italy.

High-quality seismic data is available from the National Institute of Geophysics and Volcanology, INGV. Conveniently, the guy who literally wrote the book on natural radio signals has monitoring equipment, streaming live from up near Turin (VLF.it). Also conveniently, in an unfortunate sense, this is an active seismic region (it was the devastating quake of 2009 around L’Aquila that got me wondering about this…not to mention the one in 1920 that reduced Villa Collamandina to rubble, a village I can see from my balcony).

But I have no idea what the model should look like yet. Early on when I was thinking about this, I had a little lightbulb moment. The convolutional networks have been shown to be really efficient at pulling out salient feature from images. A human-friendly way of representing natural radio signals is as a spectrogram. Those should be receptive to reduction by off-the-shelf shape recognition algorithms. Tricky bit is the long-term temporal axis of radio & seismic data. LSTMs probably won’t hack it, but by now there’s probably an appropriate successor. (Ideally the training/application phases will be concurrent, which is a rabbit hole in my near future).

There is an advantage to putting a project on hold for a while, however inadvertent. The software equivalent of Sun Tzu’s “If you wait by the river long enough, the bodies of your enemies will float by.”. Someone else will figure out the algorithms you need.

If it really needs stating, I’m way behind the curve of developments in Deep Learning. But what I think I’ve gathered from the little experiments I’ve tried is that I can play at very small scale on my mediocre home computer (no GPU), acquire/pre-process the data, perhaps get a proof-of-concept (toy!) model topology together. Scale up onto a Cloud service.

Necessary for that is creating an environment in which to code…remembering how to code… Ok, I’m looking at Python, Tensorflow/Keras and/or PyTorch.

So before I consider even a toy version of anything earthquake-related, I need to gently paddle into the water. Last night I had a Brilliant Idea!

Zoltán who?

The prompt for this was probably Flight of the Bumblebee on the Theremin . (She did two takes – one for the sounds, one for the bee. I initially thought she’d ‘cheated’, using a MIDI theremin for note separation – nope. Just put it through tremolo, got her movements against it perfect).

Ok, so Close Encounters of the Third Kind. And/or, Sound of Music. Do, re, me…

With the Sol-Fa Notation (which seems better known that C, D, E… in Italy, btw), Zoltán Kodály, a 20th music teacher built on Corwen et al’s work to have kids do hand signals corresponding to their role/feeling in the scale.

Well, that would be a cool way of playing an instrument.

Naturally I googled it. Naturally, it’d been done by 2016 : MiLa: An Audiovisual Instrument for Learning the Curwen Hand Signs.

But naaaah! I don’t have access to the paper, but in the abstract it says they used ‘a Leap motion sensor‘. Apparently those are spatial tracking things like the IR etc. used with VR kit.

Why not just use a camera?

Grab a frame from webcam, convert it to 28×28 pixel greyscale, associate with one of the 7 labels. Use one of the models known to work well with the MNIST handwritten digit benchmark dataset. Play the Five Tones.

So I’m now in the process of building OpenCV-Python. Predictably my environment was a mess, Anaconda doesn’t seem to play well with QT/Wayland/Ubuntu.

All being well I can get a script together to tell me what hand shape to hold, a few k images within reasonable time. Find model, train, add bleeps.

I’m talking long before I get onto Tensorflow or whatever. Hey ho. Could wait forever for an environment configuration to float by (wasn’t that the whole point of VMs, Docker etc? But when you need one, float on by…).

Should be straightforward once the environment is set up. Which is the purpose of the exercise.

ESP8266 Shiald Progress!

I’m really tired, but while trying to watch TV got to thinking about the Wifi board I’ve been playing with (described in previous post). I’d got as far as loading firmware that allowed it to speak AT codes. Couldn’t resist having a quick look at what could be done next. Luckily I went to my bookmarks first rather than looking at my notes here, because there was a page I must have bookmarked early on and forgotten about : Arduino UNO + ESP8266 ESP-12E UART WIFI Shield. It contains code for a minimal web server.

Looking at an image in this post reveals that the Shield there is there very same Shiald [sic] I have. Only problem, the author uses a USB-serial adapter to talk to it, something I don’t have. But wait – I found a way of rigging the Arduino to act as such an adapter (previous post).

I saw somewhere, and confirmed (by using a tablet to scan for WiFi networks) that the default IP address for the Shiald is rather an obscure one, off my local subnet anyway. But a bit of googling gave me the info necessary to set the IP to something else.

After fiddling a bit with the baud rate, a little blue light started flashing next to the ESP8266 chip, and it worked!

In the IDE:

In the Serial console:

And in a browser! Woo-hoo!

Here’s my tweaked version of the script:

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>
const char* ssid = "AllPay Danny";
const char* password = "not this";
ESP8266WebServer server(80); // HTTP server on port 80

IPAddress ip(192, 168, 0, 14); // where xx is the desired IP Address
IPAddress gateway(192, 168, 0, 1); // set gateway to match your network
IPAddress subnet(255, 255, 255, 0); // set subnet mask to match your network

void setup() {
 Serial.begin(115200); 
 WiFi.disconnect(); // Disconnect AP
 
 WiFi.config(ip, gateway, subnet);
 
 WiFi.mode(WIFI_STA); 
 WiFi.begin(ssid, password); // Connect to WIFI network
// Wait for connection
 while (WiFi.status() != WL_CONNECTED) {
 delay(500);
 Serial.println(".");
 }
 Serial.print("Connected to ");
 Serial.println(ssid);
 Serial.print("IP address: ");
 Serial.println(WiFi.localIP());
server.on("/", [](){
 server.send(200, "text/plain", "Hello World");
 });
server.begin(); // Start HTTP server
 Serial.println("HTTP server started.");
}
void loop() {
 server.handleClient();
}

The blog post goes on to Part 2 Upload code to Arduino, which I’ll try next – when I’m properly rested 🙂

PS.

Just tried this Part 2 bit, essentially comms between Shiald & Arduino. It nearly worked :

Data received: . .
Conoected to AllPay Danoy
IP address: 092.168.0.04C!⸮⸮⸮⸮ٕɁ⸮⸮

I’ve read somewhere that the Software Serial struggles at high baud rates, and this example is using 115200 so presumably that’s the problem. Bit of tweaking required.

PPS.

I flipped the baud rate in the code based on that in the blog post to 9600, and with the Arduino as serial converter uploaded the new code to the Shiald (at 115200 baud), set as NodeMCU 1.0. Uploading took a good few attempts, but finally it worked.

I also changed the Arduino part of the code to use different ports :

#include <SoftwareSerial.h>
SoftwareSerial mySerial(2, 3); // RX, TX on Arduino
void setup() {
 Serial.begin(9600);
 mySerial.begin(9600);
}
void loop() {
 if (mySerial.available()) {
 String msg = mySerial.readString();
 Serial.print("Data received: ");
 Serial.println(msg);
 }
}

The wiring I now have as :

Arduino   | Shiald

GND       - Debug GND
+5v       - Debug 5v
Digital 2 - Digital 0
Digital 3 - Digital 1

The switches on the Shiald are at (1,2,3,4) On, On, Off, Off.

And finally, the Hello World is still visible on the IP address I set. And what’s more, in the serial monitor (now set to 9600 baud) I see:

Connected to AllPay Danny
IP address: 192.168.0.14
HTTP server started.

Yay! It all works.

So…

Next

First thing I should do is pull together all the various bits from the last post and this, with relevant material from linked pages, and write it up as a from scratch to here procedure. I won’t remember, and also anyone that buys the same boards will stand a chance of getting things going.

Then I need to think about what I’m going to do on the analog/sensor side. What I can do with the hardware I’ve got is fairly limited – a key factor being the speed of the data acquisition on the Arduino. But I should have the necessary for me to build something that operates end-to-end with essentially the same topology as my target design.

Regarding code on the Arduino & Shiald, the next steps will be to :

1. Get the data from the single Analog Input on the Shiald, buffer/filter it and expose it on a local web server. With a little analog pre-amp & filter this should be enough for a single-channel seismometer.
2. Do the code necessary on a regular computer to access and do something with the data from the web server on the Shiald.
3. Get the data from the 6 Analog Inputs on the Arduino, buffer/filter it, transfer it to the Shiald and again expose on a local web server. I might well try the analog bandpass filter idea mentioned in my previous post.
4. As 2. but for the 6 channels.

A global job to put together in parallel with the above is the code necessary for self-description of the units to provide status information alongside the data. RDF and Web of Things time!

So now I’ve got fairly fun jobs to get on with on every side of this project :

1. Sensor hardware
2. Arduino/Shiald software
3. Comms/post-processing software – I can get on with the Deep Learning bits using online sources, haven’t looked at that for weeks
4. Notification system – hook the Deep Learning bit output to Twitter

I may have to get the dice out…

//// note to self

danny@lappie:/dev$ esptool.py --port ttyACM0 --baud 9600 flash_id
esptool.py v2.2.1
Connecting........_____....._____
Detecting chip type... ESP8266
Chip is ESP8266EX
Uploading stub...
Running stub...
Stub running...
Manufacturer: c8
Device: 4016
Detected flash size: 4MB
Hard resetting...

Summary for my Mother

She doesn’t have a science/technology/coding background, but asked me. But she does understand purple prose, mailed her this:

This is my view : it has been observed that many earthquakes have been preceded by unusual electrical activity in the atmosphere, and this has been made evident in the Extremely Low Frequency/Very LF radio bands.

These happen to be the frequency bands that audio works at, so a lot of existing technologies can be applied to its analysis (although the signals are radio rather than acoustic). There are plausible scientific explanations for these phenomena – if virtually any kind of rock is heavily compressed or stretched it generates electricity, at least some of which is expressed as radio waves. Rocks get squeezed and pulled before a split. Research to date has generally been inconclusive, even negative on this having potential for predicting earthquakes.

The biggest problem in my opinion is that there is a massive amount of unrelated noise in the system – things like lightning strikes are very loud in the wavebands in question, let alone solar flares. But recent developments in machine learning, aka Deep Learning, have been very good at pulling out features of interest from a very confused situation.

My hypothesis is that modern neural nets will be able to detect patterns well enough (beyond what traditional techniques can find) to provide an early warning system. Don’t get me wrong, I’m only half convinced myself, but if there’s a 10% of the idea making sense, 10% chance of it being codable (that bit’s 100%, but hypothetically) and a 10% chance of being able to exploit it, I’ll be out in the garden in a tent and will live to talk bollocks again. Or come back in shame-faced – but still alive.

Data Sources

Two data sources are key to this project: natural VLF/ULF radio signals and seismic vibration. Other environmental signals have been examined in relation to earthquake prediction, several are described on Wikipedia. Three are of note here:

Prior seismic events – in the long term, patterns are recognisable in the progress of earthquake systems. In the short term, major events are often associated with numerous lower magnitude events, after the fact these get classified into foreshocks, mainshocks or aftershocks. Given that seismic signals will be captured by the system proposed here, no extra effort is needed to include these factors into analysis.

Radon gas emissions – while still the subject of ongoing research, spikes in geological radon emissions have been noted preceding earthquakes. Measuring the typical level of radon at a given location is relatively straightforward, by accumulating particulate matter at the location over weeks or months and then measuring its radioactivity, exposing the contribution of radon decay products. But continuous monitoring is much more involved given the low levels of radioactivity involved. Given this difficulty, the radon approach isn’t a high priority in this project. Maybe later.

Animal Behaviour – there is anecdotal evidence that animals exhibit unusual activity prior to an earthquake. This will be left out of scope here due to the difficulty in exploiting such a phenomenon (if it exists). Also, I have two dogs and they displayed absolutely nothing out of the ordinary prior to the last tremor we felt here, they remained asleep. During and after the tremor, they were perturbed…

PS. See Biological Anomalies around the 2009 L’Aquila Earthquake – a lot were reported.

There are additional environmental signals that may have little or no correlation with seismic events but might still improve prediction.

Earth’s magnetic field – the geomagnetic field changes over time and is linked to variations in geology, especially in the Earth’s core. The proposed VLF/ELF receivers will pick up rapid variations in this field, additionally given that it is relatively straightforward to detect ultra-low frequency changes, it will probably be useful to gather this data.

Static electrical field – similar to the magnetic field, the ‘static’ field at a given location can be seen as the ultra-low frequency aspect of signals that the VLF/ELF receivers will pick up. ‘Spherics’, the radio signals generated by lightning strikes are a major feature of VLF/ELF radio signals, and nearby thunderstorms strongly modify the local static electrical field. Again, such data may well be a useful contribution to the system, and measuring this field is relatively straightforward.

Solar and lunar cycles – these bodies exert significant gravitational forces over the Earth and hence are likely to be a factor in geological changes. It should be straightforward to include such data in analysis.

Sunspot activity – whether or not solar discharges have an influence on the occurance of earthquakes (it seems unlikely), they have a major impact on radio signals. It should be feasible to find an online source of related data and include it in the analysis.

Ambient temperature – the reception of radio waves is influenced by the weather and temperature is a key aspect. Also any other measurement circuitry will, to some extent, suffer from temperature drift. Being able to effectively subtract this factor from other signals may help improve results. Temperature sensors are relatively easy to set up, so measuring this seems worthwhile.

Acoustic noise – ie. ambient audio. As with temperature, changes here are is likely to be picked up to some extent by other sensors, especially the seismic subsystem. For example, tractors quite often pass by here, causing a deep rumble that is likely to be detected as a seismic signal. It will be useful to have the corresponding sound signal to help discount such artificial signals. Recording audio is simple so again this seems worthwhile.

[[TODO – move this paragraph to an overview page]] It should be noted that as no completely reliable prediction method has yet been discovered, the general attitude surrounding any approach is that of scepticism. This mood is amplified by the political aspects of prediction – action on a prediction (eg. mass evacuation) is liable to be expensive, so false positives will be costly. Additionally the lack of prediction advice when a major event has occurred can have significant fallout. A notable example being the charging of six Italian scientists with manslaughter following the devastating 2009 l’Aquila earthquake. They were subsequently acquitted (though a government official’s charges were upheld). That such charges were even considered shows the sensitivity in this field.

Neural Network Strategies

[work in progress 2017-04-15]

 

Provisional Architecture

 

 

Provisional Network Architecture

A Critical Issue

If we assume that significant seismic events are indeed preceded by distinctive radio emissions generated by geological stresses, and that these patterns may be detected reasonably reliably, there remains at least one significant issue in applying them to the prediction problem. This the time delay between the radio event and the seismic event.

Reports in the literature describe a latencies anywhere between hours and weeks, and it seems likely that this may confound the association between earthquakes and precursors. There could be any number of unknown variables in the physical environment affecting the generation and propagation of the radio signal. For all practical purposes these are effectively random, and so probably best considered another form of noise – in this case in the temporal dimension.

It may be that the radio signal patterns of precursors have recognisable correlation with not only the ensuing seismic events but the length of time before those events. Another (optimistic) possibility is that the neural networks applied will learn this correlation.

At this stage, the best bet seems experimentation…