I didn’t really know where we were tracking this stuff so i have made a post.
The current status is that Danny has ordered the parts (battery, esp32 and air sensor). I think hooking them up is pretty easy. I was having a look at how to publish the data and the airapperent guys piggy back off a site called luftdata.info. I like how literal the germans are since the site is airdata.info (99 luft ballons).
It looks like it’s a platform.io project (ive not used this before) and it should be simple enough to build and deploy. Also in the readme they say they have gps. I don’t really think it matters but if we want one i have a spare one at home for the prototype.
I’ve had a look at the Luftdaten API - definitely possible to fire up a local version to test against.
(I’ve got a version running using docker-compose - it’s not entirely self contained as you need to manually initialise the DB via command line, but that’s workable with)
@Al_B - The API’s a Django app so you might be familiar with the structure (also uses Redis cache + Postgres db) with some minimal web UI with basic API tests built in.
I haven’t figured out the default login yet and have only started playing with it via the test pages in the app…
…Created my own admin user with CLI: python manage.py createsuperuser --username admin
then (in the browser) logged in and went to /admin/ to start setting things up
Not looking great - the web UI doesn’t match the database schema setup in the docker/ docker-compose definitions.
I applied the below to the database to get the pages working:
ALTER TABLE sensors_sensorlocation ADD COLUMN altitude INTEGER NULL;
ALTER TABLE sensors_node ADD COLUMN indoor BOOLEAN NULL;
ALTER TABLE sensors_node ADD COLUMN inactive BOOLEAN NULL;
ALTER TABLE sensors_node ADD COLUMN exact_location BOOLEAN NULL;
but there’s still issues - can’t save a location, so can’t add a sensor definition to submit data against…
Web forms:
/admin/sensors/sensorlocation/add/ - (required to allow a node + sensor to be added via admin) fails with an integer validation error on fields that aren’t integers…
/admin/sensors/sensortype/works
/adddevice/appears to be a public-facing device enrollment form - requires at least one entry to be added via /admin/sensors/sensortype/, but other than that it looks like it’s supposed to add the rest for you… …but it falls over with a Django No URL to redirect to. Provide a success_url. error when you submit the form.
Could have saved some time looking at the build errors on that repo: Travis CI - Test and Deploy with Confidence
I don’t think this repo can match what’s running live as it’s broken, which is unfortunate.
we may just need to submit the data to their live system if they let us enroll a sensor.
Got something working by rolling back (git checkout 0b45aa6) to this commit that was identified as last good build - build history
and the admin site started working (I still had to set up the database and create the admin user, but at least something worked)
I can create sensor data entries via the admin, but the webform for testing this isn’t working (pretty sure the API is working though - I’ll try to test that at some point so we can do some offline testing)
Normally for something like this there is a public schema so you know what data to publish. Im surprised they done have one for their main site.
If the current master is broken i wonder which version they are actually running in production and if the data schemas match.
Lufdaten are really set up for using their firmware and their live API/separate meine.luftdaten.info site rather than coding from sctatch/testing etc. Their API repo doesn’t give the whole picture
I’m leaning towards testing with OpenSenseMap, so I’ll try and fire up their API.
We can always just point the test devices at live API’s, but it feels a bit wrong…
The open sense api is much more what I was expecting. I would have thought they had a test api to allow submission of test data.
I looked at the luft data it looks like the firmware is not designed to be changed and only configured. If it works it will be simple of not its probably going to be a nightmare. It’s probably not that much work to just try it.
Is the open sense api another access point to luft data or something else entirely?
Only “Own API” and InfluxDB allow a server to be specified in the web GUI in the firmware.
I’ve managed to get the Firmware onto an ESP32 - ended up going via Arduino as their firmware flasher left it in a boot-loop.
Needed some tweaks, so not ideal, but workable.
(no sensors attached yet, not pointed at an API yet either)
This might be what we use on Wed’s as the data sink/visualisation to prove the sensors are working.
(I’ve ordered an extra NodeMCU ESP8266 and a DHT22 for a belt & braces approach to having kit to run with, but we’re likely to get the ESP32 running anyway)
When I got home from the shed tonight there was a short piece on Ian Robinson and air quality monitoring in Avonmouth and Bristol on BBC R4 Costing the Earth. It’s available on demand from BBC Sounds. The clip is about 20 mins in at 21:20.
Ok, after frustrating myself with wifi and other issues during the session and making little progress, there were really only 2 things I had to fix (aside from decent Wifi):
The DHT22 sensor was fine with a simple Arduino example - the (Adafruit) DHT library in the firmware project is old - updated it and updated the code that reads the sensor - now getting readings.
In my original patches to get this running on an ESP32, I’d set it up to use the “WifiSecureClient” - that meant it was expecting an SSL connection, which I’ve not got setup on my dev instance of InfluxDB - switched to “WifiClient” (and added a specific debug message for connection failure) and it all started working!
First data from an ESP32 with DHT22 (temp/humidity) sensor running (modified) Luftdaten firmware -> InfluxDB -> Grafana =
As I said in the session: “some sensors do work, some of the time”
Other notes
the Arduino serial monitor does pick up debug messages from the Luftdaten firmware
we really need a reliable Wifi router (that supports 2.4GHz so ESP’s can connect to it) - I’m pretty sure we’d have solved some of this at the Shed tonight if not for the Wifi Issues
I’ve still not had a working firmware flash from the Luftdaten flashing tool to ESP32 or ESP8266 on a Mac…
Still need to get the Particulate sensor working
Part of the reason for an ESP32 instead of an ESP8266 was firstly to make use of the better deep-sleep mode, and secondly to open up potential for live Bluetooth readings for people in the immediate vicinity & potentially other, related interactive stuff, but the Luftdaten FW is likely to make that difficult, so we’ll see where we get to and what’s viable (the power usage difference may limit options with the ESP8266, but non-standard firmware for the ESP32 could cause headaches later)…
There’s an option just to write a (massively simplified) firmware that just reads the sensors and sends the data. The Luftdaten FW has a bunch of “fleet management” features for updates etc. and supports a bunch of stuff we’re not using…
I had another go at testing the SDS011 particulate sensor.
Tried with simple serial tests and with libraries for that sensor - no joy.
I’ve ordered another (should be there for this session) to see if we’ve got a dud + found a £10 testing board from China that will take longer to get here, but seems worthwhile to make sure we don’t waste effort on bad hardware.
I think the ESP32 setup might be a non-starter if we’re using the Luftdaten firmware (unless there’s a very specific board, but then we’d have to look into the low power mode on there, which may mean modifying the FW).
The ESP8266 I had last week was a bit weird - i don’t think it was the same model as advertised (wouldn’t flash from the Luftdaten tool).
Used a proper NodeMCU ESP8266 and the firmware flasher works, as does the particulate sensor, which is good as I was wondering what the hell was going on.
If we wanted to do something with ESP32s for low power mode or BLE access, we’d need to go our own way, but for now we can work with the ESP8266 and see the data via InfluxDB+Grafana, or any of the other options in the Luftdaten FW.
p.s. need to re-test the original dust sensor as I was testing only with new sensors to rule out the first one being a dud.
While looking at suitable enclosures for the sniffer, I found a blog which mentions power supply problems with the sensor board. It suggests making a carrier board to ease the process of directing the power to the best terminals. This may be a solution if power is still a problem… Will?
The blog is here: https://www.robmiles.com/journal/2019/04/27/psu
It’s worth noting that the current design (from the Luftdaten project) is primarily about particulate matter rather than Gas sensors like NO2 (nitrogen dioxide), CO (carbon monoxide) etc. to measure those, we’ll need to either work on a different setup or try to contribute to the Luftdaten project to support other sensors.
Supported sensors in Luftdaten/AirRohr (Firmware version: NRZ-2019-125-B1):
SDS011 (particulate matter)
Plantower PMS(1,3,5,6,7)003 (particulate matter)
Honeywell PM (particulate matter)
Sensirion SPS30 (particulate matter)
PPD42NS (particulate matter)
DHT22 (temperature, humidity)
HTU21D (temperature, humidity)
BMP180 (temperature, air pressure)
BMP280/BME280 (temperature, humidity, air pressure)
DS18B20 (temperature)
DNMS (LAeq - sound/noise levels)
GPS (NEO 6M location)
Another temperature/humidity sensor has been added since that version, but no Gas sensors built into it at the moment.
I’m not sure whether the in-car pollution levels in the Guardian article are gases or particulate (or both) it’s pretty wooly on what was measured (it only mentions NO2 - the pollutants may be correlated, but they don’t really mention it & it’s important to know what’s filtered and what’s not etc.).
I think we’ll need to be clear about what we’re measuring in any stuff we put out.
A few links with a bit more detail:
