Serverless IoT with CouchDB and the ESP8266
I’ve recently dipped into IoT and was overwhelmed with the options from companies such as Amazon, IBM and Microsoft. They have great solutions for big corporate projects but seem to need a lot of configuration and knowledge setting everything up. With my recent experience in CouchDB and serverless applications I decided to try something new: Serverless IoT. Is it possible to capture sensor data and plot the data on a chart with just an instance of a CouchDB database running?
The hardware used for this project is the WEMOS D1 mini Pro with a WEMOS DHT Shield. The D1 mini Pro is based on an ESP-8266EX, the DHT shield is based on an DHT11. To enable the device from waking itself up from deep sleep, the GP16 pin (D0 on the WEMOS device) is connected to the RST pin. The final piece is a battery connected to the device so that everything can work wireless.
The firmware used for this device is NodeMCU. Building a firmware image can be done by using the NodeMCU custom builds tool. I included the following modules in my build:
dht, file, http, net, node, tmr, wifi, tls
TLS can be included by using the TLS option in the builder. It is necessary for certain CouchDB providers such as Cloudant.
The ESP8266 can be flashed using esptool.py. I used the following command to do so:
- Connect to WiFi
- If successful, read DHT sensor
- If successful, do HTTP POST request with sensor data to server
The script does not have a while loop or repeat flow. It ends after sending the data. After waking up, the EPS8266 is reset and we automatically start at the beginning of the script.
Safety first, so use HTTPS where possible and create three user accounts:
- admin, for administrative purposes (write and read permissions)
- sensor, for storing sensor data (write permission only)
- client, for reading sensor data (read permission only, doesn’t necessarily need a password)
Create a database named
measurement. You would figure that saving the sensor data is as easy as posting it directly to this database doing an HTTP POST request from the ESP8266 but that does have a little problem.
The problem being that we need a timestamp on every sensor value. The EPS8266 could connect to an NTP server to get the current time but for a battery powered device that would be a very costly operation. Another approach is to use an update handler and let CouchDB append a timestamp to each document inserted in to the database.
By doing an HTTP POST request to the following URL we can add a document:
An example of a document we can post:
As stated earlier, I only want to run a CouchDB instance to manage the project. Luckily, we can serve the client from CouchDB by inserting the HTML page as an attachment. Yes, serving HTML straight from the database! As Nolan Lawson mentioned:
But for all its developers’ humility, CouchDB is a really exciting technology. When you step back and look at it, it’s a daring, crazy proposition, a bold statement about how awesome web development would be if we could just let it be the web. It’s a raving streetside lunatic, grabbing random people by the shoulders and screaming at them with frantic urgency: “We don’t need the server anymore! We only need the database! The database is the server!”
To serve the client as HTML from the database, create a design document in the measurement database using the provided JSON. After creating the document, edit it and add the client HTML file as an attachment.
The client is now accessible from the following URL:
The last thing we need is a rewrite rule to make the URL prettier. CouchDB uses HTTP Rewrite Handler for redirecting requests to make it easier for the end user to type the URL of your application. Edit the design document we just created and add the rewrites property. Your final document should look like what is shown below.
The client is now available on:
The final URL is still a bit ugly, so to finish everything off we use Virtual Hosts. Add a virtual host for the URL above to forward requests. In my case I used a subdomain and added a CNAME record towards my Cloudant account.
The client is now available at:
The database API is available at:
Wrapping it up
We ended up with a solution that uses CouchDB and CouchDB only for storing sensor data, retrieving data and displaying it on an HTML page. We used Design Documents, Update Functions, HTTP Rewrite and Virtual Hosts to achieve those tasks. Instead of the route of Client » Application Server » Database Server we go straight from the client to the database and put the business logic inside of it. We also serve the HTML from the database instead of a traditional HTTP server.
It’s a different way of thinking, very suitable for some projects but not for all.
A couple of interesting articles for further reading:
- CouchDB doesn’t want to be your database. It wants to be your web site.
- What is CouchApp?
- Couchapps, revisited
- Pretty URLs with Cloudant
- Rendering Content Based-On Multiple Documents with List Functions
- Flashing the firmware - NodeMCU Documentation
- ESP8266/NodeMCU Deep Sleep