Environment Readings – Server and Client

Goal:

Reading the environment sensor data of the noise level and soil moisture level for a plant.

Sensors used

WINGONEER 5PCS Water Level Sensor Liquid Water Droplet Depth Detection Sensor

DEVMO 5PCS Microphone Sensor High Sensitivity Sound Detection Module Compatible with Arduino PIC AVR

Process: getting sensor date from Arduino is working

  1. circuit
  2. soldering
  3. Arduino Library downloads
  4. Arduino code combination
  5. Arduino with WifiNiNA

Schematics – to be uploaded

Server

Getting a database from Digital Ocean

Followed making the virtual host from digital ocean

helpful tutorial to assist the tutorial above

How to Create SSH Keys with PuTTY on Windows

It takes a while to create the key tho. Then I realized that I need to move my keys lol to generate the randomness.

generated the public key for digital ocean SSH

questions for office hour

  1. can not see the website from the droplet with the adduser? How to see the web page to know that the link would be the server address?
  2. in that way i can upload the data for it?

Solution

need to mkdir in the home folder and fork the NodeExamples

succeed in getting something shown in my digitalocean ipv4 with the port 8080

discovered that windows do not like single quotes. should replace ” with “”.

experimenting with glitch node express to understand how to post sensor data to server before using the digital ocean server

Ran into problems of showing status code of -2

can not get the posts from sensor data from arduino in glitch

when i changed the String path to “/posts”, finally it shows something new

I then changed the port 443 to 3000 since thats the one for glitch – does not work

For office hour

open glitch web and sensorpost_3

questions for office hour

  1. whats the flow of making digital ocean as the server? – should i use ngix?
  2. how to post the arduino data to server – can glitch be one?
  3. how to do so with digital ocean?
  4. would i be creating my own ssl certificate with digital ocean?

What I learned

if you close the terminal after you started it, it will close the server

to have server always running

making things talk – forever or pm2 – they keep a log

sudo npm install forever -g

forever start script.js

forever log or forever list

If I want to schedule to close it after running for a while, I can use crontab. Or if I want to avoid using forever, I can get the process id from terminal outside of the folder of the script.js and kill the process number

Tuesday night to do before ddl

  1. make vscode remote following this
  2. finish writing the posting code
  3. schedule with forever and crontab
  4. getting the data for the web interface

set up ssh key

tried multiple times still show the error below

succeeded in posting the data to the server

However, i can not see when accessing the /data

Do not know how to get the data to show in the website if I were to fetch it.

Should I make it in client.js?

I then created an array for the data

Still, not showing on the get part

It is working. But it only shows the latest data rather than the collection of data.

But it is only showing one value. I think it is because I am sending the content from post which only is a pair of digits

showing micVals in an array works on “/data”;

From office hour with Tom:

To do

  1. learn mysql
  2. create database and check out Don’s repo to learn to deal with the database.
  3. Experiment with the cheatsheet to see how to access the data from the API and then replace the API in the app.js from the repo.

form the right sequel query before writing the code

Working on SQL

  1. Following the tutorial here
  2. Went into problems

Then fixed it by following this tutorial

I used the

SHOW VARIABLES LIKE 'validate_password%';

SET GLOBAL validate_password.policy=0;

Steps to check the table

to install mysql node_modules, need to use

sudo npm install -g mysql

got my user from mysql

I deleted the root user since it gives me error saying the access was denied when I was asked with the password

deleted the root@localhost

having trouble logging onto mysql agin

update and upgrade the apt install

I followed the following tutorial to uninstall mysql

sudo apt-get remove --purge mysql* 
sudo apt-get autoremove 
sudo apt-get autoclean

it is working now.

created a new user name “home”@ “localhost”

server is up and running. For accessing the user version, use code

mysql -u home -p

An alternative to create the user woud be follow Don’s repo here and copy and paste that in the terminal.

When I copy and paste the code from Don’s repo to the terminal, I get to see the conn-dev database in the databases.

  1. Created a table, how should I access the data from Arduino?
  2. what did they use to constantly update the data?

OH questions:

  1. node app.js is not working
  2. how to create the .env file?
  3. .env is created – if were to see, should use the ls – al to see all the hidden files. Files beginning with “.” are hidden files.
ls -al
  1. should I create a table first and use the app.js to read the data from Arduino and post the data in the table?

OH solutions

  1. the app.js is actually running but it just does not console.log anything when it is running.
  2. The dummy macAddress and sessionKey are placeholders for the program. For my application, I do not actually need them. The problem of “unauthorized” went away when I cleaned the code of the session key and mac address.

MySql

Cheatsheet

I created mysql schematics as below. Detailed tutorial can be found here.

describe data (name of the table);

When I write below, I get a sample of what I inserted.

select * from data;

Also ran into problem saying that the address is in use. Turns out I have two applications using the address and app.js. So I typed as below:

sudo lsof -i :8081

Based on the PID, I followed the example

kill -9 {PID}

When I ran the code to see the program running the port 8081, it shows nothing. Problem solved.

I found a helpful tutorial to write SQL with Node

I deleted the parts of code I do not need and combined the code that was working in the app2.js.

The get is working but I was waiting for the POST from Arduino to work here. Still needs to figure that out.

Next Steps

  1. Make sure the code for the POST works
  2. add forever to update the code to the mySQL
  3. create simple web interface

Questions:

How to solve this?

quick ways to access the data from conn_dev in the database of sql:

 SHOW DATABASES; USE conn_dev; SHOW TABLES;
 SELECT * FROM data;
 INSERT INTO data (micVal, waterVal) VALUES (20,100);

Now I finally have post information but there is an error

I realized that I need to do things as below in mysql:

  • ‘root’ would be the username I have for my user.
  • ‘password’ would be the password I made for my sql.
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password' 

then

flush privileges;

then quit sql

quit;

In your MySqlServer folder, write

source .env

then

node app2.js

Different error yay!

After adding the params in the posting code

It is writing the data in my sql! I am so happy!

In order to get, I will use the same syntax to get the table as below;

successful in getting the data

writing more get functions to get individual data

successful in getting individual data

Following the tutorial for forever

Forever restart

It is working. The video shows the forever starting in the background adding the data to the sql database.

Inspired from the timestamp

From the friendsgroup, I should use the time stamp. Check the tutorial here.

SELECT mac_address,recorded_at,data_point
FROM readings s1
WHERE recorded_at= (SELECT MAX(recorded_at) FROM readings s2 WHERE s1.mac_address = s2.mac_address)
ORDER BY mac_address, recorded_at;

created_at start time:

2020-12-01 13:57:04

Using the crontab to schedule

https://crontab.guru/#0_8_30_1_*

how to do the basic oauth for the weather api

Things to get for the time range

SELECT created_at FROM data WHERE created_at BETWEEN '2020-12-01 15:00:00' AND '2020-12-01 17:30:00';

SELECT created_at FROM data WHERE created_at BETWEEN ${from} AND ${to};

trying to use the PUT

time change

to set the time range, I followed Name’s code for Weather Band API by using the moment library

I was struggling to get the date working but now I figured that out!

http://68.183.121.203:8080/data?from=2020/12/01?to=2020/12/02

Determine the size of the query

SELECT id FROM data WHERE id BETWEEN 2 AND 10;

Select the recent data from the sql database

SELECT * FROM data ORDER BY id DESC LIMIT 10;

Following the transationID

based on the id, select the data

deleted the transaction ID.

http://68.183.121.203:8080/data/3

kill one forever

changed the api for the id

wrong with calling with the time range

forever is restarting and running

OH questions

  1. how to set the size
  2. how to set the time range – it does not work that well?
  3. will forever automatically update the app3.js once i edit and save after starting forever

SELECT * FROM data WHERE created_at BETWEEN '2020-12-02 02:50:29' AND '2020-12-02 02:50:49';

If i were to create the range between id number

stopped at 5 pm wednesday – when i stopped the wifi, it stops recording?

how to solve it?

shoud decode in app3.js and then

Checking to connect to network for Arduino to WIFI – Tom’s Wifi 101 example

browser link for time: 
http://68.183.121.203:8080/data/date?from=%272020-12-01%2011:00:00%27&to=%272020-12-01%2013:00:00%27

NO need to single quotes

BROWSER working 
http://68.183.121.203:8080/data/date?from=2020-12-02%2013:20:00&to=2020-12-02%2015:20:00

${} does not work for me

Simple way of combining string works

quoting single quote using [tutorial]

\'

I finally got SSH to work

Followed the tutorial here and here.

I made a website and it is here. You can check the size as well as the time range of the data. The client side is quite similar to another project I was working for weather band API for the same class. You can check my blog here.

CSS generator

I used several generators to speed up my process.

Some examples I used include

query is slow – not enough space?

dashboard on digital ocean – if the cpu and memory is on top of the level, then 5 dollar digital ocean one is not enough

send email notification when the system is down?

node js script – go through the query – runs via crontab every 10 minutes

for weather band

replace the macaddress with location

Published by Yiting Liu

NYU ITP '21

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create your website with WordPress.com
Get started
<span>%d</span> bloggers like this: