How to push data to data.sparkfun.com for Internet of Things

How to push data to data.sparkfun.com for Internet of Things

This post shows how to connect Arduino platform to data.sparkfun.com for Internet of Things.
* Scope of post
* Internet Connectivity : W5500 Ethernet Shield
* Cloud Service : data.sparkfun.com
* Arduino IDE 1.7.6 (arduino.org); surpports Ethernet libraries for W5500
* HTTP Query

data.sparkfun.com

  • What is Phant?
    • Phant is a open source cloud server platform by powered Sparkfun Electronics.
    • Sparkfun created data.spartfun.com ,which is a free cloud service running phant. –
    • To collect data from your device to cloud service, you just need to register a new stream.
    • After register, you get two keys for accessing the data; one is q private key is required to update that stream, other is a public key grants access to any other stream on the service.
    • All communication with Phant is carried out over HTTP. So, your device should be acted as HTTP Client.
    • http://data.sparkfun.com/input/%5BpublicKey%5D?private_key=%5BprivateKey%5D&%5Bfield1%5D=%5Bvalue%5D&%5Bfield2%5D=%5Bvalue%5D

  • Phant : Phant.io

    Phant is a modular node.js based data logging tool for collecting data from the Internet of Things. It is the open source software that powers data.sparkfun.com, and is actively maintained by SparkFun Electronics. Phant is short for elephant. Elephants are known for their remarkable recall ability, so it seemed appropriate to name a data logging project in honor of an animal that never forgets.

Prepare materials

  • Hardware
    hardware//embedr.flickr.com/assets/client-code.js

  • Tool : Arduino IDE
  • Registration on data.sparkfun.com
    To create a data stream, head over to data.sparkfun.com, and click “CREATE”.

    • Create a Data Stream
      • New Stream example
        ![Registration2 @ data.sparkfun.com](C:\Users\root\Desktop\CC_AUG\FIG\EDITED\E_New Stream.png)

        • Fields – This comma-separated list of words defines data stream to post a list of unique values.
        • Stream Alias – This testbox defines domain name for you Data Stream
      • Create Data Steam: After creating a data Stream, you will confirm URL, Keys for accessing for your data stream.
        ![Registration1 @ data.sparkfun.com](C:\Users\root\Desktop\CC_AUG\FIG\EDITED\E_New Stream1.png)

Software

Codes flow

  • Configuration Arduino’s I/O pins

    • D3 – Active-low momentary button (pulled high internally)
    • A1 – Photoresistor (which is combined with a 10k resistor to form a voltage divider output to the Arduino).
    • A2 – Temporature Sensor (LM35)
  • Configuration Phat Stuff
    • Insert your publicKey
    • Insert your privateKey
    • Generat Fileds; ‘Files name shoud be same “field name” in Create Stream form’
  • setup()
    • Call Serial.begin(115200);
    • Setting Input pins
    • Call setupEthernet(): do DHCP Client and writing MAC Addrerss
  • loop()
    • If the trigger pin (3) goes low, send the data.
      • Get sensing datas by using analogread()
      • Call postData
        • Open socket as TCP Client
        • Try to connet TCP server (data.sparkfun.com); if needs, do DNS clinet for getting IP address of server
        • Make query string based on Phant frame
        • Send query
        • Check for a response from the server, and route it out the serial port.

Arduino’s I/O pins:

const int triggerPin = 3;
const int lightPin = A1;
const int tempPin = A2; 

void setup()
{
...

  // Setup Input Pins:
  pinMode(triggerPin, INPUT_PULLUP);
  pinMode(lightPin, INPUT_PULLUP);
  pinMode(tempPin, INPUT_PULLUP);
...
}

Phant Stuff

const String publicKey = "insert_your_publicKey"; 
const String privateKey = "insert_your_privateKey";

const byte NUM_FIELDS = 2;
const String fieldNames[NUM_FIELDS] = {"lux", "temp"}; // Fileds shoud be same "field names" in Create Stream.
int fieldData[NUM_FIELDS];

Make Query string over HTTP

client.print("GET /input/");
client.print(publicKey);
client.print("?private_key=");
client.print(privateKey);
for (int i=0; i<NUM_FIELDS; i++)
{
  client.print("&");
  client.print(fieldNames[i]);
  client.print("=");
  client.print(fieldData[i]);
}
client.println(" HTTP/1.1");
client.print("Host: ");
client.println(server);
client.println("Connection: close");
client.println();

Phant_Ethernet.ino

/*****************************************************************
Phant_Ethernet.ino
Post data to SparkFun's data stream server system (phant) using
an Arduino and an Ethernet Shield.
Jim Lindblom @ SparkFun Electronics
Original Creation Date: July 3, 2014
Roy Kim(Soohwan Kim) embeddist@gmail.com
Modified DateL August 26, 2015
S

This sketch uses an Arduino Uno to POST sensor readings to 
SparkFun's data logging streams (http://data.sparkfun.com). A post
will be initiated whenever pin 3 is connected to ground.

Before uploading this sketch, there are a number of global vars
that need adjusting:
1. Ethernet Stuff: Fill in your desired MAC and a static IP, even
   if you're planning on having DCHP fill your IP in for you.
   The static IP is only used as a fallback, if DHCP doesn't work.
2. Phant Stuff: Fill in your data stream's public, private, and 
data keys before uploading!

Hardware Hookup:
  * These components are connected to the Arduino's I/O pins:
    <Original>
    * D3 - Active-low momentary button (pulled high internally)
    * A0 - Photoresistor (which is combined with a 10k resistor
           to form a voltage divider output to the Arduino).
    * D5 - SPDT switch to select either 5V or 0V to this pin.
    <Modified>
    * D3 - Active-low momentary button (pulled high internally)
    * A1 - Photoresistor (which is combined with a 10k resistor
           to form a voltage divider output to the Arduino).
    * A2 - Temporature Sensor (LM35)

  * A CC3000 Shield sitting comfortable on top of your Arduino.

Development environment specifics:
    <Original>
    IDE: Arduino 1.0.5 
    Hardware Platform: RedBoard & PoEthernet Shield
    <Modified>
    IDE: Arduino 1.7.6 
    Hardware Platform: Arduino DUE & W5500 Ethernet Shield

This code is beerware; if you see me (or any other SparkFun 
employee) at the local, and you've found our code helpful, please 
buy us a round!

Much of this code is largely based on David Mellis' WebClient
example in the Ethernet library.

Distributed as-is; no warranty is given.
*****************************************************************/
#include <SPI.h> // Required to use Ethernet
//#include <Ethernet.h> // The Ethernet library includes the client for W5100
#include <Ethernet2.h> // The Ethernet library includes the client for W5500
//#include <Progmem.h> // Allows us to sacrifice flash for DRAM //@Modified: Don't use

///////////////////////
// Ethernet Settings //
///////////////////////
// Enter a MAC address for your controller below.
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
//IPAddress server(54,86,132,254);  // numeric IP for data.sparkfun.com
char server[] = "data.sparkfun.com";    // name address for data.sparkFun (using DNS)
// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192,168,0,177);

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

/////////////////
// Phant Stuff //
/////////////////
const String publicKey = "6JZbNolApzF4om2l9yYK";
const String privateKey = "Ww0vPW1yrkUNDqWPV9jE";

const byte NUM_FIELDS = 2;
const String fieldNames[NUM_FIELDS] = {"lux", "temp"};
int fieldData[NUM_FIELDS];

//////////////////////
// Input Pins, Misc //
//////////////////////
const int triggerPin = 3;
const int lightPin = A1;
const int tempPin = A2; 

float tempC;
int reading;
//String name = "Ether-anon";
String name = "Roy";
boolean newName = true;

void setup()
{
  Serial.begin(115200);

  // Setup Input Pins:
  pinMode(triggerPin, INPUT_PULLUP);
  pinMode(lightPin, INPUT_PULLUP);
  pinMode(tempPin, INPUT_PULLUP);

  // Set Up Ethernet:
  setupEthernet();

  Serial.println(F("=========== Ready to Stream ==========="));
  Serial.println(F("Press the button (D3) to send an update"));
#if 0 // don't use
  Serial.println(F("Type your name (no spaces!), followed by '!' to update name"));
#endif
}

void loop()
{
  // If the trigger pin (3) goes low, send the data.
  if (!digitalRead(triggerPin))
  {
    // Gather data:
        fieldData[0] = analogRead(lightPin);
        fieldData[1] = analogRead(tempPin);
    //fieldData[2] = name;

    Serial.println("Posting!");
    postData(); // the postData() function does all the work, 
                // check it out below.

    delay(1000);
  }
#if 0 // don't use
  // Check for a new name input:
  if (Serial.available())
  {
    char c = Serial.read();
    if (c == '!')
    {
      newName = true;
      Serial.print("Your name is ");
      Serial.println(name);
    }
    else if (newName)
    {
      newName = false;
      name = "";
      name += c;
    }
    else
    {
      name += c;
    }
  }
#endif
}

void postData()
{
  // Make a TCP connection to remote host
  if (client.connect(server, 80))
  {
    // Post the data! Request should look a little something like:
    // GET /input/publicKey?private_key=privateKey&light=1024&switch=0&name=Jim HTTP/1.1n
    // Host: data.sparkfun.comn
    // Connection: closen
    // n
    client.print("GET /input/");
    client.print(publicKey);
    client.print("?private_key=");
    client.print(privateKey);
    for (int i=0; i<NUM_FIELDS; i++)
    {
      client.print("&");
      client.print(fieldNames[i]);
      client.print("=");
      client.print(fieldData[i]);
    }
    client.println(" HTTP/1.1");
    client.print("Host: ");
    client.println(server);
    client.println("Connection: close");
    client.println();
  }
  else
  {
    Serial.println(F("Connection failed"));
  } 

  // Check for a response from the server, and route it
  // out the serial port.
  while (client.connected())
  {
    if ( client.available() )
    {
      char c = client.read();
      Serial.print(c);
    }      
  }
  Serial.println();
  client.stop();
}

void setupEthernet()
{
  Serial.println("Setting up Ethernet...");
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println(F("Failed to configure Ethernet using DHCP"));
    // no point in carrying on, so do nothing forevermore:
    // try to congifure using IP address instead of DHCP:
    Ethernet.begin(mac, ip);
  }
  Serial.print("My IP address: ");
  Serial.println(Ethernet.localIP());
  // give the Ethernet shield a second to initialize:
  delay(1000);
}

Demo:

Serial Monitor

  1. DHCP Clinet message
  2. Press the button to send query to server, then postData() is called.
  3. Confirm the response message
    office monitoring@ data.sparkfun.com

https://data.sparkfun.com/office_monitoring

Advertisements

mbed RPC with W5500 Ethernet Shield

This post shows how to use mbed RPC with W5500 Ethernet Shield and how to port an Ethernet application used lwIP to W5500Interface.

Remote Procedure Call

In computer science, a remote procedure call (RPC) is an inter-process communication that allows a computer program to cause a subroutine or procedure to execute in another address space (commonly on another computer on a shared network) without the programmer explicitly coding the details for this remote interaction.
http://en.wikipedia.org/wiki/Remote_procedure_call

RPC
*source – http://uw714doc.sco.com/en/SDK_netapi/rpcpD.how_RPC_works.html

HW – FRDM-KL25Z + W5500 Ethernet Shield

FRDM-KL25Z + W5500 Ethernet Shield
http://developer.mbed.org/platforms/KL25Z/
http://developer.mbed.org/components/W5500-Ethernet-Kit-for-IoT/
HW connection : D10 – SCS / D11 – MOSI / D12 – MISO / D13 – SCLK

SW – mbed RPC

Server that executes remote procedure call (RPC) commands through HTTP.
mbed RPC @developer.mbed.org/handbook
In RPC libaray, a simple HTTP Server can execute RPC commands sent from HTTP Client.
This library uses EthernetInterface (lwIP) and mbedOS.

RPC command

The RPC command is encoded in this way :

  • container that wraps a skeleton’s ID
  • method that will be invoked
  • parameters that will be inputted (optional)
    • Command examples
      PUT command : “/DigitalOut/new?arg=LED2&name=led2”
      GET command : “/led2/write?arg=1”

Request handlers

To process requests, the server relies on RequestHandler. Each RequestHandler is assigned to a request type. Each type of request is assigned to a certain role :

  • PUT requests to create new objects
  • DELETE requests to delete objects
  • GET requests to call a function of an object

Porting EthernetInterface to W5500Interface

Here is a guide how to modify :

  • Folk HTTP-Server in you program workspace
  • Delete 2 Folders; EthernetInterface(lwIP) & mbed-rtos in HTTP-Server
  • Added array for MAC Address which was written at Source Hardware Address Register in W5500.
    uint8_t mac_addr[6] = {0x00, 0x08, 0xDC, 0x00, 0x01, 0x02};`
    
  • Change EthernetInterface() and set SPI Interface for W5500
        /*  Use EthernetInterface
         *EthernetInterface eth;
         *if(eth.init())
         *int ret = eth.init("192.168.77.34", "255.255.255.0", "192.168.77.1");    
         */
        /* ----- Use W5500 Ethernet Shied -----*/
        SPI spi(D11, D12, D13);      /* mosi, miso, sclk */
        //spi.frequency(12500000); /* Optional : set proper SPI clock */
        EthernetInterface eth(&spi, D10, D9); /* spi, cs, reset(dummy) */
        ...
        /* ----- Use W5500 Ethernet Shied -----*/
    
  • Set Network configuration: I will use fixed Address.
        /*  in case of using W5500 Ethenret Shield */
        int ret = eth.init(mac_addr, "192.168.77.34", "255.255.255.0", "192.168.77.1");    
        /* in case of using W550io (has a unique real MAC address) */
        //int ret = eth.init("192.168.77.34", "255.255.255.0", "192.168.77.1");   
    

Repository : Users » embeddist » Code » HTTP-Server_W5500Interface

http://developer.mbed.org/users/embeddist/code/HTTP-Server_W5500Interface/

Demo – Using a browser for HTTP Client

Here is a quick guide how to run this program :

  • Compiles this program and copies it to the mbed
  • Open TeraTerm (install it if you don’t have it), select serial and choose the port named “mbed Serial Port”
  • Reset your mbed
  • The IP address should appear in teraterm. In this example, I will use 192.168.77.34
    Terminal
  • Open your browser and go to http://192.168.77.34
    connecttingl
  • If everything is ok, you should see a webpage.

  • Create Red LED as arg=LED1 and name=RLED
    RGB LED
    Create Red LED

  • To procedure, send command: RLED/write 0
    Reset Red LED

  • To procedure, send command: RLED/write 1
    Reset Red LED

SDFileSystem for mbed platfrom

This post shows how to use SDfile System on W5500 Ethernet Shield for mbed platfrom.

SDFile System

A library to allow SD Cards to be accessed as a filesystem, using a SPI interface

SDFileSystem @developer.mbed.org/handbook

This library supports:
– Fat12/ FAT16 / FAT32
– SD / SDHC cards up to 32Gb

HW – W5500 Ethernet Shield with MicroSD Slot

W5500 Ethernet Shield

W5500 Ethernet Shield

  • ARM mbed compatible operation
  • Arduino Pin-compatible
  • Ethernet (W5500 Hardwired TCP/IP chip)
  • MicroSD Slot
  • Ethernet port
  • I2C I/F
  • UART I/F

W5500 Ethernet Shiel Pin Map

W5500 Ethernet Shiel Pin Map

  • SPI Bus support Arduino and mbed platfrom
    MOSI MISO SCK
    D11 D12 D13
    MOSI(ICSP-4) MISO(ICSP-1) SCLK(ICSP-3)
  • SPI_CS and SD_CS pins are allowed to select as below,

    • SPI_CS : D8 / D9 / D10
    • SD_CS : D4 / D5 / D6

SW1 – SDFile System for LPC824 Xpresso (mbed platform)

/**  import SDFiles System Library **/
#include "mbed.h"
#include "SDFileSystem.h"

/*for LPC824 Xpresso */
SDFileSystem sd(D11, D12, D13, D6, "SD"); // the pinout on the mbed Cool Components workshop board
int main() {
    printf("Hello World!n");   
    mkdir("/SD/mydir", 0777);
    FILE *fp = fopen("/SD/mydir/sdtest.txt", "w");
    if(fp == NULL) {
        error("Could not open file for writen");
    }
    fprintf(fp, "Hello fun SD Card World!");
    fclose(fp); 
    printf("Goodbye World!n");
}

SW2 – SDFile System for FRDM-KL25Z (mbed platform)

/**  import SDFiles System Library **/
#include "mbed.h"
#include "SDFileSystem.h"

/*for FRDM-KL25Z */
SDFileSystem sd(D11, D12, D13, D4, "SD"); // the pinout on the mbed Cool Components workshop board
int main() {
    printf("Hello World!n");   
    mkdir("/SD/mydir", 0777);
    FILE *fp = fopen("/SD/mydir/sdtest.txt", "w");
    if(fp == NULL) {
        error("Could not open file for writen");
    }
    fprintf(fp, "Hello fun SD Card World!");
    fclose(fp); 
    printf("Goodbye World!n");
}

Demo.

Notice: SD card name shoud be same variable in sd(mosi, miso, clk, “SDname”).
* Serial terminal
minicom
* Check SD card by using card reader
read SD card
* Confirm stdtest.txt
read SD card