Articles

Reinforcement Learning Autonomous Navigation Robots

01/02/2025

▼

github RL_robot_navigation

First approch Neural Networks and Robots

01/02/2017

▼

First approach to neural networks for robots

This guide shows how to build a robot with Raspberry Pi zero or a board H3 Allwinner with the addition of some sensors using openFrameworks, dlib and ofxGPIO as a development environment, to build a robot and software for remote control and showing a first approach to face recognition and imagenet predict with machine learning DLIB and openFrameworks

The programming of the robot is very simple, all transactions will be done on the remote PC that deals with controlling the robot via TCP protocol, the only operations carried out by the robot are: open a two-way socket for control of the motors and the receipt of sensor data as MPU or compass open a second socket that deals with the streaming video webcam.

The basic components for this project:

PC or PC Portable (with a good video card)
Raspberrypi zero or orangepi one H3
Dongle Wifi
Motore Stepper bipolar NEMA-14
sn754410ne H-Bridge
MPU6050 or HMC5883L (only compass)
Camera module with cable raspberry pi zero or (USB OTG webcam)
Pair of wheels Pololu 70mm
Battery Samsung EB-L1G6LLU 2100 mAh
5V Mini USB 1A Lithium Battery Charging
DC-DC 3V to 5V 1A Module Boost Converter
Metal brackets (Angled and straight)
GPS breakout v3 Adafruit
OLED display 128x64

Operating system use:

Debian Jessie Lite or Armbian Jessie Server

C++ environment use:

openFrameworks 63bit and ARM
ofxGPIO
dlib

We proceed with the construction of the robot, the first step is to build the skeleton that supports our engines. (Here you can unleash your fantasy)

we continue with the wiring motors and sensor and display OLED for raspberry pi zero, scheme:

(Separate cabling schemes are thus simpler)

then add the camera module for raspberry

Once completed the build your robot, proceed with writing in SD card of OS debian lite for raspberry and installing openframeworks.

Here are the links to complete the installation for raspberry:
install debian on SD card
install openframeworks for ARM6

and here for H3 Allwinner orangepi one:
install debian on SD card
install openframeworks for ARM7

After installing debian and openframeworks we go into ssh on our raspberry and prepare openframeworks to compile the program to control our robot. Download this addons: ofxGPIO is put in: raspberrypi/openframeworks/addons/

Now we download RobotDNN that contains the firmware for the robot running on our raspberry and the control program we run on our PC.

The folder robots that you find in RobotDNN should be placed on our robot in the folder: raspberrypi/openframeworks/apps/myApps/

while the folder: controller and the program to control the remote robot and acquire RTP stream for processing our machine learning algorithms, controller must be placed in the folder of your PC: pc/openframeworks/apps/myApps/

controller has these dependencies:
ofxDelaunay
ofxJoystick
download and put in folder: pc/openframeworks/addons/
while positioning: dlib in: pc/openframeworks/apps/myApps/controller/libs/

Start with the robots program, which is on the raspberry:

raspberrypi$ cd /home/pi/openframeworks/apps/myApps/robots/
raspberrypi$ make
raspberrypi$ ./run 192.168.1.10 5555 11999 320 240 robot001

run is a small script that exec ./bin/robots
./bin/robots wants mandatory parameters: ./run ip-address/host-name gst-port motor-port gst-width gst-height name-robot
./run script also runs rtp server for streaming the camera module that will be acquired by "controller" on our PC, moreover in bin/data/config_pi_motor.txt you can set the PINs to control our engines

laptop$ cd /home/laptop/openframeworks/apps/myApps/controller/
laptop$ make
laptop$ ./run "192.168.1.10,5555,11999,320,240" "robot002.lan,5555,11999,320,240" ...

run is a small script that exec ./bin/controller
./bin/controller accept more parameter block for control multiple robots:
./run "ip-address1/host-name1,gst-port1,motor-port1" "ip-address2/host-name2,gst-port2,motor-port2" etc...

Controller if used with joystick records the position of the robot at each movement, find it in: "bin/data/timestamp_data_log.dat" You can drag and drop this file inside the controller window to tell the robot to repeat the path.

The code is easily extensible, in the next article show the implemmentations of:
autonomous path planning
obstacles recognition

@@@https://github.com/kashimAstro/RobotDNN

Wifi Midi ESP / OpenFrameworks

02/06/2015

▼

Micro Midi Wifi

Here are the steps to build a small wireless midi controller
with ESP8266, in this example we use module version ESP-03.
The first thing to do is programming with a new firmware the for ESP-03

ESP-03 programming mode:

As soon as we are ready, we load our firmware using Arduino IDE
(to configure Arduino IDE with code esp8266, see here in section "configuration IDE Arduino")

This firmware reads INPUT of all the pin provided by ESP-03 and he takes care to convert push button in MIDI notes.

code ESP-03 Arduino IDE:


/*

***************************************

  ESP-03:
        vcc      wifi_ant
        14       ch_pd (vcc)
        12       18
        13       rx
  (gnd) 15       tx
        2        nc
  (gnd) 0        gnd

***************************************

  ESP-07:
        rst      tx
        adc      rx
  (vcc) ch_pd    5
        16       4
        14       0   (gnd)
        12       2
        13       15  (gnd)
        vcc      gnd

***************************************

*/

#include "ESP8266WiFi.h"

const char *ssid     = "********";
const char *password = "********";
const char *host     = "192.168.1.2"; // change this ip with address of your midi servers

String ip            = "";
#define NUM 6

int pin  [NUM] = { 14, 12, 13, 2, 18 };
int note [NUM] = { 65, 66, 67, 68,69 };
int state[NUM] = { 0,  0,  0,  0, 0  };
int tmp  [NUM] = { 1,  1,  1,  1, 1  };

void setup() {
  for (int i = 0; i < NUM; i++)
  {
    pinMode(pin[i], INPUT);
  }
  Serial.begin(115200);
  delay(10);
  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  ip = WiFi.localIP().toString();
}

void loop() {
  for (int i = 0; i < NUM; i++)
  {
    state[i] = digitalRead(pin[i]);
    if (state[i] != tmp[i])
    {

      WiFiClient client;
      const int port = 9055;
      if (!client.connect(host, port)) 
      {
        Serial.println("connection failed");
        return;
      }
      
      Serial.print(ip);
      Serial.print(":");
      Serial.print(note[i]);
      Serial.print(":");
      Serial.println(state[i]?"0":"100");
      
      client.print(ip);
      client.print(":");
      client.print(note[i]);
      client.print(":");
      client.println(state[i]?"0":"100");
      client.stop();
      
    }
    tmp[i] = state[i];
  }
  delay(1);
}

ESP-03 work mode:

ESP-03 work mode with two button:

For testing i connected only two buttons, but you can connect up to five buttons.
Now we prepare the TCP Midi server with openFrameworks ofxMidi and ofxNetwork addons, the server creates a virtual MIDI device, and waits for a connection from an ESP-03 client sending the notes to it when you press and release the button to simulate the ON and OFF of the midi protocol. The server provide more client connection, so you can connect more ESP-03 to midi server

code server TCP to midi:


#include "ofMain.h"
#include "ofxMidi.h"
#include "ofxNetwork.h"
#include "ofAppNoWindow.h"

class ofApp : public ofBaseApp
{
        public:
        ofxTCPServer TCP;
        ofxMidiOut midi;
        int channel, port, mport;

        ofApp(vector conf)
        {
                port    = ofToInt(conf[0]);
                mport   = ofToInt(conf[1]);
                channel = ofToInt(conf[2]);
        }

        void setup()
        {
                TCP.setup(port);
                TCP.setMessageDelimiter("\n");
                midi.listPorts();
                midi.openPort(mport);
        }

        void update()
        {
                for(unsigned int i = 0; i < (unsigned int)TCP.getLastID(); i++)
                {
                        if( !TCP.isClientConnected(i) )
                        {
                                continue;
                        }
                        else
                        {
                                string str = TCP.receive(i);
                                if(str!="")
                                {
                                        vector pars = ofSplitString(str,":");
                                        string ip    = pars[0];
                                        int note     = ofToInt(pars[1]);
                                        int velocity = ofToInt(pars[2]);

                                        if(velocity > 0)
                                        {
                                                midi.sendNoteOn(channel, note, velocity);
                                        }
                                        else
                                        {
                                                midi.sendNoteOff(channel, note, velocity);
                                        }

                                        if(TCP.disconnectClient(i))
                                        {
                                                ofLog()<<"disconnect: "+ip;
                                        }
                                }
                        }
                }
                ofSleepMillis(5);
        }

        void exit()
        {
                TCP.close();
                midi.closePort();
        }
};

int main(int argc, char *argv[])
{
        if(argc > 3)
        {
                ofAppNoWindow w;
                ofSetupOpenGL(&w, 1024,768, OF_WINDOW);
                vector conf;
                conf.push_back(argv[1]);
                conf.push_back(argv[2]);
                conf.push_back(argv[3]);
                ofRunApp( new ofApp(conf));
        }
        else
        {
                cout << "Error: parameter[ tcp-port, midi-port, midi-channel ]\n\t./bin/tcpmidiserver 9055 0 1\n";
        }
}

The server is a batch, which includes as input the TCP port, the port for the MIDI device and channel

Sample server run:

./bin/tcpmidiserver 9055 0 1

@@@https://github.com/kashimAstro/MiniMidi/