Arduino controlled robot arm

I enjoy tinkering around with robots and electronics. Bridging the invisible world of software with the real world of physical things.

I discovered I could glue a breadboard to the side of the base of this robotic arm, and that I could hold the arduino board to the base with elastic bands. and that the adafruit motor board left 5 analog pins free to use, and that I could put a switch, a potentiometer and an hbridge in the breadboard with these five free pins. And that I could substitute the broken led with a new one with my soldering iron 😀

It is now all ready and working. Every time I hold down the switch button it activates one of the five motors iteratively. With the potentiometer I can have the motor run in one direction or the other. And the LED at the hand of the robot arm shines while the button is pressed.

The only problem pending is I need to change all the worn out gears from the motors as they are eroded from previous experiments (the problem with dc motors as opposed to servo motors is that you can’t know where they are, so I overextended them eroding the gears)

Machine Learning 101

I started self-training on machine learning. I bought a copy of “Hands-On Machine Learning with Scikit-Learn, Keras & Tensorflow” and am enjoying it a lot. I have a small moleskine with which I break down the concepts one day at a time 🙂

So far I have not yet encountered any concept which is not trivial. This means I am progressing. In my experience things are either trivial or impossible. Our job is to break down impossible tasks until they become trivial 🙂

Setting up Lightning Network [3/3] – what to do with your lnd node?

Now that you have your node set up and you have established a few channels you can start routing payments. You can see my node here I have opened three channels for now, and with it I am able to route payments anywhere.

You can set up a tippin.me account to receive tips in satoshis. It creates a custodial lighning wallet where people can send you tips. You can send me some satoshis at https://tippin.me/@Rogeman (as of today 1 satoshi equals 0.00003416 Euros, so don’t be shy, if you go wild and send 1,000 satoshis it will be 3 cents of a euro 😉 )

To send a tip to tippin.me you need to use the payinvoice command, instead of the sendpayment command as you won’t have an invoice and you can send any amount you want.

lncli --lnddir /opt/lnd-data/ --no-macaroons  payinvoice --pay_req lnbc1pw8aa5app5t3zqgnsq2e80s47s33a4rqc4w3spvamwp5tch732zucgglcyefwqdph235hqurfdcs9ymm8v4kkzm3q9p6xjursd9hzumt99y582vfj8ycny2gcqzysxqyz5vqu05rc3mk3m5cteh8gsvh3pf696p4wfdeezq29kkxcdrwjmtj64njk5lj4nv4jul96jcsqvw94a57y4722lrzdygu87jfkw2leu9ceqqpqmmcgl --amt 100

On the testnet a very cool test to do is buy a Blockaccino at starbloqs, check out a video here

You can request others to send you money by sending them an invoice

lncli --lnddir /opt/lnd-data/ --no-macaroons addinvoice 100 --expiry 36000
{
        "r_hash": "f9ba7fd5b0be3d768c29aa951b4a215ea7e37133ebb41fd5a7d424e7981551de",
        "pay_req": "lnbc1u1pw8a7u6pp5lxa8l4dshc7hdrpf4223kj3pt6n7xufnaw6pl4d86sjw0xq4280qdqqcqzysxqypr9qfrglxdwv8jdc9xlsyatugwztdvsn89y4hmlm2mgds5wyl9k3c963uk66zhntp6940yxpfz5fa0au9mcg4c0sfc77eg589fmpnjhcs6gp49ghuh",
        "add_index": 1
}

Once you’ve created the invoice you can send the pay_req to the payer (in the above example lnbc1u1pw8a7u6pp5lxa8l4dshc7hdrpf4223kj3pt6n7xufnaw6pl4d86sjw0xq4280qdqqcqzysxqypr9qfrglxdwv8jdc9xlsyatugwztdvsn89y4hmlm2mgds5wyl9k3c963uk66zhntp6940yxpfz5fa0au9mcg4c0sfc77eg589fmpnjhcs6gp49ghuh)

Others can send you money with the sendpayment command by paying your invoice

lncli --lnddir /opt/lnd-data/ --no-macaroons sendpayment --pay_req=nbc1u1pw8a7u6pp5lxa8l4dshc7hdrpf4223kj3pt6n7xufnaw6pl4d86sjw0xq4280qdqqcqzysxqypr9qfrglxdwv8jdc9xlsyatugwztdvsn89y4hmlm2mgds5wyl9k3c963uk66zhntp6940yxpfz5fa0au9mcg4c0sfc77eg589fmpnjhcs6gp49ghuh

With your lightning network node you can send payments which are:

  • Borderless: The internet knows no boundaries, it has no borders, you can as seamlessly send payments within your country, your continent, your planet, or your solar system. In the future we may need to send money to mars!
  • Anonymous: Since payments are routed via a mesh of nodes which can’t see the whole end to end payment route, payments are anonymous. This is different to on-chain bitcoin payments, which are only pseudo-anonymous and can be tracked to you.
  • Instantaneous: Once you start playing with lnd you will see that it takes less than a second to route a payment. This is mindblowing if you think about it. Internet packets are being communicated throughout the planet, from one internet provider to another, from one router to another, from one lnd node to another, until they reach their destination.
  • Low cost: The nodes (yours as well) set the fees they want to charge to route payments. The route chosen is always the cheapest one so this incentivises nodes to reduce the fees. Sending payments over the lightning network costs cents of euro equivalent in satoshis.
  • World scale: Lightning is capable of millions to billions of transactions per second across the network. Visa, which is currently the benchmark can transfer less than 2,000 transactions per second.

One problem lightning network has is that you can only send or receive as much money as you have route bandwidth for, this means that if you want to be able to receive payments worth 500 Euro, you need to have channels open worth 500 Euro. This can be pretty expensive for high value payments. Due to this constraint the lightning network can work very well for low value payments, like buying a coffee, and not so well for high value payments like buying a house. In any case high value payments can be done directly using the bitcoin blockchain without using the lightning network layer and are usually not done in high volumes. (Maybe a third layer protocol could be used to spit large payments into groups of smaller ones similarly how tcp over ip is capable of splitting large packets of information into smaller ones and guarantee order and completeness, you could split a payment of 1000 satoshis into two payments of 500)

This is the future of how robots will pay each other.

Setting up lightning network [1/3] – Set up your bitcoind node

Lightning network is a Layer 2 protocol that sits on top of the bitcoin/litecoin blockchain. It enables instant, low cost, anonymous payments by routing payments through point to point channels in a similar way to what the correspondent banking system uses today to transfer fiat currency.

You can check out the map of channels of the lightning network at https://graph.lndexplorer.com/ as of today, there are 3488 nodes

In order to set up your lightning network node, first you need to set up your bitcoin node. Here you have simple instructions to set it up by compiling it from source.

First we clone the git repository for the bitcoin core node

sudo apt-get install git
mkdir $HOME/src
cd $HOME/src
git clone https://github.com/bitcoin/bitcoin.git

you will need libzmq3 in order for the lighning node to communicate with your bitcoin node, so we need to install libzmq3-dev

sudo apt install libzmq3-dev

now we configure and install the node

cd bitcoin
./autogen.sh
./configure
make
sudo make install

bitcoind is installed! 🙂 now we need to create a config file and create an rpc user and password so you can communicate with your bicoind node

./share/rpcauth/rpcauth.py roge
#This command will give you the user and password you will need to include in your bitcoin.conf
String to be appended to bitcoin.conf:
rpcauth=roge:793845a197311a324722f93e8360e166$3dfb9011930a6fcb85c99e0e1ad2e0309958b2aa863955faae831e0eeec3894f
Your password:
34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A=

Let’s create a directory where all bitcoin data will be stored. Be warned that this directory will hold a copy of the whole bitcoin blockchain which is as of today 243 GB.

We will copy a bitcoin.conf file to the bitcoin data folder

mkdir /opt/bitcoin-data/
cp $HOME/src/bitcoin/share/examples/bitcoin.conf /opt/bitcoin-data/

now we edit the bitcoin.conf file, the following variables are important

vi /opt/bitcoin-data/bitcoin.conf
#set testnet to 1 if you'd like to run a node for testing purposes not using real bitcoins.
testnet=1 
#set daemon=1 to launch bitcoind as a daemon running in the background
daemon=1
#set rpcauth to the rpcuser you created earlier
rpcauth=roge:793845a197311a324722f93e8360e166$3dfb9011930a6fcb85c99e0e1ad2e0309958b2aa863955faae831e0eeec3894f

We are set. Now we can start our bitcoind node with the following command. It could take one or two days to synchronise with the bitcoin blockchain.

 bitcoind --datadir=/opt/bitcoin-data/ -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A=

you can monitor progress looking at debug.log

tail -333f /opt/bitcoin-data/debug.log

you can connect to your bitcoind node using bitcoin-cli which allows you to manage your node

 watch bitcoin-cli -datadir=/opt/bitcoin-data/ -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= getblockchaininfo

Now encrypt your wallet don’t forget this password. Write it down on a piece of paper and store that paper somewhere safe.

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= -stdin encryptwallet
writehereyoursupersecretpassword
Ctrl-d

now you can get an address where to deposit bitcoin in (you can buy these bitcoins in exchange for fiat currencies such as Euros or Dollars in exchanges like kraken.com or blockchain.org) do not send real bitcoin to a testnet wallet, they will be lost. You can get free testnet bitcoins by googling testnet bitcoin faucet. Here’s one (link)

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= getnewaddress
#This will return the new address where you can send bitcoin to
# the command will return something like 3NUvF2fvitxrU1fY43rCQivx9RtCgvXuEb

Once you have transferred bitcoins to your wallet you can see your balance

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= getbalance
#This command will output your bitcoin balance
0.00001

In order to transfer bitcoins to another address you first need to unlock your wallet for x seconds

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= -stdinn walletpassphrase
enteryoursupersecretpassword
enternumberofsecondsthatthewalletwillremainunlocked
Ctld-d

Now you can transfer bitcoin to another wallet

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= sendtoaddress enterthedestinationaddress entertheamounttotransfer

You can find all the bitcoin-cli commands here: https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list

Now that our bitcoin node is ready, let’s set up our lightning network node! 🙂

Playing with MongoDB

MongoDB is a nosql database. This means that you don’t use SQL to query the data, instead you use javascript and receive responses in json objects. I enrolled in course M001: Mongodb basics to get a basic understanding of how it works and it seems extremely easy.

mongodb-for-giant-ideas-bbab5c3cf8

The cool part about it being javascript based is that you can become a “full stack” web developer just by learning javascript. The front end with angular, and the backend with node.js and mongodb can be sufficient for you to build web apps. (I still shudder thinking that javascript is being used for anything, yet here we are..)

Mongodb is extremely easy to start playing with, for free you can create a database cluster in the cloud with 512MB by registering in mongodb.com and getting a Mongodb atlas.

MongoDB Compass is a graphical client with which you can connect to your database and analyse your data. It analyses the data and gives you insights on the types of data, you can also filter just by selecting with your mouse the data you want to filter on in an intuitive way.

The CLI is the best way to power-query your data. below my cheat-sheet after completing the basic training course:

MongoDB cheatsheet

  1. Show databases
    1. Shows the databases
  2. Show collections
    1. Once within a database , shows the collections
  3. Insert data
    1. db.moviesScratch.insertMany([],{“ordered”:false)
      1. With ordered:false, it does not stop when a duplicate _id key is found,
  4. Query data
    1. db.movies.find().pretty()
  5. Comparison operators
    1. Greater than
      1. db.movieDetails.find({runtime:{$gt:90}})
    2. Greater than and Less than
      1. db.movieDetails.find({runtime:{$gt:90, $lt:120}},{_id:0,title:1,runtime:1})
    3. Greater or equal, less or equal
      1. db.movieDetails.find({runtime:{$gte:90, $lte:120}},{_id:0,title:1,runtime:1})
    4. Not equals
      1. db.movieDetails.find({rated: {$ne:”UNRATED”}},{_id:0,title:1,runtime:1})
    5. In
      1. db.movieDetails.find({rated: {$in: [“G”,”PG”]}},{_id:0,title:1,runtime:1,rated:1})
  6. Filter for null or non existing fields
    1. db.movies.find({mpaaRating:{$exists:true}}).pretty()
    2. db.data.find({atmosphericPressureChange:{$exists:false}}).pretty().count()
  7. Filter fields by type (double, int, string…)
    1. db.movies.find({viewerRating:{$type:”double”}})
  8. Filter with multiple “OR”
    1. db.movieDetails.find({$or: [{“tomato.meter”:{$gt:95}},{“metacritic”:{$gt:88}}]},{_id:0,title:1,”tomato.meter”:1,”metacritic”:1})
    2. db.shipwrecks.find({$or:[{watlev:”always dry”},{depth:0}]}).count()
  9. Filter with multiple “AND” (this is only needed when we want to apply many filters on the same key
    1. db.movieDetails.find({$and: [{“metacritic”:{$ne:null}},{“metacritic”:{$exists:true}}]},{_id:0,title:1,”tomato.meter”:1,”metacritic”:1})
  10. Array field moderators
    1. Get all objects which contain elements in array
      1. db.movieDetails.find({genres:{$all: [“Comedy”,”Crime”,”Drama”]}},{_id:0, title:1, genres:1}).pretty()
    2. Get size of an array
      1. db.movieDetails.find({countries:{$size:1}}).count()
      2.  db.data.find({sections:{$size:2}}).count()
    3. Find a single object with multiple elements which match all the criteria
      1. db.movieDetails.find({boxOffice: {$elemMatch: {“country”: “Germany”, “revenue”: {$gt: 16}}}})
      2.  db.surveys.find({results:{$elemMatch:{“product”:”abc”,”score”:7}}}).count()
  11. Regular expressions
    1. db.movieDetails.find({“awards.text”: {$regex: /^Won.* /}}, {_id: 0, title: 1, “awards.text”: 1}).pretty()

Docker cheat sheet

Another year, another euskalencounter.org 🙂 This year I’ve binge watched Altered Carbon, and katakoda’s Docker training, below is my cheat sheet for future reference.

  1. docker search java
    1. search for a docker image (java in this case)
  2. running a docker image
    1. docker run -d openjdk
      1. run detached (-d) the openjdk docker
    2. docker run -d –name redisHostPort -p 6379:6379 redis:latest
      1. –name creates a user friendly name for your running docker
      2. -p <machine host-port:container-port> exposes docker image port in the host port
      3. redis:latest is the docker image:the version
    3. docker run -d –name redisDynamic -p 6379 redis:latest
      1. -p 6379 assigns a random host port
    4. docker port redisDynamic 6379
      1. shows to which host ip:port is docker running image redisDynamic’s port 6379 is assigned
    5. docker run -d –name redisMapped -v /opt/docker/data/redis:/data redis
      1. -v host-path:docker-path maps a docker path to a host path in order to persist data
    6. docker run -it ubuntu bash
      1. -it runs an interactive docker image.
      2. you can run a specific command on the docker image (bash)
    7. docker run -d –name my-production-running-app -e NODE_ENV=production -p 3000:3000 my-nodejs-app
      1. -e VARIABLE=value sets an environment variable in the running container
  3. How to run a command inside a running container
    1. docker exec -d ubuntu_bash touch /tmp/execWorks
  4. How to create docker data volumes
    1. Create a new container to start it afterwards (instead of running it directly) this is useful mainly to create data storage dockers and use them from other images.
    2. docker create -v /data –name dataContainer ubuntu
    3. docker run –rm –volumes-from dataContainer ubuntu ls -la /data
    4. docker cp config.conf dataContainer:/config/
      1. copies file config.com from the host to the docker container
    5. to move data containers from one machine to another you can export/import them to .tar
      1. docker export dataContainer > dataContainer.tar
      2. docker import dataContainer.tar
      3. docker import http://example.com/exampleimage.tgz
  5. How to create a network between containers
    1. docker network create backend-network
      1. creates a network to which we can attach containers. Docker sets a dns server in ip 127.0.0.11 and all connected containers use it by docker adding it to resolv.conf on each container
    2. docker network connect frontend-network redis
      1. connects a container to a network
    3. docker network connect –alias db frontend-network2 redis
      1. –alias <alias_name> creates an alias for the network name
    4. docker network ls
      1. shows all networks
    5. docker network inspect frontend-network
      1. shows all containers connected to a network and their ip addresses
    6. docker network disconnect frontend-network redis
      1. disconnect a container from a network
    7. docker run -d –name=redis –net=backend-network redis
      1. launches a redis container linked to the network backend-network.
  6. Building a docker image
    1. docker build -t webserver-image:v1 .
      1. -t <image name:tag> is used to give a name and version to the image.
      2. in the . directory you need to create a file called Dockerfile with yaml notation
    2. .dockerignore
      1. echo passwords.txt >> .dockerignore
        1. all fine names included into file .dockerignore will be ignored when building the image
    3. Dockerfile
      1. FROM nginx:alpine
        1. uses nginx:alpine as base for this docker image
      2. COPY . /usr/share/nginx/html
        1. copies from the host to the image
      3. FROM node:7-alpine
      4. RUN mkdir -p /src/app
        1. RUN’s command inside the docker image
      5. WORKDIR /src/app
        1. set the working directory in the docker image
      6. EXPOSE 3000
        1. EXPOSEs the port from the docker image to a random port in the host system
      7. CMD [ “npm”, “start” ]
        1. runs a command in the docker image you can add parameters as items of the array
      8. ONBUILD COPY . /usr/src/app
        1. onbuild commands won’t be executed until the built image is used as a base image
    4. docker images
      1. returns a list of images installed in the host system
  7. removing a docker image
    1. docker container ls -a
      1. lists all containers (running or not)
    2. docker container rm < CONTAINER ID>
      1. removes the container
    3. docker rmi hello-world
      1. removes image hello-world
  8. docker ps
    1. show docker processes
  9. docker inspect openjdk
    1. show details of running docker
  10. docker logs openjdk
    1. shows logs for running docker