Found this beauty second hand (link). The kids and I had a lot of fun building and grabbing things with it. 🙂

Create a bird watching Artificial Intelligence that runs on a raspberry and tweets short videos every time it detects a bird. You can find mine at https://twitter.com/birdfeederAI

1. Get the hardware

To build this bird watcher you will need the following hardware parts. Total cost EUR 255

1. (optional) PoE switch (link) cost EUR 118
2. (optional) PoE hat for Raspberry Pi (link) cost EUR 28.99
3. Raspberry pi 4 8 GB (link) cost EUR 81.66
4. Raspberry pi camera (link) cost EUR 33.90
5. 64GB SD card (link) cost EUR 11.99 (update, since raspberry pi 3B you can boot from USB which is faster and fails less, so better get a 64GB usb 3.1 pendrive (link) 11.63 EUR )

2. Set up base infrastructure

1. Install raspberry pi OS (link)
2. Activate the camera (link)
3. Log into the raspberry pi and create a python virtual environment with base libraries
   1. mkdir -p /home/pi/dev/birdfeederAI
   2. cd /home/pi/dev/birdfeederAI
   3. python3 -m venv ./venv
      1. This creates the python virtual environment.
   4. source ./venv/bin/activate
      1. This activates the python virtual environment. All modules installed by pip will be installed only for this virtual environment
   5. which python
      1. Now that we have created and activated our virtual environment this should return: “/home/pi/dev/birdfeederAI/venv/bin/python”
   6. which pip
      1. Now that we have created and activated our virtual environment this should return: “/home/pi/dev/birdfeederAI/venv/bin/pip”
   7. which pip3
      1. Now that we have created and activated our virtual environment this should return: “/home/pi/dev/birdfeederAI/venv/bin/pip3”
   8. pip list
      1. this will list the python modules we have installed for our virtual environment.
   9. pip install --upgrade pip
      1. This will install the latest version of pip
   10. pip install opencv-python twython
       1. install the opencv python module to process video, twython to send tweets
4. Fix all the missing libs
   1. sudo apt install apt-file
   2. sudo apt-file update
   3. for i in find /home/pi/dev/birdfeederAI/venv/lib |grep so$|xargs ldd|grep "not found"|awk '{print $1;}'; do apt-file search $i|awk 'BEGIN{FS=":"};{print $1;}'; done|sort|uniq|xargs apt install
      1. This will list the libraries which are not installed and install them for you. If something went wrong look here (link)
5. Download the pre-trained model MobileNet-SSD
   1. cd /home/pi/dev/birdfeederAI
   1. git clone https://github.com/chuanqi305/MobileNet-SSD.git
6. Check that the base infrastructure is correctly installed
   1. raspistill -o mypicture.jpg
      1. this should create a picture from the camera and store it as mypicture.jpg
   2. raspivid -t 5000 -o myvideo.h264
      1. this should create a video from the camera and store it as myvideo.h264
   3. copy the below code to pycamtest.py and run it with python pycamtest.py. If everything is correctly set up you should see the output of your camera in a window.

import cv2
cv2.namedWindow("TestCV2")
vc = cv2.VideoCapture(0)
if vc.isOpened():
    rval,frame = vc.read()
else:
    rval = False
while rval:
    frame = cv2.flip(frame,-1)
    cv2.imshow("TestCV2", frame)
    rval, frame = vc.read()
    key = cv2.waitKey(20)
    if key ==27:
        break
vc.release()
cv2.destroyWindow("TestCV2")

3. Set up twitter functionality

1. Set up a twitter account
2. Activate a twitter developer account in https://developer.twitter.com/ and generate your API keys which you will need in the next step
3. Obtain your API keys and copy them to /home/pi/dev/birdfeederAI/auth.py
   1. cat>/home/pi/dev/birdfeederAI/auth.py
   2. consumer_key = 'puthereyourconsumerkey'
   3. consumer_secret = 'puthereyourconsumersecret'
   4. access_token = 'puthereyouraccesstoken'
   5. access_token_secret = 'puthereyouraccesstokensecret'
   6. Ctrl-C

4. Code your birdfeeder

1. Code your program, you can use my code to set up a headless tweeting bird detecting camera 🙂
   1. git clone https://github.com/Rogeman/birdfeederAI.git

import numpy as np
import cv2
import random
import os
import logging
from twython import Twython
from twython import TwythonError

from auth import (
        consumer_key,
        consumer_secret,
        access_token,
        access_token_secret
)
twitter = Twython(
        consumer_key,
        consumer_secret,
        access_token,
        access_token_secret
)


confidence_thr = 0.5
CLASSES = ["background", "aeroplane", "bicycle", "bird", "boat",
    "bottle", "bus", "car", "cat", "chair", "cow", "diningtable",
    "dog", "horse", "motorbike", "person", "pottedplant", "sheep",
    "sofa", "train", "tvmonitor"]
COLORS = np.random.uniform(0, 255, size=(len(CLASSES), 3))
birdfeeder_dir=os.path.dirname(os.path.abspath(__file__))
logging.basicConfig(filename=birdfeeder_dir+'/log/birdfeederAI.log', level=logging.DEBUG, format='%(asctime)s %(message)s')
mobilenet_dir=birdfeeder_dir+'/MobileNet-SSD/'
net = cv2.dnn.readNetFromCaffe(mobilenet_dir+ 'deploy.prototxt' , mobilenet_dir+ 'mobilenet_iter_73000.caffemodel')
blob=None

def applySSD(image):
    global blob
    mybird = bool(False)
    blob = cv2.dnn.blobFromImage(cv2.resize(image, (300, 300)), 0.007843, (300, 300), 127.5)

    # pass the blob through the network and obtain the detections and
    # predictions
    net.setInput(blob)
    detections = net.forward()

    # loop over the detections
    for i in np.arange(0, detections.shape[2]):
        # extract the confidence (i.e., probability) associated with the
        # prediction
        confidence = detections[0, 0, i, 2]

        if confidence > confidence_thr:
            idx = int(detections[0,0,i,1])
            if CLASSES[idx]=="bird":
                mybird=bool(True)
    return mybird

def birdRatio(videoName):
    totalBirdFrames = 1
    totalFrames = 1
    vc2 = cv2.VideoCapture(videoName)
    if vc2.isOpened():
        rval2,frame2 = vc2.read()
    else:
        rval2 = False

    while rval2:
        birdinFrame = applySSD(frame2)
        rval2, frame2 = vc2.read()
        if (birdinFrame):
            totalBirdFrames = totalBirdFrames + 1
        totalFrames = totalFrames + 1

    vc2.release()
    return totalBirdFrames/totalFrames

videoLength=8*60*60*1000
randomsec=random.randint(0,videoLength)


#vc = cv2.VideoCapture(birdfeeder_dir+"/birds_video.mp4")
# If you want to record birds using your camera comment the above line and uncomment the below line. If you want to find birds in a video uncomment the line above and comment the line below 🙂
vc = cv2.VideoCapture(0)
vc.set(cv2.CAP_PROP_POS_MSEC, randomsec)
if vc.isOpened():
    width = vc.get(cv2.CAP_PROP_FRAME_WIDTH)
    height = vc.get(cv2.CAP_PROP_FRAME_HEIGHT)
    fps = vc.get(cv2.CAP_PROP_FPS)
    fcount = vc.get(cv2.CAP_PROP_FRAME_COUNT)
else:
    logging.error('Can\'t open video')

    exit()

recording= False
framerecorded = 0
framecounter = 0
birdinFrame=False
fourcc = cv2.VideoWriter_fourcc(*'h264')
#out = cv2.VideoWriter('output.mp4',fourcc,20.0,(640,480))
out = cv2.VideoWriter(birdfeeder_dir+'/output.mp4',fourcc,fps,(int(width),int(height)))

if vc.isOpened(): # try to get the first frame
    rval, frame = vc.read()
    (h, w) = frame.shape[0] , frame.shape[1]
else:
    rval = False

logging.debug('Started main loop')
while rval:
    #You enter this loop once per frame
    rval, frame = vc.read()
    #uncomment the below line if you need to flip the camera upside down.
    frame = cv2.flip(frame,-1)
    key = cv2.waitKey(20)
    if key == 27: # exit on ESC
        break
    framecounter = framecounter + 1
    if (framecounter > 60):
    # Write frame to disk every 60 frames so we can see what the camera is seeing
        framecounter = 0
        cv2.imwrite(birdfeeder_dir+"/webserver/currentframe.jpg",frame)
    if (birdinFrame==False):
        #Check if this frame has a bird in it
        birdinFrame= applySSD(frame)
    if (birdinFrame== True and recording== False):
        #You have detected the first bird in a frame, start recording
        logging.info('Started recording video')
        recording=True
    if (recording == True):
        #write the frame to file keep track of how many frames you have saved.
        framerecorded = framerecorded + 1
        out.write(frame)
    if (framerecorded > 200):
        #after 200 frames stop recording
        logging.info('Checking recorded video')
        recording = False
        birdinFrame=False
        framerecorded = 0
        out.release()
        filename = birdfeeder_dir+"/output.mp4"
        birdsinvideo= birdRatio(filename)
        logging.debug('percentage of bird in video: '+birdsinvideo)
        if (birdsinvideo> 0.50):
            # if the recorded video has more than 50% of frames with a bird in it then tweet it
            logging.info('Tweeting bird video')
            video = open(filename,'rb')
            try:
                response = twitter.upload_video(media=video, media_type='video/mp4', media_category='tweet_video', check_progress=True)
                twitter.update_status(status='birdfeeder 0.5', media_ids=[response['media_id']])
            except TwythonError as e:
                logging.error('Twitter error:'+str(e))
            birdsinvideo=0
            video.close()
        randomsec=random.randint(0,videoLength)
        vc.set(cv2.CAP_PROP_POS_MSEC, randomsec)
        os.remove(birdfeeder_dir+'/output.mp4')
        out = cv2.VideoWriter(birdfeeder_dir+'/output.mp4',fourcc,fps,(int(width),int(height)))



vc.release()

5. Create an nginx webserver to see what the camera is seeing

1. Install Docker

sudo apt-get update && sudo apt-get upgrade
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker pi
sudo reboot

1. Run nginx docker container serving documents from the birdfeeder webserver folder
   1. docker run -it --rm -d -p 8080:80 --name web -v /home/pi/dev/birdfeederAI/webserver/:/usr/share/nginx/html nginx
2. Create an index.html which refreshes every 2 seconds showing currentframe.jpg

cat > /home/pi/dev/birdfeederAI/webserver/index.html
<html>
    <head>
        <title>Birdfeeder</title>
        <meta http-equiv="refresh" content="2" />
    </head>
    <body>
    <img src=./currentframe.jpg>
    </body>
</html>

Ctrl+C

You can now open a web browser to your raspberry pi’s ip address port 8080 and see what your camera is seeing

6. Add birdfeeder service to systemd

We add birdfeeder to systemd so it starts on boot.

1. Create a bash script that runs birdfeeder in a loop. Run it with nice so it does not consume 100% of cpu (running at 100% for long makes the sd card non-responsive and the system unstable).
   1. vim /home/pi/dev/birdfeederAI/bin/birdfeeder.sh
   2. chmod +x /home/pi/dev/birdfeederAI/bin/birdfeeder.sh

#!/bin/bash
docker run -it --rm -d -p 8080:80 --name web -v /home/pi/dev/birdfeederAI/webserver/:/usr/share/nginx/html nginx
source /home/pi/dev/birdfeederAI/venv/bin/activate
while [ 1 -eq 1 ]
do
nice python /home/pi/dev/birdfeederAI/birdfeeder.py
done

1. Create service file
   1. sudo vim /lib/systemd/system/birdfeeder.service

 [Unit]
 Description=birdfeeder service
 After=multi-user.target

 [Service]
 Type=idle
 ExecStart=/home/pi/dev/birdfeederAI/bin/birdfeeder.sh

 [Install]
 WantedBy=multi-user.target

1. Grant pemissions, add the service and reboot system
   1. sudo chmod 644 /lib/systemd/system/birdfeeder.service
   2. sudo systemctl daemon-reload
   3. sudo systemctl enable birdfeeder.service
   4. sudo reboot

I enjoy tinkering around with robots and electronics. Bridging the invisible world of software with the real world of physical things.

I discovered I could glue a breadboard to the side of the base of this robotic arm, and that I could hold the arduino board to the base with elastic bands. and that the adafruit motor board left 5 analog pins free to use, and that I could put a switch, a potentiometer and an hbridge in the breadboard with these five free pins. And that I could substitute the broken led with a new one with my soldering iron 😀

It is now all ready and working. Every time I hold down the switch button it activates one of the five motors iteratively. With the potentiometer I can have the motor run in one direction or the other. And the LED at the hand of the robot arm shines while the button is pressed.

The only problem pending is I need to change all the worn out gears from the motors as they are eroded from previous experiments (the problem with dc motors as opposed to servo motors is that you can’t know where they are, so I overextended them eroding the gears)

I started self-training on machine learning. I bought a copy of “Hands-On Machine Learning with Scikit-Learn, Keras & Tensorflow” and am enjoying it a lot. I have a small moleskine with which I break down the concepts one day at a time 🙂

So far I have not yet encountered any concept which is not trivial. This means I am progressing. In my experience things are either trivial or impossible. Our job is to break down impossible tasks until they become trivial 🙂

Now that you have your node set up and you have established a few channels you can start routing payments. You can see my node here I have opened three channels for now, and with it I am able to route payments anywhere.

You can set up a tippin.me account to receive tips in satoshis. It creates a custodial lighning wallet where people can send you tips. You can send me some satoshis at https://tippin.me/@Rogeman (as of today 1 satoshi equals 0.00003416 Euros, so don’t be shy, if you go wild and send 1,000 satoshis it will be 3 cents of a euro 😉 )

To send a tip to tippin.me you need to use the payinvoice command, instead of the sendpayment command as you won’t have an invoice and you can send any amount you want.

lncli --lnddir /opt/lnd-data/ --no-macaroons  payinvoice --pay_req lnbc1pw8aa5app5t3zqgnsq2e80s47s33a4rqc4w3spvamwp5tch732zucgglcyefwqdph235hqurfdcs9ymm8v4kkzm3q9p6xjursd9hzumt99y582vfj8ycny2gcqzysxqyz5vqu05rc3mk3m5cteh8gsvh3pf696p4wfdeezq29kkxcdrwjmtj64njk5lj4nv4jul96jcsqvw94a57y4722lrzdygu87jfkw2leu9ceqqpqmmcgl --amt 100

On the testnet a very cool test to do is buy a Blockaccino at starbloqs, check out a video here

You can request others to send you money by sending them an invoice

lncli --lnddir /opt/lnd-data/ --no-macaroons addinvoice 100 --expiry 36000
{
        "r_hash": "f9ba7fd5b0be3d768c29aa951b4a215ea7e37133ebb41fd5a7d424e7981551de",
        "pay_req": "lnbc1u1pw8a7u6pp5lxa8l4dshc7hdrpf4223kj3pt6n7xufnaw6pl4d86sjw0xq4280qdqqcqzysxqypr9qfrglxdwv8jdc9xlsyatugwztdvsn89y4hmlm2mgds5wyl9k3c963uk66zhntp6940yxpfz5fa0au9mcg4c0sfc77eg589fmpnjhcs6gp49ghuh",
        "add_index": 1
}

Once you’ve created the invoice you can send the pay_req to the payer (in the above example lnbc1u1pw8a7u6pp5lxa8l4dshc7hdrpf4223kj3pt6n7xufnaw6pl4d86sjw0xq4280qdqqcqzysxqypr9qfrglxdwv8jdc9xlsyatugwztdvsn89y4hmlm2mgds5wyl9k3c963uk66zhntp6940yxpfz5fa0au9mcg4c0sfc77eg589fmpnjhcs6gp49ghuh)

Others can send you money with the sendpayment command by paying your invoice

lncli --lnddir /opt/lnd-data/ --no-macaroons sendpayment --pay_req=nbc1u1pw8a7u6pp5lxa8l4dshc7hdrpf4223kj3pt6n7xufnaw6pl4d86sjw0xq4280qdqqcqzysxqypr9qfrglxdwv8jdc9xlsyatugwztdvsn89y4hmlm2mgds5wyl9k3c963uk66zhntp6940yxpfz5fa0au9mcg4c0sfc77eg589fmpnjhcs6gp49ghuh

With your lightning network node you can send payments which are:

• Borderless: The internet knows no boundaries, it has no borders, you can as seamlessly send payments within your country, your continent, your planet, or your solar system. In the future we may need to send money to mars!
• Anonymous: Since payments are routed via a mesh of nodes which can’t see the whole end to end payment route, payments are anonymous. This is different to on-chain bitcoin payments, which are only pseudo-anonymous and can be tracked to you.
• Instantaneous: Once you start playing with lnd you will see that it takes less than a second to route a payment. This is mindblowing if you think about it. Internet packets are being communicated throughout the planet, from one internet provider to another, from one router to another, from one lnd node to another, until they reach their destination.
• Low cost: The nodes (yours as well) set the fees they want to charge to route payments. The route chosen is always the cheapest one so this incentivises nodes to reduce the fees. Sending payments over the lightning network costs cents of euro equivalent in satoshis.
• World scale: Lightning is capable of millions to billions of transactions per second across the network. Visa, which is currently the benchmark can transfer less than 2,000 transactions per second.

One problem lightning network has is that you can only send or receive as much money as you have route bandwidth for, this means that if you want to be able to receive payments worth 500 Euro, you need to have channels open worth 500 Euro. This can be pretty expensive for high value payments. Due to this constraint the lightning network can work very well for low value payments, like buying a coffee, and not so well for high value payments like buying a house. In any case high value payments can be done directly using the bitcoin blockchain without using the lightning network layer and are usually not done in high volumes. (Maybe a third layer protocol could be used to spit large payments into groups of smaller ones similarly how tcp over ip is capable of splitting large packets of information into smaller ones and guarantee order and completeness, you could split a payment of 1000 satoshis into two payments of 500)

This is the future of how robots will pay each other.

Lightning network is a Layer 2 protocol that sits on top of the bitcoin/litecoin blockchain. It enables instant, low cost, anonymous payments by routing payments through point to point channels in a similar way to what the correspondent banking system uses today to transfer fiat currency.

You can check out the map of channels of the lightning network at https://graph.lndexplorer.com/ as of today, there are 3488 nodes

In order to set up your lightning network node, first you need to set up your bitcoin node. Here you have simple instructions to set it up by compiling it from source.

First we clone the git repository for the bitcoin core node

sudo apt-get install git
mkdir $HOME/src
cd $HOME/src
git clone https://github.com/bitcoin/bitcoin.git

you will need libzmq3 in order for the lighning node to communicate with your bitcoin node, so we need to install libzmq3-dev

sudo apt install libzmq3-dev

now we configure and install the node

cd bitcoin
./autogen.sh
./configure
make
sudo make install

bitcoind is installed! 🙂 now we need to create a config file and create an rpc user and password so you can communicate with your bicoind node

./share/rpcauth/rpcauth.py roge
#This command will give you the user and password you will need to include in your bitcoin.conf
String to be appended to bitcoin.conf:
rpcauth=roge:793845a197311a324722f93e8360e166$3dfb9011930a6fcb85c99e0e1ad2e0309958b2aa863955faae831e0eeec3894f
Your password:
34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A=

Let’s create a directory where all bitcoin data will be stored. Be warned that this directory will hold a copy of the whole bitcoin blockchain which is as of today 243 GB.

We will copy a bitcoin.conf file to the bitcoin data folder

mkdir /opt/bitcoin-data/
cp $HOME/src/bitcoin/share/examples/bitcoin.conf /opt/bitcoin-data/

now we edit the bitcoin.conf file, the following variables are important

vi /opt/bitcoin-data/bitcoin.conf
#set testnet to 1 if you'd like to run a node for testing purposes not using real bitcoins.
testnet=1 
#set daemon=1 to launch bitcoind as a daemon running in the background
daemon=1
#set rpcauth to the rpcuser you created earlier
rpcauth=roge:793845a197311a324722f93e8360e166$3dfb9011930a6fcb85c99e0e1ad2e0309958b2aa863955faae831e0eeec3894f

We are set. Now we can start our bitcoind node with the following command. It could take one or two days to synchronise with the bitcoin blockchain.

 bitcoind --datadir=/opt/bitcoin-data/ -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A=

you can monitor progress looking at debug.log

tail -333f /opt/bitcoin-data/debug.log

you can connect to your bitcoind node using bitcoin-cli which allows you to manage your node

 watch bitcoin-cli -datadir=/opt/bitcoin-data/ -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= getblockchaininfo

Now encrypt your wallet don’t forget this password. Write it down on a piece of paper and store that paper somewhere safe.

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= -stdin encryptwallet
writehereyoursupersecretpassword
Ctrl-d

now you can get an address where to deposit bitcoin in (you can buy these bitcoins in exchange for fiat currencies such as Euros or Dollars in exchanges like kraken.com or blockchain.org) do not send real bitcoin to a testnet wallet, they will be lost. You can get free testnet bitcoins by googling testnet bitcoin faucet. Here’s one (link)

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= getnewaddress
#This will return the new address where you can send bitcoin to
# the command will return something like 3NUvF2fvitxrU1fY43rCQivx9RtCgvXuEb

Once you have transferred bitcoins to your wallet you can see your balance

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= getbalance
#This command will output your bitcoin balance
0.00001

In order to transfer bitcoins to another address you first need to unlock your wallet for x seconds

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= -stdinn walletpassphrase
enteryoursupersecretpassword
enternumberofsecondsthatthewalletwillremainunlocked
Ctld-d

Now you can transfer bitcoin to another wallet

bitcoin-cli -datadir=/opt/bitcoin-data -rpcuser=roge -rpcpassword=34ZofsEbG95rWITDv8w03crrzIYBioGAKfMqDq1yY1A= sendtoaddress enterthedestinationaddress entertheamounttotransfer

You can find all the bitcoin-cli commands here: https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list

Now that our bitcoin node is ready, let’s set up our lightning network node! 🙂

One of the funnest games I’ve played this year. Awesome Music, graphics, and playability. I’ve enjoyed it a lot! 😀

MongoDB is a nosql database. This means that you don’t use SQL to query the data, instead you use javascript and receive responses in json objects. I enrolled in course M001: Mongodb basics to get a basic understanding of how it works and it seems extremely easy.

The cool part about it being javascript based is that you can become a “full stack” web developer just by learning javascript. The front end with angular, and the backend with node.js and mongodb can be sufficient for you to build web apps. (I still shudder thinking that javascript is being used for anything, yet here we are..)

Mongodb is extremely easy to start playing with, for free you can create a database cluster in the cloud with 512MB by registering in mongodb.com and getting a Mongodb atlas.

MongoDB Compass is a graphical client with which you can connect to your database and analyse your data. It analyses the data and gives you insights on the types of data, you can also filter just by selecting with your mouse the data you want to filter on in an intuitive way.

The CLI is the best way to power-query your data. below my cheat-sheet after completing the basic training course:

MongoDB cheatsheet

1. Show databases
   1. Shows the databases
2. Show collections
   1. Once within a database , shows the collections
3. Insert data
   1. db.moviesScratch.insertMany([],{“ordered”:false)
      1. With ordered:false, it does not stop when a duplicate _id key is found,
4. Query data
   1. db.movies.find().pretty()
5. Comparison operators
   1. Greater than
      1. db.movieDetails.find({runtime:{$gt:90}})
   2. Greater than and Less than
      1. db.movieDetails.find({runtime:{$gt:90, $lt:120}},{_id:0,title:1,runtime:1})
   3. Greater or equal, less or equal
      1. db.movieDetails.find({runtime:{$gte:90, $lte:120}},{_id:0,title:1,runtime:1})
   4. Not equals
      1. db.movieDetails.find({rated: {$ne:”UNRATED”}},{_id:0,title:1,runtime:1})
   5. In
      1. db.movieDetails.find({rated: {$in: [“G”,”PG”]}},{_id:0,title:1,runtime:1,rated:1})
6. Filter for null or non existing fields
   1. db.movies.find({mpaaRating:{$exists:true}}).pretty()
   2. db.data.find({atmosphericPressureChange:{$exists:false}}).pretty().count()
7. Filter fields by type (double, int, string…)
   1. db.movies.find({viewerRating:{$type:”double”}})
8. Filter with multiple “OR”
   1. db.movieDetails.find({$or: [{“tomato.meter”:{$gt:95}},{“metacritic”:{$gt:88}}]},{_id:0,title:1,”tomato.meter”:1,”metacritic”:1})
   2. db.shipwrecks.find({$or:[{watlev:”always dry”},{depth:0}]}).count()
9. Filter with multiple “AND” (this is only needed when we want to apply many filters on the same key
   1. db.movieDetails.find({$and: [{“metacritic”:{$ne:null}},{“metacritic”:{$exists:true}}]},{_id:0,title:1,”tomato.meter”:1,”metacritic”:1})
10. Array field moderators
    1. Get all objects which contain elements in array
       1. db.movieDetails.find({genres:{$all: [“Comedy”,”Crime”,”Drama”]}},{_id:0, title:1, genres:1}).pretty()
    2. Get size of an array
       1. db.movieDetails.find({countries:{$size:1}}).count()
       2.  db.data.find({sections:{$size:2}}).count()
    3. Find a single object with multiple elements which match all the criteria
       1. db.movieDetails.find({boxOffice: {$elemMatch: {“country”: “Germany”, “revenue”: {$gt: 16}}}})
       2.  db.surveys.find({results:{$elemMatch:{“product”:”abc”,”score”:7}}}).count()
11. Regular expressions
    1. db.movieDetails.find({“awards.text”: {$regex: /^Won.* /}}, {_id: 0, title: 1, “awards.text”: 1}).pretty()

Another year, another euskalencounter.org 🙂 This year I’ve binge watched Altered Carbon, and katakoda’s Docker training, below is my cheat sheet for future reference.

1. docker search java
   1. search for a docker image (java in this case)
2. running a docker image
   1. docker run -d openjdk
      1. run detached (-d) the openjdk docker
   2. docker run -d –name redisHostPort -p 6379:6379 redis:latest
      1. –name creates a user friendly name for your running docker
      2. -p <machine host-port:container-port> exposes docker image port in the host port
      3. redis:latest is the docker image:the version
   3. docker run -d –name redisDynamic -p 6379 redis:latest
      1. -p 6379 assigns a random host port
   4. docker port redisDynamic 6379
      1. shows to which host ip:port is docker running image redisDynamic’s port 6379 is assigned
   5. docker run -d –name redisMapped -v /opt/docker/data/redis:/data redis
      1. -v host-path:docker-path maps a docker path to a host path in order to persist data
   6. docker run -it ubuntu bash
      1. -it runs an interactive docker image.
      2. you can run a specific command on the docker image (bash)
   7. docker run -d –name my-production-running-app -e NODE_ENV=production -p 3000:3000 my-nodejs-app
      1. -e VARIABLE=value sets an environment variable in the running container
3. How to run a command inside a running container
   1. docker exec -d ubuntu_bash touch /tmp/execWorks
4. How to create docker data volumes
   1. Create a new container to start it afterwards (instead of running it directly) this is useful mainly to create data storage dockers and use them from other images.
   2. docker create -v /data –name dataContainer ubuntu
   3. docker run –rm –volumes-from dataContainer ubuntu ls -la /data
   4. docker cp config.conf dataContainer:/config/
      1. copies file config.com from the host to the docker container
   5. to move data containers from one machine to another you can export/import them to .tar
      1. docker export dataContainer > dataContainer.tar
      2. docker import dataContainer.tar
      3. docker import http://example.com/exampleimage.tgz
5. How to create a network between containers
   1. docker network create backend-network
      1. creates a network to which we can attach containers. Docker sets a dns server in ip 127.0.0.11 and all connected containers use it by docker adding it to resolv.conf on each container
   2. docker network connect frontend-network redis
      1. connects a container to a network
   3. docker network connect –alias db frontend-network2 redis
      1. –alias <alias_name> creates an alias for the network name
   4. docker network ls
      1. shows all networks
   5. docker network inspect frontend-network
      1. shows all containers connected to a network and their ip addresses
   6. docker network disconnect frontend-network redis
      1. disconnect a container from a network
   7. docker run -d –name=redis –net=backend-network redis
      1. launches a redis container linked to the network backend-network.
6. Building a docker image
   1. docker build -t webserver-image:v1 .
      1. -t <image name:tag> is used to give a name and version to the image.
      2. in the . directory you need to create a file called Dockerfile with yaml notation
   2. .dockerignore
      1. echo passwords.txt >> .dockerignore
         1. all fine names included into file .dockerignore will be ignored when building the image
   3. Dockerfile
      1. FROM nginx:alpine
         1. uses nginx:alpine as base for this docker image
      2. COPY . /usr/share/nginx/html
         1. copies from the host to the image
      3. FROM node:7-alpine
      4. RUN mkdir -p /src/app
         1. RUN’s command inside the docker image
      5. WORKDIR /src/app
         1. set the working directory in the docker image
      6. EXPOSE 3000
         1. EXPOSEs the port from the docker image to a random port in the host system
      7. CMD [ “npm”, “start” ]
         1. runs a command in the docker image you can add parameters as items of the array
      8. ONBUILD COPY . /usr/src/app
         1. onbuild commands won’t be executed until the built image is used as a base image
   4. docker images
      1. returns a list of images installed in the host system
7. removing a docker image
   1. docker container ls -a
      1. lists all containers (running or not)
   2. docker container rm < CONTAINER ID>
      1. removes the container
   3. docker rmi hello-world
      1. removes image hello-world
8. docker ps
   1. show docker processes
9. docker inspect openjdk
   1. show details of running docker
10. docker logs openjdk
    1. shows logs for running docker

This month I returned to moneyconf, a yearly meeting of fintech enthusiasts. Apart from a pair of trendy LHoFT sunglasses I came back with a bunch of insights and inspiration to fuel me until next year.

Insights from moneyconf:

1. 20 million SME in the USA don’t accept electronic payments today @thefriley @Square CFO
2. The finance part of fintech hasn’t been disrupted yet (as opposed to retail) there’s a lot of legacy just signing an nda can take 3months. @thefriley @Square CFO
3. “When we work with the banks it is slow going…” @thefriley @Square CFO
4. The internet deserves a native currency (bitcoin) cashapp allows to seamlessly buy/sell bitcoin. Fiat has a lot of friction, a lot of value lost in sending money overseas. This shouldn’t be the case. I’m all in to disrupt that. @thefriley @Square CFO
5. I believe the way to get great decisions is to have diversity. If everyone agrees, someone is not thinking. Inclusion is needed for diversity to stay. Feminine role models in leadership inspire. Only 5% women in leadership, there is systemic bias @thefriley @Square CFO
6. If you only interview women you will be able to find a woman (surely in the whole world there will be one woman for the job) push your hr team to find them. @thefriley @Square CFO
7. The internet is broken. It has no digital identity construct. With blockchain we can have identity. Social context can attest they are a citizen. Zug in switzerland is an experiment on cryptovoting @ethereumJoseph Consensys Founder
8. Blockchain is a force for universal disintermediation. Intermediaries will be right-sized as to how much value they extract from a transaction. @ethereumJoseph Consensys Founder
9. com disintermediation for music. Civil, fake news control. Data storage, heavy conputation, kwh… all is being disintermediated @ethereumJoseph Consensys Founder
10. Web 3.0 will bring native digital money, greater security, trusted transactions, decentralised storage, bandwidth, heavy compute… @ethereumJoseph Consensys Founder
11. Bullshit sharing economy (uber, lyft, airbnb…) vs. actual sharing economy will happen thanks to ethereum smart contracts with networked business models @ethereumJoseph Consensys Founder
12. Siloed infrastructure is moving towards collaborative, distributed infrastructures. Enterprise ethereum alliance has over 500 members building this shared infrastructure @ethereumJoseph Consensys Founder
13. We can imagine a dystopian world where we have no privacy. We can imagine a world where privacy is everywhere. Regulators have a big role on this. @JoelKatz Chief Cryptographer at Ripple
14. Xrapid is a ripple product which exchanges usd for xrp. Xrp is sent to an exchange that takes mexican pesos. These xrp are then changed for pesos. This is done in under two minutes in a way which is cheaper and faster with current remmitance systems @JoelKatz Chief Cryptographer at Ripple
15. Volatility if xrp is lower in two minutes than the mexican peso over two days. @JoelKatz Chief Cryptographer at Ripple
16. The real utility of blockchain hasn’t happened yet. This is the reason for the big volatility. @JoelKatz Chief Cryptographer at Ripple
17. In 20 years payments will be invisible, inexpensive, and seamlesss. Similar to opening a webpage today @JoelKatz Chief Cryptographer at Ripple
18. Utility tokens are an aberration. If you are raising capital it’s a security. I’m surprised it took so long for the SEC to step in. Patrick Byrne @OverstockCEO
19. ICOs enable financial inclusion. Access to capital for any enterpreneur in a third world country. This is groundbreaking. @jeffpulver
20. It’ll be possible for the underbanked to receive loans peer to peer without a bank Patrick Byrne @OverstockCEO
21. Blockchain will bring more chain than the internet did. It’s a bigger disruption than the internet was Patrick Byrne @OverstockCEO
22. Formation of capital, central banking, cap markets, identity, supply chains,voting will be disrupted by blockchain. We will be able to give a collapsing economy a central bank as a service. Patrick Byrne @OverstockCEO
23. Regulators have good understanding of the blockchain revolution. It’s too late to stop it. In some countries the regulators work for the companies they should regulate. Patrick Byrne @OverstockCEO
24. Bitcoin’s been through 6 booms and busts. If you zoom out it doesn’t look like a bubble. There’s more and more widespread, infrastructure is growing. @DavidFBailey CEO at BTC Media
25. Bitcoin offers decentralised trust. Censorship resistant. There is an intrinsic value in this. @DavidFBailey CEO at BTC Media
26. Bitcoin either is or is not a store of value. If it is the upside is huge. Email is much bigger than the postal+fax industry. If bitcoin is not a store of value the value will be zero. @DavidFBailey CEO at BTC Media
27. The bitcoin blockchain is extremely secure. Hacks have happened in the perifery infrastructure. @DavidFBailey CEO at BTC Media
28. Usdtether has counterparty risk. If the assets backing it are seized the value would drop. @DavidFBailey CEO at BTC Media
29. If the price of bitcoin booms 10x or 100x, more people will spend more money on mining and the energy consumption of the bitcoin blockchain will go from 0.5% of the world consumption to maybe 10%. What then? @DavidFBailey CEO at BTC Media
30. Ethereum helps build better business models. This scares incumbents at first. @ethereumJoseph Consensys Founder
31. R3 is running out of gas. They built a blockchain-like infrastructure. @ethereumJoseph Consensys Founder
32. Master node in dash is a PoW algorithm. We have instant confirmations. Part of the block rewards go to treasury which can be used to pay for development. Stakeholders can vote on how the rewards get used. @fernando Dash CMO
33. There is space for many crypto projects. Dash and ethereum can coexist and interoperate. @ethereumJoseph @fernando
34. Forks are good, it allows for better systems to compete. @ethereumJoseph Consensys Founder
35. Layer two tech (lightning network) has problems of its own. @fernando Dash CMO
36. The spark of crypto is still alive and growing thanks to it becoming a reality @fernando Dash CMO
37. Switzerland is welcoming for ICOs due to their liberal approach to financial innovation @OlgaFeldmeier Smartvalor CEO
38. In the us there is no clarity on ICOs whereas in switzerland the guidelines are clear and they are welcoming. @ShapeShiftCOO
39. Different cryptotokens need to be regulated differently we need to understand what they are. @ShapeShiftCOO
40. Security dealer licenses, banking licenses, money transfer licenses… the regulator needs to simplify @OlgaFeldmeier Smartvalor CEO
41. The business models for crypto are new. The industry needs to self-regulate with oversight from authorities. Regulator needs to empower the industry players. @OlgaFeldmeier Smartvalor CEO
42. Digital pictures of cats are not securities. Regulators are struggling to understand how crypto works. @ShapeShiftCOO
43. Legacy regulation does not catch the bad actors. They are able to evade it anyway. It impedes the good actors and does not allow for fast advance. @ShapeShiftCOO
44. $300bn market cap on ICO. Bitcoin 38%. @edithyeung Partner at 500Startups
45. 10 Things to Look for in a Token Project @edithyeung Partner at 500Startups
46. Avalanche family. New consensus protocol that came out a month ago. Uses montecarlo polling sufficient times for the probability of an incorrect output is insignificant. @el33th4xor Professor at Cornell University
47. Bitcoin solves the problem of making uncensorable inconfiscable money @ToneVays Derivatives Trader.
48. fmwill have it’s own crypto. If you have a question fos someone you can offer a payment in crypto in exchange for a response @askfmio
49. Bitstamp found a regulatory home in luxembourg @nejc_kodric Bitstamp CEO
50. We started targeting banks. But we realised they were moving too slow so we had to change the strategy. @marcusswanepoel Luno CEO
51. The biggest struggle is bridging the legacy banking system with the crypto system. Banks are slow and sometimes hostile. @bitstamp @nejc_kodric Bitstamp CEO
52. In 1980s we went from open outcry markets to electronic markets. Reducing friction and increasing velocity. Next step is the digital age of securities making it 24/7/365 worldwide. Get rid of paper certificates into the blockchain. Tokenize real assets. @APompliano Founder at Morgan Creek Digital Assets
53. In the future we’ll say: “I bought a share of apple”. The blockchain settlement part will be transparent. @APompliano Founder at Morgan Creek Digital Assets
54. Ledger is working on Ledgerboard saas solution for enterprise investors who want to manage cryptocurrencies. @EricLarch Ledger CEO
55. PSD2 is making banks reassess their technical infrastructure. There is an opportunity there. Cabannes @SocieteGenerale Deputy CEO
56. Psd2 gave banks a kick in the butt. Anyone can come along and access your clients data. This gave banks a wake up call and pushed them to colllaborate with the fintech echosystem @RWandhofer Global Head of Regulatory Strategy at Citigroup
57. Psd2 increases the surface open to hacker attack. Cabannes @SocieteGenerale Deputy CEO
58. Psd2 opens the opportunity for collaboration/acquisition of fintechs. This brings innovation to banks. Cabannes @SocieteGenerale Deputy CEO
59. Banks were reticent to psd2. Now it’s a given so it has been accepted. Psd2 will evolve banking into platforms. There will be competition with winners and losers. This will create a new ecosystem. @deBrouwerEBF Chief Policy Officer at European Banking Federation
60. Banks have to move from years long IT projects to months, weeks, or even days. Cabannes @SocieteGenerale Deputy CEO
61. We have to digitize, cleanse, secure, redesign data in big banks @RWandhofer Global Head of Regulatory Strategy at Citigroup
62. Non-cash payments are growing 82% in india in 2018. India has tripled digital payments in the last three years. @ceouaeexchange CEO at UAE exchange
63. Millenials care about the experience. Cashless is more intuitive for them. @ceouaeexchange CEO at UAE exchange
64. In africa most people have cellphones, and few have bank accounts. @ceouaeexchange CEO at UAE exchange
65. Security issues could pose threats towards cashless societies. Nathan Gill @Verifone Head of Solutions
66. The simple things that can scale are the things that will proliferate. Nfc works, qr code works. Retina scanning, vein scanning are sexy, but they are more complex. Nathan Gill @Verifone Head of Solutions
67. Bitcoin fulfills aristotle’s 4 conditions for sound money @maxkeiser CIO Heisenberg Capital
68. The banking system as we know it will not exist in the next 10-50 years. @maxkeiser CIO Heisenberg Capital
69. Ireland aspires to be the main tech hub for europe. @campaignforleo Prime minister of Ireland
70. This year, the Irish central bank will start a fintech innovation hub. @campaignforleo Prime minister of Ireland
71. Credit karma has 80 million users in usa. Helping millenials find financial instruments @kennethlin Founder of Credit Karma
72. Through autonomous finance we could give a way for financially illiterate people to make better financial decisions. Similar to the gps helping us reach our destination. @kennethlin Founder of Credit Karma
73. Worldremit helps the unbanked send and receive money via mobile phone @Ismail_WR Worldremit CEO
74. 3bn (half of the world) people are invisible to the global financial system. Poor farmers get paid once a year with the harvest and don’t have a way to safely store their money. They need to walk for miles to pay an utility bill to keep the lights on @MichaelSchlein Accion CEO
75. We use financial tools every day. The underbanked live in a cash world and don’t have access to these tools. @MichaelSchlein Accion CEO
76. World bank’s report on financial inclusion focuses in bank accounts. There are many underbanked people which created an account and use it less than once a year (they received a government grant and cashed it out, they have an account but don’t use it) @MichaelSchlein Accion CEO
77. Blockchain can make a difference for cheap remittances without going through the slow and expensive bank based system @Ismail_WR Worldremit CEO
78. Blockchain is a fundamental new layer to the internet. It enables value exchange, governance and trust. Every fiduciary and record keeping industry will be affected by it. @jerallaire Circle CEO
79. Accounting. Insurance. Corporate and commercial law. Voting and governance. Record keeping. These will be transformed by the blockchain. @jerallaire Circle CEO
80. Crypto tokens can be currencies, commodities, and securities. @jerallaire Circle CEO
81. Stablecoins tether, usd coin, maker, basis can enable settlement without volatility risk @jerallaire Circle CEO
82. Crypto securities represent some rules based financial contract. @jerallaire Circle CEO
83. We are at the start of a “tokenisation of everything”. Bonds, fiat currency, insurance contracts, property tokens (land, houses, cars), decision tokens (votes, governance) @jerallaire Circle CEO
84. To realise the tokenisation of everything we need: 1) mature and scalable public blockchains. 2) fiat backed stablecoins. 3) mature marketplaces 4) regulatory clarity @jerallaire Circle CEO
85. Fiat stablecoins are mandatory for this to work. We need to control the volatility. @jerallaire Circle CEO
86. Sharing value globally in the same way we can share information. Cheap. Fast. Easy. Fundamental economic integration. @jerallaire Circle CEO
87. In China a lot of merchants don’t accept cash any more. Visa, wechat, alipay, but no cash. @AlainFalys Chairman Yoyo Wallet
88. I just came back from china. I was at a restaurant and some people just stood up and left, some walked to the cashier, some needed the waiter to charge. Everyone paying with the mobile phone. @AlainFalys Chairman Yoyo Wallet
89. We believe voice is the next frontier. Today 50bn/month searches are done by voice. Voice enables a natural experience. Giulio Montemagno. Director Europe @amazonpay
90. Mobile topping can be done worldwide in an unregulated way. I can send mobile balance to any phone in the world. This is a workaround for remittances. @MRding Executive Chairman at Ding
91. Remittances are unfairly expensive. @cwinesLondon Founder at WorldRemit
92. Gdpr and blockchain don’t work together at all. Blockchain is immutable while gdpr requires user data to allow to be deleted. @DariaRippingale CEO at billpro
93. Customer demand for cryprocurrency trading has massively exploded. @yoniassia CEO at eToro
94. Traditional markets are too regulated with too many entry barriers. Crypto markets are the wild west, trial and error. @yoniassia CEO at eToro
95. You can’t expect to make 40x on all of your investments, right? But you are 27 so you do @yoniassia CEO at eToro
96. I’m optimistic. In 20 years any financial asset and central bank money will be tokenised in a blockchain. @yoniassia CEO at eToro
97. Bitcoin has the brand awareness of cocacola. @yoniassia CEO at eToro
98. Compliance protects the business to avoid risk. Learn to love the audits. Create a team that focuses on responding to audit and loves it. Jacqueline Molnar, Chief Compliance Officer at @WesternUnion
99. Fintech in europe is very exciting. In europe there is a lot of interest to increase the 5% of online payments. @chughesjohnson COO at Stripe
100. Governments want to enable startups because they will enable job creation and GDP growth. @chughesjohnson COO at Stripe
101. Stripe wants to level the playing field allowing any startup to send and receive cross border payments. @chughesjohnson COO at Stripe
102. Availability of developer resources is according to nielsen the biggest constraint for companies. @chughesjohnson COO at Stripe
103. Payments is extremely local. Regulation, banking, tradition … Stripe is trying to make it uniform globally for startups to sell worldwide seamlessly @chughesjohnson COO at Stripe
104. 1) find your product and focus on it full speed 2) If you don’t hire the right talent your company will not grow. 3) tempo. Keep the speed as you grow. @chughesjohnson COO at Stripe
105. The developer is increasingly the decision maker. The technical decisions you take are integral to your business strategy. Developers should be at the decision making table. If you treat them as a service they will leave. @chughesjohnson COO at Stripe
106. Before wechatpay and alipay people in china has no credit card. These services leapfrogged and today it’s difficult to be able to pay with cash in china. @edithyeung Partner at 500Startups
107. Chinese people leave china for hongkong, malta, japan, san francisco… to avoid chinese regulatory blocking and continue their blockchain projects. @edithyeung Partner at 500Startups
108. We have a process we follow whenever we open up in a new country. If the process does not work we tweak it. We are hungry. @NStoronsky CEO at Revolut.