I got a Huion tablet for Santa Claus. I’m having a lot of fun animating the characters of a comic I used to draw back at university: cat art>/dev/null
I feel this could be the piece I was missing to keep crawling forward on my years-long intention of creating my own videogame. A graphic adventure based on the games I loved when I was a kid: Monkey Island, Indiana Jones, Kings Quest.. π
My wife’s PiboyDMG stopped working and wouldn’t boot. After some investigation it was due to the SD card being fried (no surprise there, SD cards are horribly unreliable) so I tried setting it up to boot from a 128GB mini usb drive. I followed the same steps I would have with an sd drive:
spacers to stack the raspberry pis (link) cost EUR 9.99
Total cost EUR 630
2. Install base infrastructure
Set up the PoE Hats and stack the raspberry pis using the spacers.
Install raspberry pi OS and base configuration (link)
Since Raspberry pi 3B you can boot directly from USB. No more io errors from unreliable SD cards π
To perform a headless install create a file called ssh in /boot/ folder, this will enable ssh so you can access your pis remotely without need for a monitor (link)
Install tmux and get this script (link) to simultaneously modify all 4 raspis
3. Initialize Kubernetes Control Plane on the master node
Choose one raspberry pi to be your master node from which you will control the cluster. This is called the Kubernetes Control Plane. Run the below commands on the master node.
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
rm -rf .kube/
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
vi $HOME/.bashrc
# Add the below line to the end of .bashrc
export KUBECONFIG=$HOME/.kube/config
2. Set up the kubernetes network (I am using flannel) on the master node.
3. If all went well you should see all your nodes ready with command kubectl get nodes. You are now ready to create deployments, pods and services.
5. Destroy your kubernetes infrastructure
After you are done playing, or in case things stop working and you want to start from scratch you can use the below instructions to destroy your kubernetes infrastructure.
Remove nodes by running the below in your control plane instead of raspiclustern you can use the hostnames you have set up for your machines.
Today I βdid a boxβ for the birdfeederAI and set it up to work in the garden. Within a couple hours it had detected and tweeted itβs first real life bird video! π
Create a bird watching Artificial Intelligence that runs on a raspberry and tweets short videos every time it detects a bird. You can find mine at https://twitter.com/birdfeederAI
1. Get the hardware
To build this bird watcher you will need the following hardware parts. Total cost EUR 255
64GB SD card (link) cost EUR 11.99 (update, since raspberry pi 3B you can boot from USB which is faster and fails less, so better get a 64GB usb 3.1 pendrive (link) 11.63 EUR )
Check that the base infrastructure is correctly installed
raspistill -o mypicture.jpg
this should create a picture from the camera and store it as mypicture.jpg
raspivid -t 5000 -o myvideo.h264
this should create a video from the camera and store it as myvideo.h264
copy the below code to pycamtest.py and run it with python pycamtest.py. If everything is correctly set up you should see the output of your camera in a window.
import numpy as np
import cv2
import random
import os
import logging
from twython import Twython
from twython import TwythonError
from auth import (
consumer_key,
consumer_secret,
access_token,
access_token_secret
)
twitter = Twython(
consumer_key,
consumer_secret,
access_token,
access_token_secret
)
confidence_thr = 0.5
CLASSES = ["background", "aeroplane", "bicycle", "bird", "boat",
"bottle", "bus", "car", "cat", "chair", "cow", "diningtable",
"dog", "horse", "motorbike", "person", "pottedplant", "sheep",
"sofa", "train", "tvmonitor"]
COLORS = np.random.uniform(0, 255, size=(len(CLASSES), 3))
birdfeeder_dir=os.path.dirname(os.path.abspath(__file__))
logging.basicConfig(filename=birdfeeder_dir+'/log/birdfeederAI.log', level=logging.DEBUG, format='%(asctime)s %(message)s')
mobilenet_dir=birdfeeder_dir+'/MobileNet-SSD/'
net = cv2.dnn.readNetFromCaffe(mobilenet_dir+ 'deploy.prototxt' , mobilenet_dir+ 'mobilenet_iter_73000.caffemodel')
blob=None
def applySSD(image):
global blob
mybird = bool(False)
blob = cv2.dnn.blobFromImage(cv2.resize(image, (300, 300)), 0.007843, (300, 300), 127.5)
# pass the blob through the network and obtain the detections and
# predictions
net.setInput(blob)
detections = net.forward()
# loop over the detections
for i in np.arange(0, detections.shape[2]):
# extract the confidence (i.e., probability) associated with the
# prediction
confidence = detections[0, 0, i, 2]
if confidence > confidence_thr:
idx = int(detections[0,0,i,1])
if CLASSES[idx]=="bird":
mybird=bool(True)
return mybird
def birdRatio(videoName):
totalBirdFrames = 1
totalFrames = 1
vc2 = cv2.VideoCapture(videoName)
if vc2.isOpened():
rval2,frame2 = vc2.read()
else:
rval2 = False
while rval2:
birdinFrame = applySSD(frame2)
rval2, frame2 = vc2.read()
if (birdinFrame):
totalBirdFrames = totalBirdFrames + 1
totalFrames = totalFrames + 1
vc2.release()
return totalBirdFrames/totalFrames
videoLength=8*60*60*1000
randomsec=random.randint(0,videoLength)
#vc = cv2.VideoCapture(birdfeeder_dir+"/birds_video.mp4")
# If you want to record birds using your camera comment the above line and uncomment the below line. If you want to find birds in a video uncomment the line above and comment the line below π
vc = cv2.VideoCapture(0)
vc.set(cv2.CAP_PROP_POS_MSEC, randomsec)
if vc.isOpened():
width = vc.get(cv2.CAP_PROP_FRAME_WIDTH)
height = vc.get(cv2.CAP_PROP_FRAME_HEIGHT)
fps = vc.get(cv2.CAP_PROP_FPS)
fcount = vc.get(cv2.CAP_PROP_FRAME_COUNT)
else:
logging.error('Can\'t open video')
exit()
recording= False
framerecorded = 0
framecounter = 0
birdinFrame=False
fourcc = cv2.VideoWriter_fourcc(*'h264')
#out = cv2.VideoWriter('output.mp4',fourcc,20.0,(640,480))
out = cv2.VideoWriter(birdfeeder_dir+'/output.mp4',fourcc,fps,(int(width),int(height)))
if vc.isOpened(): # try to get the first frame
rval, frame = vc.read()
(h, w) = frame.shape[0] , frame.shape[1]
else:
rval = False
logging.debug('Started main loop')
while rval:
#You enter this loop once per frame
rval, frame = vc.read()
#uncomment the below line if you need to flip the camera upside down.
frame = cv2.flip(frame,-1)
key = cv2.waitKey(20)
if key == 27: # exit on ESC
break
framecounter = framecounter + 1
if (framecounter > 60):
# Write frame to disk every 60 frames so we can see what the camera is seeing
framecounter = 0
cv2.imwrite(birdfeeder_dir+"/webserver/currentframe.jpg",frame)
if (birdinFrame==False):
#Check if this frame has a bird in it
birdinFrame= applySSD(frame)
if (birdinFrame== True and recording== False):
#You have detected the first bird in a frame, start recording
logging.info('Started recording video')
recording=True
if (recording == True):
#write the frame to file keep track of how many frames you have saved.
framerecorded = framerecorded + 1
out.write(frame)
if (framerecorded > 200):
#after 200 frames stop recording
logging.info('Checking recorded video')
recording = False
birdinFrame=False
framerecorded = 0
out.release()
filename = birdfeeder_dir+"/output.mp4"
birdsinvideo= birdRatio(filename)
logging.debug('percentage of bird in video: '+birdsinvideo)
if (birdsinvideo> 0.50):
# if the recorded video has more than 50% of frames with a bird in it then tweet it
logging.info('Tweeting bird video')
video = open(filename,'rb')
try:
response = twitter.upload_video(media=video, media_type='video/mp4', media_category='tweet_video', check_progress=True)
twitter.update_status(status='birdfeeder 0.5', media_ids=[response['media_id']])
except TwythonError as e:
logging.error('Twitter error:'+str(e))
birdsinvideo=0
video.close()
randomsec=random.randint(0,videoLength)
vc.set(cv2.CAP_PROP_POS_MSEC, randomsec)
os.remove(birdfeeder_dir+'/output.mp4')
out = cv2.VideoWriter(birdfeeder_dir+'/output.mp4',fourcc,fps,(int(width),int(height)))
vc.release()
5. Create an nginx webserver to see what the camera is seeing
You can now open a web browser to your raspberry pi’s ip address port 8080 and see what your camera is seeing
6. Add birdfeeder service to systemd
We add birdfeeder to systemd so it starts on boot.
Create a bash script that runs birdfeeder in a loop. Run it with nice so it does not consume 100% of cpu (running at 100% for long makes the sd card non-responsive and the system unstable).
I enjoy tinkering around with robots and electronics. Bridging the invisible world of software with the real world of physical things.
I discovered I could glue a breadboard to the side of the base of this robotic arm, and that I could hold the arduino board to the base with elastic bands. and that the adafruit motor board left 5 analog pins free to use, and that I could put a switch, a potentiometer and an hbridge in the breadboard with these five free pins. And that I could substitute the broken led with a new one with my soldering iron π
It is now all ready and working. Every time I hold down the switch button it activates one of the five motors iteratively. With the potentiometer I can have the motor run in one direction or the other. And the LED at the hand of the robot arm shines while the button is pressed.
The only problem pending is I need to change all the worn out gears from the motors as they are eroded from previous experiments (the problem with dc motors as opposed to servo motors is that you canβt know where they are, so I overextended them eroding the gears)