Emergent behaviors and IT Projects

Did you know that the largest known ant colony goes from northern Italy to the Spanish Atlantic coast? Yet ants don’t have an architect overseeing their construction, or a centralized coordination group which tells the rest of the ants what they have to do.

Ants are simple individuals with very limited communication skills, yet they are capable of forming minimum paths from the colony to the food and form efficient ant chains moving the food to the colony.

Grasscutter ants harvest tons of grass every year which they can’t digest, bring it to their colony, cut it to the right size and place it into a garden of fungus, which eats the grass and the ants eat the fungus. It’s not one or two intelligent outliers that do this. It’s the colony that does this to feed it’s 5.000 individuals.

Termites create 6 meter high “cathedrals”, with no central coordination, just simple rules that are wired into each individual’s DNA guiding them to release certain chemicals in response to stimuli such as proximity to other chemicals.

Birds fly in organized flocks without having radio communication to coordinate their positions and speeds, following 3 simple rules: 1) avoid getting too close to neighbor, 2) steer towards average heading of neighbors 3) steer towards average position of neighbors.

Wolves hunt in packs circling the prey with no communication needed, just following two rules: 1) get close to the prey 2) when close enough to the prey, move away from other wolves.

Fish swim in groups, homogeneous groups are called schools and heterogeneous are called schoals. This reduces their chance of being eaten by a predator. This chance is further reduced by joining a schoal of bigger fish, which will be preferred by predators. One puzzling aspect of schoal selection is how a fish can choose to join a schoal of fish similar to themselves, given that they cannot know their own appeareance.

The set of rules that the individuals follow is easy to understand, but the emerging behavior of the group is in no transparent way derived from this set of rules.

Would there be a set of rules that would enable us to work efficiently in projects without the need for central coordination?

ants

Ease with uncertainty using the cosmic scale

“The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.”

The Call of Cthulhu – H.P. Lovecraft

As you read this, sitting on your chair, sipping on your coffee, you are hurling through a dark, cold and incomprehensibly vast space on a gigantic ball of dirt which is rotating around our sun at 30 km/second. Us and our neighboring galaxies are rushing at 1,000km/second towards a structure we call the great attractor, with a mass of 100 quadrillion times that of our Sun and 150 million light-years away.

You are not an individual. The visible universe is composed of 1080 atoms. You are composed of 1027 of these same atoms, 99% of which form hydrogen, oxygen and carbon. These atoms that compose you, group in molecules, which group in cells, which create you. These cells are living and performing tasks like digesting your food, transporting endorphins to make you happy, or killing other beings which are inside your body so you don’t get sick.

Check out below, a kinesin protein which is walking around inside one of your cells right now, transporting some vesicles to keep the cell working.

kinesin

The universe has existed for over 14 billion years, and will exist for many billion years to come. If we compare the age of the universe to one calendar year, on the first of January we would have had the big bang, around September the solar system would have formed. Dinosaurs would have become extinct on 29th of December, and Jesus Christ would have been born on the 31st of December 5 seconds before midnight. Columbus would have reached the Americas 1.2 seconds before midnight. Your whole life, with all your trials and tribulations will have gone by in less than 0.23 cosmic seconds.

Think about this when you are worried about your performance review or your project getting delayed.

Why do we get angry?

“Anybody can become angry – that is easy, but to be angry with the right person and to the right degree and at the right time and for the right purpose, and in the right way – that is not within everybody’s power and is not easy. “
Aristotle

Wouldn’t it be great if we could control our temper at all times? If emails or conversations never resulted in our anger emotion being triggered and us feeding this feeling by responding with an aggressive language or a snappy, self-serving response?

Humans have a common set of primary emotions that become triggered due to external circumstances:

  1. Happiness: Happiness is triggered by things we like. Too little happiness is bad, causes depression and lack of will to live. Too much Happiness can be bad, it can make us take unreasonable risks and pay the consequences.
  2. Sadness: Sadness is triggered by a loss we don’t accept. Too little sadness is bad, sadness helps cope and empathise with others. Too much sadness causes depression.
  3. Fear: Fear is triggered by a threat. Being unable to feel fear will lead to foolishness and taking unreasonable risks and pay the consequences. Being too fearsome will incapacitate you from living a normal life.
  4. Disgust: Disgust helps us avoid certain parts of something. For example I may not like the noise someone makes while chewing food, but I can separate that from the whole person, I don’t reject the whole person, just a small part.
  5. Anger: Anger is triggered by disrespect of boundaries, either someone does not respect a limit you impose on them or someone is imposing a limit on you that you don’t accept.

Us humans cannot control feeling the emotion, it being triggered, what we can work on is what we do with it after it has been triggered.

  1. When happiness is triggered we can work on taking care of how we express it to avoid making others feel sad or angry. For example it’s important to be a good winner and not rub it in your opponent’s face when you win.
  2. When sadness is triggered we can work on understanding what loss we are not accepting, and finding ways to cope.
  3. When fear is triggered we can work on measuring the size of the threat. More often than not the scenarios we play in our minds are worse than the real outcome.
  4. When disgust is triggered we can work on understanding which part of the whole we don’t like and make a differentiation. Accept the person even if they wear a watch of your rival football team.
  5. When anger is triggered we can analyse which boundary the other person is breaching or imposing on you. Figure out why and what stories are you telling yourself: Is the other person a villain? (Is the other evil or stupid?) Are you a victim? (I am good, this is happening to me) Are you helpless? (Are all courses of action pointless?) Is the boundary being pressed immovable? Should you not accommodate?

In particular, email is a wonderful tool to ignite the flames of angry discussion, since email is always interpreted (what did he mean by “fine”, does he think I’m stupid? I’m sure he thinks I’m stupid! Arggg, I’ll respond copying his boss!). It would be great if we were all more conscious of our emotional intelligence, and used it to remove the obstacles limiting us from perfect collaboration.

anger

Bitcoin Algorithmic trading

I’ve been playing for the last three weeks with a fun project: Bitcoin algorithmic trading.

Kraken.com has a super easy restful api which allows to place orders with just a few lines of code. Using the api is free and you only need to pay 0.26% per trade.

For example, I can query my balance with three lines of code

<?php require_once ‘header.php’;
$res = $kraken->QueryPrivate(‘Balance’);
print_r($res);
?>

I can place an order to buy or sell bitcoin with nine lines

<?php require_once ‘header.php’;
$res = $kraken->QueryPrivate(‘AddOrder’, array(
‘pair’ => ‘XXBTZEUR’,
‘type’ => ‘sell’,
‘ordertype’ => ‘limit’,
‘price’ => ‘1200’,
‘volume’ => ‘0.0001’
));
print_r($res);
?>

I can query the last trades executed in the market in three lines

<?php require_once ‘header.php’; $res = $kraken->QueryPublic(‘Trades’, array(‘pair’ => ‘XXBTZEUR’,’since’=>’1486857244103748648′));
print_r($res);
?>

These three functions are all is needed to be able to build a program that looks at the direction the trades are going, check my balance and place an order.

I had a stroke of insight, an algorithm which could not fail: “If the last three trades are going up, buy it all. If the last three trades are going down, sell it all, else wait” I thought to myself my first lamborghini would be orange while I got to work building the program.

Instead of letting it loose with my money I thought it would be a good idea to backtest it first, to simulate how my algorithm would fare.

You can download the whole history of bitcoin trades for each market here. The following script scheduled with cron to run every morning at 5:00 am downloads the whole history of prices, loads it into a mysql table, runs my algorithm and outputs a file with each decision taken.

#!/bin/bash
cd /home/roge/programando/kraken/histdata
wget http://api.bitcoincharts.com/v1/csv/krakenEUR.csv.gz
gunzip -f krakenEUR.csv.gz
cd ..
mysql kraken <truncate_krakenEUR.sql
mysqlimport kraken –local histdata/krakenEUR.csv –columns timestamp,price,volume –fields-terminated-by=’,’
mysql kraken <create_ruleofthree.sql
php backtest_ruleofthree.php >backtest_ruleofthree.output

the output is 4.8 million lines long so I thought I’d use big data analysis tools such as jupyter/pandas/matplotlib to see what my algorithm had done in a nice chart. Below you can see how my algorithm is super efficient at giving all my money to kraken in fees. In less than 30.000 ticks I would have blown through all my savings. Back to the drawing board 🙂

bitcoinalgotrading

Leadership and CPR

To learn certain lessons a price must be paid. Eggs may have to be broken. 

Ten years ago, I started my MBA in Instituto de Empresa, a Spanish business school. The first weekend we had a kind of offsite, to get to know each other and start working together in a fun way.

In one of the exercises, each team was given one raw egg, 10 straws, 1 meter of duct tape and scissors. We had one hour to invent a way for the egg to be dropped from two meters and not break.

Being an MBA student I was obsessed with the idea of leadership, I took charge and convinced my team to choose my idea over the others: we would build three layers of straw protection around the egg, the air inside the straws would surely protect the egg from breaking when hitting the ground.

We worked for one hour cutting the straws, putting them together with duct tape, and carefully ensuring the egg was fully covered with this air cushion. All under my upbeat leadership, guidance and weeding out of the other, lesser ideas.

Time was up, it was go time, I stood in front of the whole class, with my team backing me and let the egg fly, sure that we would triumph where others had fallen.

Splat. The egg broke making our structure look like something an intellectually challenged three year old would have thought of, totally ignoring my leadership. I had imposed my leadership on the team, imposed my idea on the team, and led them to jump off the cliff. All with a smile on my face, nice manners, and convincing words.

I was hit with a realisation I have still not forgotten:

  1. Leadership is about convincing everyone to do what you think is best. You may be totally wrong about what is best and you will only realise this once it’s too late.
  2. Let the experts lead. Listen to the experts, they will probably have better ideas than you do.
  3. The universe will act regardless of what you think it will do.
  4. Your actions and decisions have very real consequences. 

I have the utmost respect for knowledge, I believe knowledge should be in the lead. When I hire members of my team, expert knowledge is the key driver. 

If you see a man having a heart attack on the street, you may have the best leadership attitude and jump in trying to save him, but if you don’t know CPR the man will die. A bad leader will pretend to know CPR until the man is dead. A good leader will call someone who knows CPR. An amazing leader, a Leonidas, an Achilles, a John Mclane, an Optimus Prime, the stuff legends are made of, will know CPR.