Word Clouds to Gauge Political Sentiment

My R skills have been steadily improving,  and now I do try my hand at some eccentric projects as well. Of course, the more I use R, the more I feel inadequate given the range of possibilities for its use. It’s a steep learning curve, and some off beat projects keep me interested in scaling it.

One such use is mining social media sites for trends – Financial, Political, Branding Related etc. I am still in the process of refining my algorithms and parameters to feel confident about relying on them, but at the moment, I think I do get a fair insight into things such as Brand Perception or Political trends.

As a test, I looked to see what the sentiment for Flipkart was on Twitter. This histogram below is the result of my analysis:

I was a little surprised, since being a long time Flipkart fan, I have always been very pleased with their service and I was not expecting to see a plot skewed to the left .

This may not necessarily mean that Flipkart is horrible – it could just be that more people complain about what they perceive to be bad service than wax eloquent about a positive experience. A cross section of my data does confirm this – people are complaining more often than speaking about positive experiences. Also, I have not scored non-English tweets, and they have a score of 0.

  • Case of #CHEATING and #Fraud against #Flipkart.com. Order ID: OD30728091408 [Incident: 130909-011676]. Details submitted to #ConsumerCourt
  • @Flipkart please tell me when I’ll get my #GTAV delivered to me. I’ve been waiting a long time for it. #flipkart
  • #Nokia Lumia 920 Now Available For Rs. 23999 From #Flipkart #India http://t.co/bsjauEunkY #wp8 #windowsphone
  • Arghhhh … Kaam toh dhanng ka nai hota…. Baatein badi badi … Advertisements me…. #Flipkart
  • I started hating #Flipkart Delivery Staff…
  • Ab enough… I started hating #Flipkart .. Sooo much….
  • Sigh. As usual. Flipkart shipping has no respect for computer products. Damaged in transit. Big impact on the side of the box. #flipkart
  • @Flipkart still waiting for my #GTAV copy to arrive. Don’t think it’ll come to me today. #gutted #flipkart
  • The book to grab and read! ‘The King’s Harvest’ by Chetan Raj Shrestha! Available on #flipkart http://t.co/k8WB5xug07
  • OUT FOR DELIVERY SINCE FOREVER! Fuck you #Flipkart I still don’t have my #GTA5 #GTAV

I am of course, more interested in the financial use of such social data, perhaps as an indicator for Socio-Economic Stability, changing economic/company trends and the like. And political trends too while I am at it.

I did create a Narendra Modi vs Rahul Gandhi Sentiment Analysis, but I’ll leave that for another day. For now, you can make your own inferences from these two word clouds. A word of caution though – the Rahul Gandhi word cloud was made from 350 tweets less than the Narendra Modi cloud, since Twitter’s cap on my hourly API access had kicked in.

Share this:

Chaos is Money

Most emerging market currencies have been slaughtered over the past 3 months. Much has been written about why this has happened. And does this sound familiar, almost no one saw the emerging market debt bubble coming?

The patterns have been similar – a looming balance of payments crisis, currency getting devalued, stock markets in a near meltdown state. The situation isn’t pretty, but if you are a speculator, you can stand to make some handsome gains.

I have had some fun with the Turkish Lira over the past few months – check the image below and you will know why.





A crisis in Syria, exiting foreign investors, soaring yield on the 10-year bond, an unstable government having to deal with a lot of internal protests are all elements of this story to which end up in just one direction – currency devaluation. Sure, the head of the Central Bank, Erdem Basci has stated that they would do anything it takes to support the Lira, with a reported USD 40 Billion war-chest, but as history and George Soros have proven, there’s only so much Central Banks can do to prevent currency devaluation if the fundamentals of an economy are not sound.

The same goes for many economies across the globe – India, Indonesia, Venezuela, Thailand amongst others.

Share this:

Cyprus: One Down, A Few More to Go

The bank run in Cyprus, is another one of the ‘once in a lifetime’ events we are witnessing for the second or third time in our lives.

The last such event I distinctly remember is the bank run on Northern Rock in 2007. Atleast that time, the Bank of England stepped in announced that they would guarantee all deposits in the bank, a marked deviation from existing guarantee limits as documented by MSE.

The amount has been changed three times. First on the back of Northern Rock, then Bradford & Bingley, then to bring the UK in line with the rest of Europe.

If you’re one of the very few who still have outstanding issues, the amount you get will depend on the time of the ‘compensation trigger’.

Defaults between 7 Oct 2008 and 30 Dec 2010: If your bank went bust between these dates, you’d get back the first £50,000 per person, per institution.

Defaults between 1 Oct 2007 and 6 Oct 2008: You’d get 100% of the first £35,000 back.

Defaults before 1 Oct 2007: You’d get 100% of the first £2,000 of your cash back and 90% of the next £33,000 on top; so you’d get £31,700 of the first £35,000 back.

Admittedly, the conditions were rather different, and the BOE and Alistair Darling were first and foremost trying to calm down fears of small depositors. The Cyprus situation seems to be one where depositors, even small depositors are being given no protection whatsoever. By imposing a 6.75% tax on those with savings below €100,000 and a 9.9% tax on those with deposits over  €100,000, this is effectively an overnight haircut. Deposit insurance programs do involve some degree of a haircuts, but it is surprising that the Cypriot officials have imposed one on small depositors too. Not that they might have had a choice. As per the BBC,

The Germans, however, were not prepared to support a larger bailout. They suspected that half of the deposits in the island’s banks were held by Russians with much of the money being laundered. Rescuing high-rolling Russians could not be sold to German taxpayers.

I the coming 6-9 months, I expect similar conditions to take place in atleast one of Spain, Portugal, Italy and Greece. Perhaps in even two of these countries. All of them face political uncertainty in addition to fiscal uncertainty, and that is never a good combination even at the best of times.


Share this:

Predicting Randomness

We are surrounded by predictive models. Every industry uses it, from the most basic of demand forecasting algorithms built into an ERP system (SAP, Oracle) to the complex algorithms used in the Hedge Fund/HFT industry. While there is no accurate way to predict a future data point no matter how good your algorithm, most users of such systems hope to be within a certain error limit, the proverbial ball park figure.

How can we be reasonably certain of our research and tests based on historical data though? There are quite a few challenges involved.

1. How much data is enough data?

Is backtesting on 3 or 5 years of tick data sufficient for you to have to confidence to put in your money or your client’s money on the system? It depends. If your time frame isn’t long enough, your model may not be able to capture different scenarios in the market.
Is having 20 years of data enough? Again, it depends. Data too old may not be relevant to you at all since markets change over time. And what type of data would be appropriate? Tick data, or EOD data?

As a general rule I have followed, systems based on volatility indicators like Donchian Channels or Bollinger Bands seem appropriate for smaller timeframes such as 1-minute or 5-minute data.

Trend following systems on the other hand, require larger time frames, such as EOD data to cut out the noise levels. An MACD based system would cause you to go mad in the scenario such as the one below I faced in 2008. Trading the Euro Bund futures, something triggered a violent drop. The price fell nearly 170 odd points in a matter of a few minutes. No one had been assassinated, no nuclear war had broken out. Strangely, it was not even a fat finger trade. Someone actually meant it. While we were waiting for an announcement from Eurex for a reversal, none came through.

In other words, it is not the quantum of data that is important, but the nature of data. Do you have enough to test your system in a variety of scenarios? Bull markets have a strange way to make people believe they are great traders, the actual test is when a bear run occurs.

2. Are you overfitting?

We have more data than we have ever had in history, yet, are we necessarily doing things better with it?  Sometimes too many data parameters render testing results useless for the real world. This is not to say that optimising your system is bad or undesirable.

Our predictive model is only as good as the predictions it makes, and not just good looking equity curves for historical data. And this is where optimisation plays a part. As with the previous point, it is vital to ensure that your parameters reflect real world scenarios, and that you test them across a variety of scenarios from the past. Going overboard with optimisation results in a scenario where your historical testing results are the “Best Case Scenario” under a given set of conditions. One way to check for this, is to see the magnitude of change occurring (positive or negative) for a very small change in a given parameter. If the test result is significantly different due to a minor change introduced, that is a good sign that you might have over optimised. The more parameters you add to this soup, the greater the complexity, the greater the divergence from a real world trading scenario.

So to have real predictive power, it is vital to not have too many parameters, and also that the parameters introduced come into play frequently.

Share this:

MATLAB Experiments

If you are interested in any level of computational finance, you will invariably have come across MATLAB. It’s a wonderful tool, and I have been spending significant amounts of time trying to get good at some MATLAB coding. Also helps that there are a lot of good resources on the internet to help you get started.

Some of the examples below are just things you could try, to get familiar with the MATLAB interface, environment and syntax. Of course, like with any language, the more time you spend on it, the higher your level of comfort with it.

1. Using MATLAB as a Calculator:

Admittedly, there are far easier to use (and less expensive) calculators available, however, this is not a bad way to get acquainted with MATLAB. When you open MATLAB, you will come across a screen that looks somewhat like the one shown below.

MATLAB - Blank Workspace





Start typing random expressions to see what you get. Goes without saying, 100*10-100 will equal 900 whether on your MATLAB screen or on your HP12C.

Or maybe use variables and assign arbitrary values such as “a=4*3/32+19-63″

MATLAB - Workspace





The output is calculated and displayed in the workspace instantly when you press “Enter”.

2. Matrices and Magic Squares:

There’s a whole lot you can do with Matrices (also known as Vectors). Start out with defining a row matrix.

[1 2 3 4]

Or assign you could assign a matrix to a variable.

a = [1 2 3 4]

MATLAB - Matrices Basic






Magic Squares are another type of arrays that MATLAB can process. Initiate a magic square by assigning the variable a = magic(3) . This results in a 3X3 matrix, as shown below.

Magic Squares - 3x3 4x4






You can read up more about Magic Squares here.

I am sure it will take me a while, but it is going to be fun by the time I can start Stochastic simulations on MATLAB. I will post a few updates in the meantime.

Share this:

Why Are Petrol Prices in India So High?

Well, this is very self-explanatory. Given that India procures most of its oil from OPEC, and that the OPEC Basket Price has somewhat been reducing in the recent past, the price of Petrol in India is rather baffling.


I have assumed some figures such as Capital Costs, Refining Costs, Transportation etc, and I might be very close to the actual figure. If you wish to play around with the numbers, you can download the Excel workbook here.


In other words:



Share this:

Putting Taleb Into Perspective

I was a trader in 2008, and some of the lessons I learnt and events I witnessed will stay with me throughout my life.

And it was precisely then that Nassim Nicholas Taleb captured public imagination, with his two books – “The Black Swan” and “Fooled By Randomness” which suddenly brought finance terminology into the mainstream. It was everywhere,  with newspapers – including tabloids, online columns, and not to forget advertisements in London Tube Stations all exalting these books and Taleb as the only sane voice amongst a group of people constantly trying to play down an impending crisis. Of course, Ben Bernanke’s irrational exuberance with comments such as the one below didn’t help.

“The risk that the economy has entered a substantial downturn appears to have diminished over the past month or so”

Ben Bernanke, June 10, 2008

I bought those books too, to understand what Taleb had to say and why he had suddenly captured popular imagination. His books were an interesting read, and rather gripping too, even though I felt he was relying too much on anecdotes and analogies.
I am of the opinion that Taleb’s assertion’s were perhaps intentionally sensational, since he had to sell books.

Let’s view some of his analogies and assertions. 
Perhaps his most famous analogy of all is the Turkey analogy -  a turkey is fed for 999 days leading up to Christmas and slaughtered on the 1000th day. We are like the turkey, with statisticians leading us to believe that each day the turkey lives, it’s well on its way to a long life. And each day a hedge fund lives, it’s closer to a blowup. While I agree that statistics can be manipulated by unscruplous people to make ’projections’, Taleb’s blanket assertions are equally bad.  


Most blowups in the recent past such as Nick Leeson/Barings Bank and Jérôme Kerviel/Société Générale, have been the result of illegal trading. So it’s not so much a turkey, as it is the 999 day old family dog who got run over by a drunk driver on its 1000th day. The end result is violent death, the reasons however are vastly different.
If anything, such instances call for tighter business controls in a trading environment rather than paranoia about swans and geese.

Secondly, Taleb seems to be a huge proponent of buying out-of-the-money options, but sadly, both his trading record as well as other academic research does not indicate that this is a very sound strategy. Oleg Bondarenko in his paper “Why are Put Options So Expensive?” states:

Simple trading strategies that involve selling at-the-money and out-of-the-money puts would have earned extraordinary profits.

 Taleb is by all indications a wealthy man, but unlike Warren Buffett or Seth Klarman or Benjamin Graham, his wealth is highly unlikely to have been earned from investing. I would take the opinions of such a person on investing with a generous pinch of salt. In that respect, he is closer to a Darren Winters than a Benjamin Graham – one who makes money from speaking about investing rather than by investing.

Share this:

Snake Oil Salesmen

If you are in the snake oil business, it’s always boom time.

Consider the Snake Oil salesmen at NiftyDirect.com, a service puportedly selling ‘tips’ for stock and options trading on the Nifty Index. They advertise often on Indian stock market related websites such as Moneycontrol.com and claim outrageous results such as those in the screenshot below.

So what would your results be, if you invested Rs. 100 with them? A millionaire in 12 months at best, and 24 months at worst!

Share this:

Lost and Found

Some of the countries which lost and then regained a AAA rating on their sovereign debt.

Share this:

Risk Free Rates and AAA Ratings

Over the past couple of weeks, I have been working on a paper estimating Risk Free Rates depending on the currency, based on similar work by Aswath Damodaran. Also part of my reading list was “Into the abyss: What if nothing is risk free?” by the same author and it almost reads like a prophecy now, given the downgrade of debt issued by the US Treasury by S&P from AAA to AA+.

In real terms though, the five-year probability of a default for AA+ rated soverign debt is the same as that of a AAA rated instrument – 0.00. The cost of borrowing isn’t likely to sky rocket either, if the experience of Japan is taken into account, especially given that two of the three major ratings agencies – Moodies and Fitch – still haven’t mentioned anything about a ratings downgrade.

However, this does add another one of those ‘once in a lifetime’ events that I have seen in my short career thusfar – The fall of Bear Stearns, Lehman going bankrupt, the Madoff scandal and the loss of the AAA status.   


Share this: