Friday, September 27, 2013
Wednesday, September 25, 2013
How do you explain Machine Learning and Data Mining to non Computer Science people?
( By Pararth Shah on quora.com)
Mango Shopping
Suppose you go shopping for mangoes one day. The vendor has laid out a cart full of mangoes. You can handpick the mangoes, the vendor will weigh them, and you pay according to a fixed Rs per Kg rate (typical story in India).
Obviously, you want to pick the sweetest, most ripe mangoes for yourself (since you are paying by weight and not by quality). How do you choose the mangoes?
You remember your grandmother saying that bright yellow mangoes are sweeter than pale yellow ones. So you make a simple rule: pick only from the bright yellow mangoes. You check the color of the mangoes, pick the bright yellow ones, pay up, and return home. Happy ending?
Not quite.
Life is complicated
Suppose you go home and taste the mangoes. Some of them are not sweet as you'd like. You are worried. Apparently, your grandmother's wisdom is insufficient. There is more to mangoes than just color.
After a lot of pondering (and tasting different types of mangoes), you conclude that the bigger, bright yellow mangoes are guaranteed to be sweet, while the smaller, bright yellow mangoes are sweet only half the time (i.e. if you buy 100 bright yellow mangoes, out of which 50 are big in size and 50 are small, then the 50 big mangoes will all be sweet, while out of the 50 small ones, on average only 25 mangoes will turn out to be sweet).
You are happy with your findings, and you keep them in mind the next time you go mango shopping. But next time at the market, you see that your favorite vendor has gone out of town. You decide to buy from a different vendor, who supplies mangoes grown from a different part of the country. Now, you realize that the rule which you had learnt (that big, bright yellow mangoes are the sweetest) is no longer applicable. You have to learn from scratch. You taste a mango of each kind from this vendor, and realize that the small, pale yellow ones are in fact the sweetest of all.
Now, a distant cousin visits you from another city. You decide to treat her with mangoes. But she mentions that she doesn't care about the sweetness of a mango, she only wants the most juicy ones. Once again, you run your experiments, tasting all kinds of mangoes, and realizing that the softer ones are more juicy.
Now, you move to a different part of the world. Here, mangoes taste surprisingly different from your home country. You realize that the green mangoes are in fact tastier than the yellow ones.
You marry someone who hates mangoes. She loves apples instead. You go apple shopping. Now, all your accumulated knowledge about mangoes is worthless. You have to learn everything about the correlation between the physical characteristics and the taste of apples, by the same method of experimentation. You do it, because you love her.
Enter computer programs
Now, imagine that all this while, you were writing a computer program to help you choose your mangoes (or apples). You would write rules of the following kind:
if (color is bright yellow and size is big and sold by favorite vendor): mango is sweet.
if (soft): mango is juicy.
etc.
You would use these rules to choose the mangoes. You could even send your younger brother with this list of rules to buy the mangoes, and you would be assured that he will pick only the mangoes of your choice.
But every time you make a new observation from your experiments, you have to manually modify the list of rules. You have to understand the intricate details of all the factors affecting the quality of mangoes. If the problem gets complicated enough, it can get really difficult to make accurate rules by hand that cover all possible types of mangoes. Your research could earn you a PhD in Mango Science (if there is one).
But not everyone has that kind of time.
Enter Machine Learning algorithms
ML algorithms are an evolution over normal algorithms. They make your programs "smarter", by allowing them to automatically learn from the data you provide.
You take a randomly selected specimen of mangoes from the market (training data), make a table of all the physical characteristics of each mango, like color, size, shape, grown in which part of the country, sold by which vendor, etc (features), along with the sweetness, juicyness, ripeness of that mango (output variables). You feed this data to the machine learning algorithm (classification/regressio
Next time you go to the market, you measure the characteristics of the mangoes on sale (test data), and feed it to the ML algorithm. It will use the model computed earlier to predict which mangoes are sweet, ripe and/or juicy. The algorithm may internally use rules similar to the rules you manually wrote earlier (for eg, adecision tree), or it may use something more involved, but you don't need to worry about that, to a large extent.
Voila, you can now shop for mangoes with great confidence, without worrying about the details of how to choose the best mangoes. And what's more, you can make your algorithm improve over time (reinforcement learning), so that it will improve its accuracy as it reads more training data, and modifies itself when it makes a wrong prediction. But the best part is, you can use the same algorithm to train different models, one each for predicting the quality of apples, oranges, bananas, grapes, cherries and watermelons, and keep all your loved ones happy :)
And that, is Machine Learning for you. Tell me if it isn't cool.
Mango Shopping
Suppose you go shopping for mangoes one day. The vendor has laid out a cart full of mangoes. You can handpick the mangoes, the vendor will weigh them, and you pay according to a fixed Rs per Kg rate (typical story in India).
Obviously, you want to pick the sweetest, most ripe mangoes for yourself (since you are paying by weight and not by quality). How do you choose the mangoes?
You remember your grandmother saying that bright yellow mangoes are sweeter than pale yellow ones. So you make a simple rule: pick only from the bright yellow mangoes. You check the color of the mangoes, pick the bright yellow ones, pay up, and return home. Happy ending?
Not quite.
Life is complicated
Suppose you go home and taste the mangoes. Some of them are not sweet as you'd like. You are worried. Apparently, your grandmother's wisdom is insufficient. There is more to mangoes than just color.
After a lot of pondering (and tasting different types of mangoes), you conclude that the bigger, bright yellow mangoes are guaranteed to be sweet, while the smaller, bright yellow mangoes are sweet only half the time (i.e. if you buy 100 bright yellow mangoes, out of which 50 are big in size and 50 are small, then the 50 big mangoes will all be sweet, while out of the 50 small ones, on average only 25 mangoes will turn out to be sweet).
You are happy with your findings, and you keep them in mind the next time you go mango shopping. But next time at the market, you see that your favorite vendor has gone out of town. You decide to buy from a different vendor, who supplies mangoes grown from a different part of the country. Now, you realize that the rule which you had learnt (that big, bright yellow mangoes are the sweetest) is no longer applicable. You have to learn from scratch. You taste a mango of each kind from this vendor, and realize that the small, pale yellow ones are in fact the sweetest of all.
Now, a distant cousin visits you from another city. You decide to treat her with mangoes. But she mentions that she doesn't care about the sweetness of a mango, she only wants the most juicy ones. Once again, you run your experiments, tasting all kinds of mangoes, and realizing that the softer ones are more juicy.
Now, you move to a different part of the world. Here, mangoes taste surprisingly different from your home country. You realize that the green mangoes are in fact tastier than the yellow ones.
You marry someone who hates mangoes. She loves apples instead. You go apple shopping. Now, all your accumulated knowledge about mangoes is worthless. You have to learn everything about the correlation between the physical characteristics and the taste of apples, by the same method of experimentation. You do it, because you love her.
Enter computer programs
Now, imagine that all this while, you were writing a computer program to help you choose your mangoes (or apples). You would write rules of the following kind:
if (color is bright yellow and size is big and sold by favorite vendor): mango is sweet.
if (soft): mango is juicy.
etc.
You would use these rules to choose the mangoes. You could even send your younger brother with this list of rules to buy the mangoes, and you would be assured that he will pick only the mangoes of your choice.
But every time you make a new observation from your experiments, you have to manually modify the list of rules. You have to understand the intricate details of all the factors affecting the quality of mangoes. If the problem gets complicated enough, it can get really difficult to make accurate rules by hand that cover all possible types of mangoes. Your research could earn you a PhD in Mango Science (if there is one).
But not everyone has that kind of time.
Enter Machine Learning algorithms
ML algorithms are an evolution over normal algorithms. They make your programs "smarter", by allowing them to automatically learn from the data you provide.
You take a randomly selected specimen of mangoes from the market (training data), make a table of all the physical characteristics of each mango, like color, size, shape, grown in which part of the country, sold by which vendor, etc (features), along with the sweetness, juicyness, ripeness of that mango (output variables). You feed this data to the machine learning algorithm (classification/regressio
Next time you go to the market, you measure the characteristics of the mangoes on sale (test data), and feed it to the ML algorithm. It will use the model computed earlier to predict which mangoes are sweet, ripe and/or juicy. The algorithm may internally use rules similar to the rules you manually wrote earlier (for eg, adecision tree), or it may use something more involved, but you don't need to worry about that, to a large extent.
Voila, you can now shop for mangoes with great confidence, without worrying about the details of how to choose the best mangoes. And what's more, you can make your algorithm improve over time (reinforcement learning), so that it will improve its accuracy as it reads more training data, and modifies itself when it makes a wrong prediction. But the best part is, you can use the same algorithm to train different models, one each for predicting the quality of apples, oranges, bananas, grapes, cherries and watermelons, and keep all your loved ones happy :)
And that, is Machine Learning for you. Tell me if it isn't cool.
Thursday, September 19, 2013
Saturday, September 14, 2013
English Language Coach
If you can pronounce correctly every word in this poem, you will be speaking English better than 90% of the native English speakers in the world.
After trying the verses, a Frenchman said he’d prefer six months of hard labour to reading six lines aloud.
Dearest creature in creation,
Study English pronunciation.
I will teach you in my verse
Sounds like corpse, corps, horse, and worse.
I will keep you, Suzy, busy,
Make your head with heat grow dizzy.
Tear in eye, your dress will tear.
So shall I! Oh hear my prayer.
Just compare heart, beard, and heard,
Dies and diet, lord and word,
Sword and sward, retain and Britain.
(Mind the latter, how it’s written.)
Now I surely will not plague you
With such words as plaque and ague.
But be careful how you speak:
Say break and steak, but bleak and streak;
Cloven, oven, how and low,
Script, receipt, show, poem, and toe.
Hear me say, devoid of trickery,
Daughter, laughter, and Terpsichore,
Typhoid, measles, topsails, aisles,
Exiles, similes, and reviles;
Scholar, vicar, and cigar,
Solar, mica, war and far;
One, anemone, Balmoral,
Kitchen, lichen, laundry, laurel;
Gertrude, German, wind and mind,
Scene, Melpomene, mankind.
Billet does not rhyme with ballet,
Bouquet, wallet, mallet, chalet.
Blood and flood are not like food,
Nor is mould like should and would.
Viscous, viscount, load and broad,
Toward, to forward, to reward.
And your pronunciation’s OK
When you correctly say croquet,
Rounded, wounded, grieve and sieve,
Friend and fiend, alive and live.
Ivy, privy, famous; clamour
And enamour rhyme with hammer.
River, rival, tomb, bomb, comb,
Doll and roll and some and home.
Stranger does not rhyme with anger,
Neither does devour with clangour.
Souls but foul, haunt but aunt,
Font, front, wont, want, grand, and grant,
Shoes, goes, does. Now first say finger,
And then singer, ginger, linger,
Real, zeal, mauve, gauze, gouge and gauge,
Marriage, foliage, mirage, and age.
Query does not rhyme with very,
Nor does fury sound like bury.
Dost, lost, post and doth, cloth, loth.
Job, nob, bosom, transom, oath.
Though the differences seem little,
We say actual but victual.
Refer does not rhyme with deafer.
Fe0ffer does, and zephyr, heifer.
Mint, pint, senate and sedate;
Dull, bull, and George ate late.
Scenic, Arabic, Pacific,
Science, conscience, scientific.
Liberty, library, heave and heaven,
Rachel, ache, moustache, eleven.
We say hallowed, but allowed,
People, leopard, towed, but vowed.
Mark the differences, moreover,
Between mover, cover, clover;
Leeches, breeches, wise, precise,
Chalice, but police and lice;
Camel, constable, unstable,
Principle, disciple, label.
Petal, panel, and canal,
Wait, surprise, plait, promise, pal.
Worm and storm, chaise, chaos, chair,
Senator, spectator, mayor.
Tour, but our and succour, four.
Gas, alas, and Arkansas.
Sea, idea, Korea, area,
Psalm, Maria, but malaria.
Youth, south, southern, cleanse and clean.
Doctrine, turpentine, marine.
Compare alien with Italian,
Dandelion and battalion.
Sally with ally, yea, ye,
Eye, I, ay, aye, whey, and key.
Say aver, but ever, fever,
Neither, leisure, skein, deceiver.
Heron, granary, canary.
Crevice and device and aerie.
Face, but preface, not efface.
Phlegm, phlegmatic, ass, glass, bass.
Large, but target, gin, give, verging,
Ought, out, joust and scour, scourging.
Ear, but earn and wear and tear
Do not rhyme with here but ere.
Seven is right, but so is even,
Hyphen, roughen, nephew Stephen,
Monkey, donkey, Turk and jerk,
Ask, grasp, wasp, and cork and work.
Pronunciation (think of Psyche!)
Is a paling stout and spikey?
Won’t it make you lose your wits,
Writing groats and saying grits?
It’s a dark abyss or tunnel:
Strewn with stones, stowed, solace, gunwale,
Islington and Isle of Wight,
Housewife, verdict and indict.
Finally, which rhymes with enough,
Though, through, plough, or dough, or cough?
Hiccough has the sound of cup.
My advice is to give up!!!
English Pronunciation by G. Nolst Trenité
After trying the verses, a Frenchman said he’d prefer six months of hard labour to reading six lines aloud.
Dearest creature in creation,
Study English pronunciation.
I will teach you in my verse
Sounds like corpse, corps, horse, and worse.
I will keep you, Suzy, busy,
Make your head with heat grow dizzy.
Tear in eye, your dress will tear.
So shall I! Oh hear my prayer.
Just compare heart, beard, and heard,
Dies and diet, lord and word,
Sword and sward, retain and Britain.
(Mind the latter, how it’s written.)
Now I surely will not plague you
With such words as plaque and ague.
But be careful how you speak:
Say break and steak, but bleak and streak;
Cloven, oven, how and low,
Script, receipt, show, poem, and toe.
Hear me say, devoid of trickery,
Daughter, laughter, and Terpsichore,
Typhoid, measles, topsails, aisles,
Exiles, similes, and reviles;
Scholar, vicar, and cigar,
Solar, mica, war and far;
One, anemone, Balmoral,
Kitchen, lichen, laundry, laurel;
Gertrude, German, wind and mind,
Scene, Melpomene, mankind.
Billet does not rhyme with ballet,
Bouquet, wallet, mallet, chalet.
Blood and flood are not like food,
Nor is mould like should and would.
Viscous, viscount, load and broad,
Toward, to forward, to reward.
And your pronunciation’s OK
When you correctly say croquet,
Rounded, wounded, grieve and sieve,
Friend and fiend, alive and live.
Ivy, privy, famous; clamour
And enamour rhyme with hammer.
River, rival, tomb, bomb, comb,
Doll and roll and some and home.
Stranger does not rhyme with anger,
Neither does devour with clangour.
Souls but foul, haunt but aunt,
Font, front, wont, want, grand, and grant,
Shoes, goes, does. Now first say finger,
And then singer, ginger, linger,
Real, zeal, mauve, gauze, gouge and gauge,
Marriage, foliage, mirage, and age.
Query does not rhyme with very,
Nor does fury sound like bury.
Dost, lost, post and doth, cloth, loth.
Job, nob, bosom, transom, oath.
Though the differences seem little,
We say actual but victual.
Refer does not rhyme with deafer.
Fe0ffer does, and zephyr, heifer.
Mint, pint, senate and sedate;
Dull, bull, and George ate late.
Scenic, Arabic, Pacific,
Science, conscience, scientific.
Liberty, library, heave and heaven,
Rachel, ache, moustache, eleven.
We say hallowed, but allowed,
People, leopard, towed, but vowed.
Mark the differences, moreover,
Between mover, cover, clover;
Leeches, breeches, wise, precise,
Chalice, but police and lice;
Camel, constable, unstable,
Principle, disciple, label.
Petal, panel, and canal,
Wait, surprise, plait, promise, pal.
Worm and storm, chaise, chaos, chair,
Senator, spectator, mayor.
Tour, but our and succour, four.
Gas, alas, and Arkansas.
Sea, idea, Korea, area,
Psalm, Maria, but malaria.
Youth, south, southern, cleanse and clean.
Doctrine, turpentine, marine.
Compare alien with Italian,
Dandelion and battalion.
Sally with ally, yea, ye,
Eye, I, ay, aye, whey, and key.
Say aver, but ever, fever,
Neither, leisure, skein, deceiver.
Heron, granary, canary.
Crevice and device and aerie.
Face, but preface, not efface.
Phlegm, phlegmatic, ass, glass, bass.
Large, but target, gin, give, verging,
Ought, out, joust and scour, scourging.
Ear, but earn and wear and tear
Do not rhyme with here but ere.
Seven is right, but so is even,
Hyphen, roughen, nephew Stephen,
Monkey, donkey, Turk and jerk,
Ask, grasp, wasp, and cork and work.
Pronunciation (think of Psyche!)
Is a paling stout and spikey?
Won’t it make you lose your wits,
Writing groats and saying grits?
It’s a dark abyss or tunnel:
Strewn with stones, stowed, solace, gunwale,
Islington and Isle of Wight,
Housewife, verdict and indict.
Finally, which rhymes with enough,
Though, through, plough, or dough, or cough?
Hiccough has the sound of cup.
My advice is to give up!!!
English Pronunciation by G. Nolst Trenité
Friday, September 6, 2013
Poll shows top CEOs prefer Modi to Rahul as PM: report
Nearly three-quarters of Indian business leaders believe the government has mismanaged the economy and want Bharatiya Janata Party (BJP) leader Narendra Modi to lead the country after an election due by May next year, according to an opinion poll published on Friday.
With India's 80-year-old Prime Minister Manmohan Singh expected to step aside, only 7 per cent of 100 chief executive officers surveyed for the Economic Times/Nielsen poll backed the ruling Congress party's Rahul Gandhi for the premiership.
Rahul represents the fourth generation of the Nehru-Gandhi dynasty that has led Congress, and India, for much of the time since independence from Britain in 1947. His late father, grandmother and great-grandfather were all prime ministers.
"After a long policy drought, CEOs are impatient for strong leadership, intent, decisions and action. Modi they seem to think has more to show than Gandhi on all these counts," the Economic Times said in its comments on the results of the poll.
The survey was conducted by Nielsen between August 1 and the beginning of September and covered the chief executive officers of companies worth more than Rs. 500 crore across different industries.
With India's 80-year-old Prime Minister Manmohan Singh expected to step aside, only 7 per cent of 100 chief executive officers surveyed for the Economic Times/Nielsen poll backed the ruling Congress party's Rahul Gandhi for the premiership.
Rahul represents the fourth generation of the Nehru-Gandhi dynasty that has led Congress, and India, for much of the time since independence from Britain in 1947. His late father, grandmother and great-grandfather were all prime ministers.
"After a long policy drought, CEOs are impatient for strong leadership, intent, decisions and action. Modi they seem to think has more to show than Gandhi on all these counts," the Economic Times said in its comments on the results of the poll.
The survey was conducted by Nielsen between August 1 and the beginning of September and covered the chief executive officers of companies worth more than Rs. 500 crore across different industries.
Indian business has in the past applauded Modi as an investor-friendly chief minister who has led Gujarat to double-digit economic growth.
Business leaders surveyed in the poll, however, thought that the economy has bottomed out with 42 percent forecasting a slight uptick in growth from the 4.4 per cent reported in the last quarter to over 5 percent this year and the next. That is still far below the level policy makers say is needed to create jobs for the millions of youth joining the workforce.
Business leaders surveyed in the poll, however, thought that the economy has bottomed out with 42 percent forecasting a slight uptick in growth from the 4.4 per cent reported in the last quarter to over 5 percent this year and the next. That is still far below the level policy makers say is needed to create jobs for the millions of youth joining the workforce.
Wednesday, September 4, 2013
Monday, September 2, 2013
India & A Blonde Tourist: An alternate account
India & A Blonde Tourist: An alternate account
Jane von Rabenau
I have read Michaela Cross's experiences in India several times by now. That makes me no different from a million plus people who too have read her blog.
What's different is that I could have gone through the same harrowing experience as her, because I too have been touring India extensively, alone. And yes, I too am white, in my 20s and a foreign female.
What's different is that I could have gone through the same harrowing experience as her, because I too have been touring India extensively, alone. And yes, I too am white, in my 20s and a foreign female.
Why has India been so different to me? Have I been simply lucky? Or have I looked at India very differently to get a very different treatment for myself?
Let me compare notes with some of Michaela's experiences to explain what I am saying.
Do Indian men stare at me? Yes, they do... and so do Indian women and kids, and other European travelers.
I guess that is a natural reaction to somebody considered exotic.
When I took my Eritrean friend to my grandmother's village in northern Germany, where foreigners are a rarity, she attracted everyone's looks, some containing a hint of racism.
The stares I got from Italian men were typically accompanied with a "Ciao Bella!" This flirtatious attitude is often welcomed and accepted as part of Italy's macho culture.
Do people take photos of me? Yes, they do... but I take many more photos of them!
We Western travellers typically shoot every monument, sight and many people they come across in India, mostly without asking for permission.
These pictures are posted in our Facebook page or travel blogs. But if an Indian takes a photo of a European, we get irritated and feel our privacy is invaded.
Again, isn't there a double standard here? A white skin's privilege is a brown skin's punishment?
Am I the centre of attention at social events? Yes, I am. Thanks to the immense hospitality of Indians, I had the opportunity to attend five weddings and several festivals.
When I was dancing, lots of people wanted to dance with me, and some also took photos and videos of me.
I was the only white person at these functions, and most people had never seen a white woman dancing to Bollywood tunes.
In my case, the attention I got in these functions was no different from the curious and welcoming attention I received at a wedding party in Kosovo.
Did I have any negative experiences with Indian men? Yes, the worst I had was a businessman in my Air India flight from London try to grope my thigh.
But my positive experiences far outweigh the negative ones, even with men. I only had to deal with people trying to sell me stuff and not leaving me alone; and people staring at me.
However, putting myself in the position of a crafts seller trying to feed his family, and knowing that there is a chance that after annoying a tourist enough, he will give in and buy something, I would also prioritise my family over the tourist.
I have hardly ever had to stay in hotels as Indians have welcomed me to stay at their family homes, and then organised me to stay at their wife's cousin's friend's house etc...
On many occasions, dhaba owners or fruit sellers insisted on not taking money from me for the food as I am a guest of India.
Can one generalise my account of India? No, one cannot.
In fact, no one's account can and should be generalised -- one sixth of humanity lives in India; there are many Indias in India; every traveller interacts only with a small fraction of Indians, and can thus only give a tiny fragment of the true Indian experience -- whatever that is.
But I believe that we make our experiences as much as our experiences make us.
I have generally been more adapting, less suspicious and more trusting.
For instance, on a recent visit to Kasauli with a female English friend, we wore Kurtis and bangles and joked and chatting with every Indian we interacted with-- chaiwalas, pandits, other Indian tourists.
We started chatting with clothes shop owner and had chai with him. A friend of his insisted on showing us around Kasauli and inviting us to his village.
We ended up having dinner with his family - some delicious daal, sabzi and chawal and looked at his beautiful family photo album.
I feel that many a traveller would have a much more exciting, and "real" experience of India if they would just be a more open and friendly towards Indians.
I am not suggesting India is a heaven for women. You don't need to hear from me the depressing daily occurrences of molestation, sexual assaults and female infanticide.
However, my experience of India and behaviour of Indians towards me has been incredibly positive. Many of my friends had similar experience.
I hope India treats more foreign travellers like it has treated me. Rather than the treatment Michaela received.
Soon, I will get back to London to continue my Bachelor's at LSE. I will have to adjust to a life without any special attention - no ghaar ka khaana from chaiwalas, no chaat papri, no Bollywood dancing and no poojas.
Maybe I will refuse to readjust and come back to India next year.
(Jane von Rabenau, 21, grew up in Frankurt, Germany. She is studying Philosophy and Economics at London School of Economics. She is on a summer internship to Delhi, working in the area of development cooperation. She has travelled across India and many parts of the world extensively)
Here is a link to Michaela Cross' blog that stirred the debate:
http://ireport.cnn.com/docs/DOC-1023053
Subscribe to:
Posts (Atom)