Statistics is like a mini skirt

I had some spared time last month and had some spared money. So I thought of investing in share market. Before investing I started my own research, and found couple of guys who give HOT TIPS. Importantly they make their old tips public as how successful they are. It comes with nice charts and all. Impressive! I was looking at their tips. Their tips spanned for one year when Sensex was in 9000 to when sensex zoomed to 17000. The investment I made when sensex was at 21,000 and have not touched during bearish market fared better than the tip period. I looked for tips when market was in 21,000 date and no tips. Wrote a mail to the customer care and got a reply saying they don't keep data beyond a year. Nice way to put data we actually give good tips when tips is no better than market sentiment.


One of my friend was buying a home in 2005. He was looking at various housing loans. An agent from a private leading bank approached him with all the details. He said bank provides 2 kind of home loan interest 1)Fixed rate and 2) floating rate. Fixed rate Interest was about 1 percent higher than the floating rate. I advised him to go for fixed rate as the interest rate is all time high and can go up at any time. And he almost agreed. Then came our counselor, the home loan agent. He said rate has never changed in last 25 years for people who opted for floating interest rate. So most probable it wont change either. Why to pay extra one more percent. And my friend got convinced and signed the document. I have not asked the question on the spot, but later realized the interest rate went down with time in last 26 years. And bank never handed the facility to borrower. Will it continue to do the same in case rate goes up? And my doubt turned true. He got loan with 7 percent and paying an interest of 11.5 percent.

Another interview

I have taken 100s of interviews. Interviewing and short listing the right candidate is one of the key requirement for my job. Last Friday my boss and HR called me to take a couple of interviews. I was not surprised as I take interviews. But then my boss said, the interviews were not for JAVA professionals, rather it is for QA engineers. He then told me the mandate. I would be looking at the skill sets beyond functional testing.

First round of the interviews were to be conducted by our senior QA engineers, who would look at functional testing use case writing etc. I would look at the knowledge beyond QA. I would look at stretch goals such as do they know how to deploy application, how to look logs and provide some useful information to development beyond a bug. Can they help development reproduce the bug in development if it were not reproducible. We have an aggressive growth plan, so need resources urgently, but we don't want to hire the wrong person. So I was anxious to do the job and tensed if I will be able to do it right.

I met with a senior QA engineer to understand what they are looking for. What are the typical questions they ask. I was working with the QA team closely, so I have a clear understanding about their work and expectations. Then came the interview time, I have to meet the candidates after met by a senior QA engineers. I was tensed for the first guy, and the tension reduced gradually. Asked them from a range of questions unit testing, functional testing to some UNIX commands and deployment and puzzles.

I figured out it was just another day with a new experience.

Statistics is like a mini skirt

and it hides more than it reveals.

Infosys was one of the best Indian corporate. I was surpised to find out that Infosys more competative than Harvard. I ask the fundamental question, what is harder? How do I decide whether X or Yis harder? And x and y can be Infosys, Harvard or any other entity. To me if I (or simillar people who wish to join both Infosys and Harvard) can get x but not y then y is harder for me. By doing the same for large number of people we can figure out what is harder. Another approach would be give Y to people who have X and X to people who have Y. In case many people who have Y are ready join X, then X is better and vice versa. Given that, many infoscians would be happy to join Harvard but not the other way. But statistics says the Infosys has a higher rejection ratio, so a tough place to get in.

Currently I was looking at Silicon India article, which said average salary of woman is 30 percent less than that of ab average man in Indian IT industry. I was stunned to read the news. I have not come across any IT organization which makes salary discrimination based on sex. I thought may be my knowledge is limited or I worked only at companies which does not discriminate or both. But I read the line again, it said the average salary of Indian females is less than that of average males. It can be possible because the ratio of men to women is higher for senior people who draw higher salary than the ratio in the lower salary. For example say a freshers gets 3,00,000 per anum and a 5 years exp clad gets 12,00,000 per anum. And the male to female ratio at 5 years exp is 3:1 which is 2:1 for freshers and the ratio of fresher to 5 yr exp is 3:1

We can take the smallest number with M1,M2,M3,F1 getting 12,00,000 per anum and M4,M5,M6,M7,M8,M9,M10,M11,F2,F3,F4,F5 getting 3,00,000 per anum.

Average salary of males = 5,45,454 per anum
Average Salary of females = 4,80,000 per anum

Which says the average salary is lower for woman. The figure gets even worse with higher ratio between fresher to senior position, higher salary difference and lower ratio at senior positions and higher ratio at fresher level. Of course it gets better with the opposite conditions, but truth will make the average salary look lower.

I would like to know the sample size and how average salary is calculated before jumping the gun and say women in IT paid lower than the men.

Then ave sal of male =

Then average sal of male = 12

Forced to paste against my and company's will

I had a policy from Bajaj Allianz. Today I tried to register for the same. It has form fields such as name id etc. It has interesting questions like mail id and confirm mail id. I typed my mail id. It take the whole mail id when I typed pa. Looked cool. When I am trying to enter confirm mail id, it did not read from the previous entered data, rather wanted me to feed it. Looked good. Manual error in copy pasting is handled well.

After putting my id and before putting the domain name I have to put @. Unfortunately I have a laptop with Swiss key board where @ comes with key stroke Alt GR + 2 instead of Shift + 2. I am habituated with it. So typed the same. Err.. It prompted me with a message you can not copy paste here. I have the option of disabling the java script and type it. Then enable the java script. Or to register from office. Both will take whatever I have entered away and have to do it again.

I copied my mail id from the mail id and pasted it in the confirm mail id by clicking right click paste.

Bajaj Allianz has a good intention of not allowing paste in the confirm mail id. But it left me with only option of pasting me the mail id. I am forced to do what Bajaj Allianz did not want. I do not understand why Alt Gr key is not allowed but right click is allowed in a site. And in case it wanted to stop pasting, then it should have blocked ctl+v instead of simillar sounding Alt GR key. I have posted the same as a feedback t bajaj Allianz and will post their reponse if any.

The error is a smaller one and won't loose any customers. I would have continued at Bajaj Allianz irrespective of the outcome. However sometimes product miss what they intend to do.

MemoryOutOfBound Error

Today when I reached office, my manager called me to look into a production issue. We were getting some MemoryOutOfBound error. He was sure that application was not huge and does not use much memory, so the error should be something else than memory not being available.

We tweaked the settings to get the dump of the heap. And systems were failing regularly. Once we bounce the server, server starts working and another server fails in 15-20 minutes. I checked memory status using jmap, and heap size was no where close to 2 GB. Development tech lead told that the system was working fine when may heap size was 512 mb in stead of 2 GB, so we reverted back to 512 mb. And suddenly it started working. And that was right, reducing heap size helped. The reason was,

We were using a 32 bit system with 4 GB memory. OS was using 2 GB for its internal usage. When we provide 2 GB for JVM heap, there were little memory left for any native calls such as creating OS thread, opening a socket etc. And we used to get MemoryOutOfBound error. Reducing it to 512 MB helped.

Paid to make mistake

In one of the leadership training, the trainer said we are paid to make mistake! I did not agree with him. How can one be paid to make mistake? I am paid to write good software and he is paid to provide appropriate training. And neither he nor I worth the salary if we don't do our job well. How are we paid to make mistake? He clarified, we are paid to make mistake and not for not doing our tasks. He explained his view:

Man is to err and will make mistake. We do mistake, even Vivek Paul makes mistake. But we should not repeat our mistakes. We should learn from mistakes. Anybody who makes and repeats mistake needs to be fired (too harsh, isn't it?). So one has the option of working in the comfort zone and not making any mistake or stretching her ability and do something new and may make mistake. And we are all paid to grow and stretch ourselves, do new things which is out of our comfort zone and make mistakes.

I was reading Michael Nygard's Release It!: Design and Deploy Production-Ready Software (Pragmatic Programmers). He also agreed with the trainer. He claims he does make mistake, but all the mistakes are unique. Once he learns about a mistake, he ensures that it does not happen twice. And as a cynic he expects something will go wrong in the production.

I got a different prospective from my manager. He at certain cases could have put the blame on certain mistakes on some of the team members including me. In stead he took the responsibility and ensured that the mistake did not happen again. It is not about blaming others but owning the mistake, fixing it and moving forward ensuring it does not happen again.

Doing it right at right time

I am following cricket like most Indians. Currently I am following CLT20. Victoria played 4 matches won 2 and lost 2 matches, so does Delhi and Bangalore. But Victoria went to semifinal where as Bangalore and Delhi were knocked out. Though Victoria have better run rate than either Delhi or Bangalore, it went to semifinal based on points without considering the run rate. I was looking into the All Star 11 It has 3 players from Bangalore, 2 from Delhi and only one from Victoria (so does 1 from Cape Cobra and New South Wales). So despite showing better individual brilliance and 2 teams have been knocked out of the tournament.

Both Delhi and Bangalore knocked out because they lost the group match to a team which went to the next round. So they went to the next round but with 0 points. Victoria lost a match but to a team which was eliminated from the group stage. That was the differentiator between Victoria and other teams.

So it is not only a mistake (loosing a match) but the mistake which got added (carried forward points) costed Bangalore and Delhi.