MemoryOutOfBound Error

Today when I reached office, my manager called me to look into a production issue. We were getting some MemoryOutOfBound error. He was sure that application was not huge and does not use much memory, so the error should be something else than memory not being available.

We tweaked the settings to get the dump of the heap. And systems were failing regularly. Once we bounce the server, server starts working and another server fails in 15-20 minutes. I checked memory status using jmap, and heap size was no where close to 2 GB. Development tech lead told that the system was working fine when may heap size was 512 mb in stead of 2 GB, so we reverted back to 512 mb. And suddenly it started working. And that was right, reducing heap size helped. The reason was,

We were using a 32 bit system with 4 GB memory. OS was using 2 GB for its internal usage. When we provide 2 GB for JVM heap, there were little memory left for any native calls such as creating OS thread, opening a socket etc. And we used to get MemoryOutOfBound error. Reducing it to 512 MB helped.