Setting a max heap on fuse-dfs
May 17th, 2011
fuse-dfs is the fuse plugin for HDFS (Hadoop Distributed File System). fuse-dfs creates a JVM through the native api and proxies fuse requests to the java api. This means the fuse-dfs process will encapsulate a JVM.
The problem is that by default that JVM will consume a fair amount of memory unless you set a max heap. The code allows you to push JVM options via the LIBHDFS_OPTS environment variable.
If using CDH, you can do this by setting a file in /etc/default/:
$ cat /etc/default/hadoop-0.20-fuse
export LIBHDFS_OPTS="-Xms128m -Xmx128m"
Leave a Reply