dfs.hosts.exclude excludes
$ hadoop dfsadmin -refreshNodes $ hadoop dfsadmin -report
$ hadoop-daemon.sh stop datanode $ hadoop-daemon.sh stop tasktracker
MongoCursor::$timeout = -1;
cursor timed out (timeout: 30000, time left: 0:0, status: 0)。参照 记一次MongoDB性能问题 找问题原因，也没解决。直到看了 PHP Mongo's driver has no effects on Mongo Cursor timeout 这里才知道，设置超时需要按上面来设置。shell> strace -f -r -p通过如下命令查询recvfrom操作的含义：recvfrom( ,
shell> apropos recvfrom receive a message from a socket或者按照下面的方式确认一下：
shell> lsof -pshell> ls -l /proc/ /fd/
libboost_python-py27.so.1.53.o No such file or directory
sudo apt-get install libboost-all-dev 后，再执行，错误依旧存在。dpkg -S libboost-python 出来的是一堆 /usr/share 下的文档， .so 还是没影。ls -al /usr/lib/i386-linux-gnu/libboost*后，果真 libboost 库在这目录下。[26641] 21 Dec 09:46:34 * Slave ask for synchronization [26641] 21 Dec 09:46:34 * Starting BGSAVE for SYNC [26641] 21 Dec 09:46:34 # Can’t save in background: fork: Cannot allocate memory [26641] 21 Dec 09:46:34 * Replication failed, can’t BGSAVE [26641] 21 Dec 09:46:34 # Received SIGTERM, scheduling shutdown… [26641] 21 Dec 09:46:34 # User requested shutdown…很明显的一个问题，系统不能在后台保存，fork进程失败。
[26641] 18 Dec 04:02:14 * 1 changes in 900 seconds. Saving… [26641] 18 Dec 04:02:14 # Can’t save in background: fork: Cannot allocate memory系统不能在后台保存，fork 进程时无法指定内存。
int rdbSaveBackground(char *filename) {
pid_t childpid;
long long start;
if (server.bgsavechildpid != -1) return REDIS_ERR;
if (server.vm_enabled) waitEmptyIOJobsQueue();
server.dirty_before_bgsave = server.dirty;
start = ustime();
if ((childpid = fork()) == 0) {
/* Child */
if (server.vm_enabled) vmReopenSwapFile();
if (server.ipfd > 0) close(server.ipfd);
if (server.sofd > 0) close(server.sofd);
if (rdbSave(filename) == REDIS_OK) {
_exit(0);
} else {
_exit(1);
}
} else {
/* Parent */
server.stat_fork_time = ustime()-start;
if (childpid == -1) {
redisLog(REDIS_WARNING,"Can't save in background: fork: %s",
strerror(errno));
return REDIS_ERR;
}
redisLog(REDIS_NOTICE,"Background saving started by pid %d",childpid);
server.bgsavechildpid = childpid;
updateDictResizePolicy();
return REDIS_OK;
}
return REDIS_OK; /* unreached */
}
The SecondaryNameNode is not a "backup NameNode". It cannot take over the primary NameNode's function. It serves as a checkpointing mechanism for the primary NameNode. In addition to storing the state of the HDFS NameNode, it maintains two on-disk structures that persist the current filesystem state: an image file and an edit log. The image file represents an HDFS metadata state at a point in time, and the edit log is a transactional log(compare to a log in a database architecture) of every filesystem metadata change since the image file was created.
During the NameNode (re)starts, the current state is recontstructetd by reading teh image file and then replaying the edit log. Obviousely, the larger the edit log is, the longer is takes to replay it and consequently start a NameNode. To improve NameNode startup performance, an edit log is priodically rolled, and a new image file is created by applying an edit log to the existing image. This operation can be fairly resource-intensive. To minimize the impact of checkpoint creation and the NameNode functioning, checkpointing is performed by the SecondaryNameNode daemon, often on a separate machine.
As a result of checkpointing, the Secondary NameNode holds a copy (out-of-date) of the primary's persistent state in the form of last image file. In the cases when and edit file is kept relatively small, a secondary node can be used to recover the filesystem's state. In this case, you must be aware of a certain amount of metadata (and corresponding data) loss, because the latest changes stored in the edit log are not available.
One of the misconceptions about Hadoop is the assumption that smaller blocks(less then the block size)will still use the whole block on the filesystem. This is not the case. The smaller blocks occupy exactly as much disk space as they require.
But this does not mean that having many small files will use HDFS efficiently. Regardless of the block size, its metadata occupies exactly the same amount of memroy in the NameNode. As a result, a large number of small HDFS files(smaller than the block size) will use a lot of the NameNode's memory, thus negatively impacting HDFS scalability and performance.
When a file is opened for writing, opening client is granted an exclusive writing lease for the file. This means that no other client can write to this file until this client completes the operation. To ensure that no "runaway" clients are holding a lease, the lease priodically expires. The use of leases effectively ensures that no two applications can simultaneously write to a given file(compared to a write lock in the database).
The lease duration is bound by a soft limit and a hard limit. For the duration of a soft limit, a writer has an exlusive access to the file. If the soft limit expires and the client fails to close the file or renew the lease(by sending a heartbeat to the NameNode), another client can preempt the lease. If the hard limit(one hour) expires and the client has failed to renew the lease, HDFS assumes that the client has quit, and automatically closes the file on behalf the writer, and then recovers the lease.
The writer's lease does not prevent other clients from reading the file. A file may have many concurrent readers.
| FIELD | DISCRIPTION |
|---|---|
| Version | A 4-byte array conaining three letters(SEQ) and a sequence file version number (either 4 or 6). The currently used version is 6. Version 4 is supported for backward compatibility. |
| Key Class | Name of the key class, which is validated against a name of the key class provided by the reader. |
| Value Class | Name of the value class, which is validated against a name of the key class provided by the reader. |
| Compression | A key/value compression flag |
| Block Compression | A block compression flag |
| Compression Codec | CompressionCodec class. This class is used only if either key/value or block compression flag are true. Othersize, this value is ignored. |
| Sync | A sync makeer |
A sync is specialized marker, which is used for faster search inside SequenceFile. A sync marker also use in MapReduce implementation - data splits are done only on sync boundaries.
| FIELD | DESCRIPTION |
|---|---|
| Record Length | The length of the record(bytes) |
| Key Length | The length of the key(bytes) |
| Key | Byte array, containing the record's key |
| Value | Byte array, containing the record's value |
| FIELD | DESCRIPTION |
| Keys lengths length | In this case, all the keys for a given block are sorted together. This field specified compressed key-lengths size(in bytes) |
| Keys Lengths | Byte array, containing compressed key-lengths block |
| keys length | Compressed keys size(in bytes) |
| Keys | Byte array, containing compressed keys for a block |
| Values lengths length | In this case, all the valeus for a given block are stored together. This field specifies compressed value-lenghts block size(in bytes) |
| Values Lengths | Byte array, containing compressed value-lengths block |
| Values length | Compressed value size(in bytes0 |
| Values | Byte array, containing compressed values for a block |
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path path = new Path("filename");
SequenceFile.Writer sequenceWriter = new SequenceFile.Writer(fs, conf, path, Key.class, Value.class, fs.getConf().getInt("io.file.buffer.size", 4096), fs. getDefaultReplication(), 1073741824, null, new Metadata());
.......................................
SequenceWriter.append(bytesWritable, bytesWritable);
.......................................
IOUtils.closeStream(sequenceWriter);
A minimal SequenceFile writer constructor(SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass)) requires the specification of the filesystem, Hadoop configuration, path(file location), and definition of both key and value classes. A constructuctor that is used in the previous example enables you to specify additonal file parameters, including the following:A bloom filter is a space-efficient, probabilistic data structure that is used to test whether an element is a member of a set. The result of the test is that the element either definitely is not in the set or may be in the set.
The base data structure of a bloom filter is a bit vector. The probability of false positives depends on the size of the element's set and size of the bit vector.
Although risking false positive, bloom filters have a strong space advantage over other data structures for representing sets, such as self-balancing binary search trees, tries, hash tables, or simple arrays or linked lists of the entries. Most of these require storing at least the data items themselves, which can require anywhere from a small number of bits(for small integers) to an arbitrary number of bits, such as for strings.(Tries are an exception, because they can share storage between elements with equal prefixes.)
This advantage of a bloom filter comes partly from its compactness(inherited from arrays), and partly from its probabilistic nature).
DATA COMPRESSION
An important consideration for storing data in HDFS files is data compression, shifting the computation load in data processing from I/O to CPU. As shown in several publications, providing a systematic evalation of the compute versus I/O trade-offs when using compression for MapReudce implementation, the benefits of data compression depend on the type of data-processing jobs. For read-heavy(I/O bound)applications(for example, text data processing), compressing given 35 to 60 percent performance savings. On the other hand, for compute-intensive applications(CPU bound), performance gains from data compression are negligible.
This does not mean that data compression is not advantageous for such applications. Hadoop clusters are shared resources, and as a result, a diminishing I/O load for one application increases the capability of other applications to use this I/O.
Does this mean that data compression is always desirable? The answer is "no". For example, if you are using text or custom binary input files. data compression might be undesirable, because compressed files are not splittable. On the other hand, in the case of SequenceFile and their derivatives, comperssion is always desirable. Finally, it always makes sense to compress the intermediate files used for shuffle and sort.
Keep in mind that the result of data compression depend greatly on the type of data being compressed and the compression algorithm.
Although HBase does not support real-time joins and queries, bach joins and/or queries via MapReduce can be easily implemented. In fact, they are well-supported by high-level systems such as Pig and Hive, which use a limited SQL dialect to execute operations.
One of the HBase optmizatiion techniques is disabling the writes to the WAL. This represents a trade-off between performance and reliability. Disabling writes to the WAL prevent recovery when a region server fails before a write operation completes. You should use such an optimization with care, and only in cases when either data loss is acceptable, or a write operation can be "replayed" based on an additional data source.
ZOOKEEPER
Zookeeper is a replicated synchronization service with eventual consistency. It is robust, because the persisted data is distributed between multiple nodes(this set of nodes is called an ensemble) and a client connects to any of them(that is, a specific "server"), migrating if a given server fails. As long as a strict majority of nodes are working, the ensemble of Zookeeper nodes is alive.
Zookeeper's master node is dynamically chosen by consensus within the ensemble. If the master node fails, the remainder of nodes picks a new master. The master is the authority for writes. This guarantees that the writes are persisted in-order(that is, writes are linear). Each time a client writes to the ensemble, a majority of nodes persist the information. This means that each write makes the server up-to-date with the master.
A canonical example of Zookeeper usage is distributed-memory computation, where some data is shared between client nodes, and must be accessed/updated in a very careful way to account for synchronization. Zookeeper offers the library to construct custom synchronization primitives, while the capability to run a distributed server avoids the single-point-of-failure issue you have when using a centralized message repository.
One thing to be aware of is the fact that HBase operates on byte arrays. All of the components of HBase data - keys, column family names, and column names - are treated by HBase as array of uninterpereted bytes. This means that all interval value comparisons and, consequently, sorting, is done in lexicographical order. this is very important to remember, especially for the row keys design to avoid unpleasant surprise. A typical example is usage of integer keys. If they are not left padded to have the same length, then, as a result of HBase sorting, a key 11, for example, will appear before 5.(key 按词典顺序进行排序，所以，假设用整数做 key，在它们未做左补 0 时，字符长度不同，这时进行排序时，key 为 11 的就会排在 5 的前面。)
| KEY | TIMESTAMP | COLUMN FAMILY:COLUMN NAME | VALUE |
|---|---|---|---|
| Row key | Timestamp of the last update | Family:column | Value |
The SecondaryNameNode is not a "backup NameNode". It cannot take over the primary NameNode's function. It serves as a checkpointing mechanism for the primary NameNode. In addition to storing the state of the HDFS NameNode, it maintains two on-disk structures that persist the current filesystem state: an image file and an edit log. The image file represents an HDFS metadata state at a point in time, and the edit log is a transactional log(compare to a log in a database architecture) of every filesystem metadata change since the image file was created.
During the NameNode (re)starts, the current state is recontstructetd by reading teh image file and then replaying the edit log. Obviousely, the larger the edit log is, the longer is takes to replay it and consequently start a NameNode. To improve NameNode startup performance, an edit log is priodically rolled, and a new image file is created by applying an edit log to the existing image. This operation can be fairly resource-intensive. To minimize the impact of checkpoint creation and the NameNode functioning, checkpointing is performed by the SecondaryNameNode daemon, often on a separate machine.
As a result of checkpointing, the Secondary NameNode holds a copy (out-of-date) of the primary's persistent state in the form of last image file. In the cases when and edit file is kept relatively small, a secondary node can be used to recover the filesystem's state. In this case, you must be aware of a certain amount of metadata (and corresponding data) loss, because the latest changes stored in the edit log are not available.
One of the misconceptions about Hadoop is the assumption that smaller blocks(less then the block size)will still use the whole block on the filesystem. This is not the case. The smaller blocks occupy exactly as much disk space as they require.
But this does not mean that having many small files will use HDFS efficiently. Regardless of the block size, its metadata occupies exactly the same amount of memroy in the NameNode. As a result, a large number of small HDFS files(smaller than the block size) will use a lot of the NameNode's memory, thus negatively impacting HDFS scalability and performance.
When a file is opened for writing, opening client is granted an exclusive writing lease for the file. This means that no other client can write to this file until this client completes the operation. To ensure that no "runaway" clients are holding a lease, the lease priodically expires. The use of leases effectively ensures that no two applications can simultaneously write to a given file(compared to a write lock in the database).
The lease duration is bound by a soft limit and a hard limit. For the duration of a soft limit, a writer has an exlusive access to the file. If the soft limit expires and the client fails to close the file or renew the lease(by sending a heartbeat to the NameNode), another client can preempt the lease. If the hard limit(one hour) expires and the client has failed to renew the lease, HDFS assumes that the client has quit, and automatically closes the file on behalf the writer, and then recovers the lease.
The writer's lease does not prevent other clients from reading the file. A file may have many concurrent readers.
| FIELD | DISCRIPTION |
|---|---|
| Version | A 4-byte array conaining three letters(SEQ) and a sequence file version number (either 4 or 6). The currently used version is 6. Version 4 is supported for backward compatibility. |
| Key Class | Name of the key class, which is validated against a name of the key class provided by the reader. |
| Value Class | Name of the value class, which is validated against a name of the key class provided by the reader. |
| Compression | A key/value compression flag |
| Block Compression | A block compression flag |
| Compression Codec | CompressionCodec class. This class is used only if either key/value or block compression flag are true. Othersize, this value is ignored. |
| Sync | A sync makeer |
A sync is specialized marker, which is used for faster search inside SequenceFile. A sync marker also use in MapReduce implementation - data splits are done only on sync boundaries.
| FIELD | DESCRIPTION |
|---|---|
| Record Length | The length of the record(bytes) |
| Key Length | The length of the key(bytes) |
| Key | Byte array, containing the record's key |
| Value | Byte array, containing the record's value |
| FIELD | DESCRIPTION |
| Keys lengths length | In this case, all the keys for a given block are sorted together. This field specified compressed key-lengths size(in bytes) |
| Keys Lengths | Byte array, containing compressed key-lengths block |
| keys length | Compressed keys size(in bytes) |
| Keys | Byte array, containing compressed keys for a block |
| Values lengths length | In this case, all the valeus for a given block are stored together. This field specifies compressed value-lenghts block size(in bytes) |
| Values Lengths | Byte array, containing compressed value-lengths block |
| Values length | Compressed value size(in bytes0 |
| Values | Byte array, containing compressed values for a block |
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path path = new Path("filename");
SequenceFile.Writer sequenceWriter = new SequenceFile.Writer(fs, conf, path, Key.class, Value.class, fs.getConf().getInt("io.file.buffer.size", 4096), fs. getDefaultReplication(), 1073741824, null, new Metadata());
.......................................
SequenceWriter.append(bytesWritable, bytesWritable);
.......................................
IOUtils.closeStream(sequenceWriter);
A minimal SequenceFile writer constructor(SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass)) requires the specification of the filesystem, Hadoop configuration, path(file location), and definition of both key and value classes. A constructuctor that is used in the previous example enables you to specify additonal file parameters, including the following:A bloom filter is a space-efficient, probabilistic data structure that is used to test whether an element is a member of a set. The result of the test is that the element either definitely is not in the set or may be in the set.
The base data structure of a bloom filter is a bit vector. The probability of false positives depends on the size of the element's set and size of the bit vector.
Although risking false positive, bloom filters have a strong space advantage over other data structures for representing sets, such as self-balancing binary search trees, tries, hash tables, or simple arrays or linked lists of the entries. Most of these require storing at least the data items themselves, which can require anywhere from a small number of bits(for small integers) to an arbitrary number of bits, such as for strings.(Tries are an exception, because they can share storage between elements with equal prefixes.)
This advantage of a bloom filter comes partly from its compactness(inherited from arrays), and partly from its probabilistic nature).
DATA COMPRESSION
2NN 向 NN 获取镜像的地址是： http://${dfs.http.address}:50070/getimage?putimage=1&port=50090&machine=0.0.0.0&token=x&newChecksum=x， dfs.http.address 的默认地址是： 0.0.0.0:50070，当 2NN 和 NN 是不同主机时，该值必须指定(${namenode-ip:50070 or namenode-hostname:50070})，否则获取镜像文件失败。dfs.http.address master1:50070 dfs.secondary.http.address master2:50070 fs.checkpoint.period 3600 fs.checkpoint.size 67108864 fs.checkpoint.dir /hadoop/dfs/namesecondary
master2master2 配置文件实际上是用来指定 SecondaryNameNode。所以内容是 master2，估计大部分 Hadoop 的初学者都会被其名字所迷惑。
From the output of the two(read/write consistency check) commands, we know that writing is more expensive than reading for the HDFS cluster. This is because the write operations, in turn, involve more operations, such as computing and recordign the checksums of the data blocks and many more.
export HADOOP_VERSION=hadoop-1.1.2
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/conf
export USE_REAL_DATASET=FALSE
export APP_JAR=${HADOOP_hOME}/hadoop-test-*.jar
export EXAMPLE_JAR=${HADOOP_HOME}/hadoop-examples-*.jar
export STREAMING_JAR=${HADOOP_HOME}/contrib/streaming/hadoop-streaming-*.jar
These two properties specify that we will use 10 small stream sort jobs with 6 reducers, and 5 small stream sort jobs with 3 reducers.streamSort.smallJobs.numOfJobs 10,5 streamSort.smallJobs.numOfReduces 6,3
export HADOOP_HOME=/usr/local/hadoop
export GRID_MIX_HOME=$HADOOP_HOME/src/benchmarks/gridmix
export APP_JAR=${HADOOP_HOME}/hadoop-test-*.jar
export EXAMPLE_JAR=${HADOOP_HOME}/hadoop-examples-*.jar
export STREAMING_JAR=${HADOOP_HOME}/contrib/streaming/hadoop-streaming-*.jar
export GRID_MIX_DATA=/gridmix1/data
export GRID_MIX_PROG=/gridmix1/programs
The last two environment variables GRID_MIX_DATA and GRID_MIX_PROG specify two directories on HDFS. So, the generated data and programs will be on HDFS.
Hibench is a benchmarking suite for Hadoop. It has nine typical workloads, including mircro, HFS, web search machine learning, and data analytics benchmarks. For example, it supports benchmarks for Nutch(a text indexing software package), PageRank(the PageRank algorithm), the Mahout machine learning argorithms and Hive queries.
The Hadoop balancer balances data blocks on HDFS according to a preconfigured threshold value, which sets the target for whether the cluster is balanced or not. A node id considered balanced if the difference between space utilization of the node and space utilization of the cluster is less than the threshold.
Sometimes, we want to limit the percentage of bandwidth used by the balancer. By default, Hadoop defines a property dfs.balance.bandwidthPerSec, which determines the maximum speed that a data block will be moved from one DataNode to another. Its default value is 1MB/s. By configuring this property to be a higher value, the balancing speed will be faster, but more resources will be used. For example, to change the value of this property to be 10MB/s, we can change this in the $HADOOP_HOME/conf/hdfs-site.xml.
mapred.output.compress true
mapred.output.compression.codec org.apache.hadoop.io.compress.GzipCodec
mapred.output.compression.type BLOCK
mapred.compress.map.output true
mapred.map.output.compression.codec org.apache.hadoop.io.compress.GzipCodec
for host in `cat $HADOOP_HOME/conf/slaves` do echo 'Copying map-site.xml file to host:' $host scp $HADOOP_HOME/conf/mapred-site.xml $host:$HADOOP_HOME/conf/ done
| Codec name | Java class |
| DefaultCodec | org.apache.hadoop.io.compress.DefaultCodec |
| GzipCodec | org.apache.hadoop.io.compress.GzipCodec |
| BZip2Codec | org.apache.hadoop.io.compress.BZip2Codec |
| SnappyCodec | org.apache.hadoop.io.compress.SnappyCodec |
| LzoCodec | org.apache.hadoop.io.compress.LzoCodec |
mapred.map.tasks.speculative.exeution false
mapred.reduce.tasks.speculative.execution false
mapreduce.job.speculative.speculativecap 0.2
mapreduce.job.speculative.slowtaskthreshold 1.0
mapreduce.job.speculative.slownodethreshold 1.0
Configuration conf = new Configuration();
conf.set("mapred.map.tasks.speculative.execution", "false");
conf.set("mapred.reduce.tasks.speculative.execution", "false");
The three properties mapreduce.job.speculative.speculativecap, mapreduce.job.speculative.slowtaskthreshold, and mapreduce.job.speculative.slownodethreshold control when the JobTracker should start a speculative task. Specifically, a speculative task for a regular task will start when the following conditions are met:
The number of map and reduce slots determines the number of concurrent map/reduce tasks for a TaskTracker, which forks multiple JVMs to run these tasks.
Configure map/reduce slots for a TaskTracker:mapred.tasktracker.map.tasks.maximum 4
mapred.tasktracker.reduce.tasks.maximum 4
mapred.child.java.opts -Xmx1024m
mapred.jobtracker.maxtasks.per.jos 3000
mapred.jobtracker.restart.recover true
mapred.jobtracker.job.history.block.size 3145728
mapred.jobtracker.taskScheduler org.apache.hadoop.mapred.JobQueueTaskScheduler
mapred.jobtracker.taskScheduler.maxRunningTasksPerJob 20
mapred.tasktracker.expiry.interval 600000
mapred.tasktracker.tasks.sleeptime-before-sigkill 6000
mapred.tasktracker.tasks.maxmemory true
mapred.tasktracker.indexcache.mb 20
mapred.tasktracker.taskmemorymanager.monitoring-interval 5000
mapreduce.tasktracker.outofband.heartbeat true
mapred.map.max.attempts 4
mapred.reduce.max.attempts 4
In a MapReduce job, map tasks outputs are aggregated into JVM buffers. The size of the in-memory buffer determines how large the data can be merged and sorted at once. Too small a buffer size can cause a large number of swap operations, incurring big overhead.
Configure the sorting parameters in the $HADOOP_HOME/conf/mapred-site.xml:io.sort.mb 100
io.sort.factor 100
io.sort.record.percent 0.05
io.sort.spill.percent 0.8
mapred.inmem.merge.threshold 1000
mapred.job.shuffle.buffer.percent 0.70
mapred.job.shuffle.merge.percent 0.66
mapred.job.reduce.input.buffer.percent 0.0
mapreduce.reduce.shuffle.maxfetchfailures 10
mapred.cluster.map.memory.mb 200
mapred.cluster.reduce.memory.mb 512
mapred.cluster.max.map.memory.mb 512
mapred.cluster.max.reduce.memory.mb 512
mapred.job.map.memory.mb 0.8
mapred.job.reduce.momory.mb 0.8
When all or part of the map tasks finish, map outputs will be copied from the map task nodes to the reduce task nodes. The parallel copying strategy is used to increase the transfer throughput.
Configure the number of parallel copies in teh $HADOOP_HOME/conf/mapred-site.xml:mapred.reduce.parallel.copies 20
mapred.child.java.opts -Xmx512M
MapReduce tasks are executed by JVM processes/threads, which are forked by the TaskTracker. The creation of a JVM, which include the initialization of execuation environmnet, is costly, especially when the number of tasks is large. In the default configuration, the number of JVMs needed to finish a job should be equal to the number of the tasks. In other words, the default setting uses one JVM to execute one task. When the execution of a task completes, its JVM will be killed by the TaskTracker.
JVM Reuse is an optimization of reusing JVMs for multiple tasks. If it is enabled, multiple tasks can be executed sequentially with one JVM.
Configure JVM Reuse in the $HADOOP_HOME/conf/mapred-site.xml:mapred.job.reuse.jvm.num.tasks 2
Reduce tasks can be startd when a certain percentage of map tasks has been finished. By setting this property with a smaller with a smaller number, the reduce tasks will start earlier, occupying the computing slots. Once the other hand, if the number is set too large, for example, very close to 1, the reduce tasks will have to wait for the majority of the map tasks to finish, prolonging the job execution time.
Configure reducer initialization time in the $HADOOP_HOME/conf/mapred-site.xml:mapred.reduce.slowstart.completed.maps 0.05