MapReduce and Apache spark: technology analysis, advantages and disadvantages t q urazmatov1 and X sh Kuzibayev

When to use MapReduce:
- In linear processing of large data sets
- When no intermediate solution is required.
When to use Apache spark:
- Fast and interactive data processing;
- When joining the data set;
- When processing schedules;
- When performing repetitive work;
- Real-time processing;
- In machine learning
3.Results:
As a result of our research, we have the following facts. We compared MapReducate and Apache spark based on 20 types of parameters. We placed the results of the studies in the table below.
Table 1. MapReduce and Apache spark comparison table.

Parametrs	MapReduce	Apache spark
Data processing	For mass processing only	Mass processing, as well as real-time data processing
Processing speed	Slower than Apache spark because if there is an input / output delay on the disk	100 times faster in memory and 10 times faster on disk
Category	Data processing mechanism	Data Analytics Engine
Costs	Cheaper compared to Apache spark	More expensive due to the large amount of RAM
Scalability	Both scales are limited to 1000 nodes per cluster	Both scales are limited to 1000 nodes per cluster
Machine learning	MapReduce is more compatible with Apache Mahout when integrated with machine learning.	Apache spark has built-in machine learning APIs
Compatibility	Compatible with all data sources and file formats	Apache spark can integrate with all data sources and file formats supported by the Hadoop cluster.
Security	The MapReduce framework is safer compared to the Apache spark	The security features of the Apache spark are evolving and becoming more sophisticated.
Scheduler	Depending on the external planner	Apache spark has its own planner
Resistance to errors	Use replication for error tolerance	Apache spark uses RDD and other storage models for fault resistance
Ease of use	Thanks to the MapReduce JAVA API, it is a bit more complicated compared to the Apache spark	Apache spark is easier to use because of the APIs
Duplication elimination	MapReduce does not support this feature	Apache spark processes each entry exactly once, thus eliminating duplication.
Language supporting	The main language is Java, but languages such as C, C ++, Ruby, Python, Perl, Groovy are also supported.	Apache spark supports Java, Scala, Python and R.
Delay	Very high delay	Faster compared to MapReduce Framework
Complexity	Codes are hard to write and debug	Easy to write and debug
Apache	An open source framework for data processing	An open source framework for high-speed data processing
Coding	More code lines	Fewer code rows
Interactive mode	Not interactive	Interactive
Infrastructure	Brand equipment	Medium and high level hardware
SQL	Hive supports query language	Spark supports via SQL