Write a Map Function - MATLAB & Simulink - MathWorks (2024)

Role of Map Function in MapReduce

mapreduce requires both an input map function that receives blocks of data and that outputs intermediate results, and an input reduce function that reads the intermediate results and produces a final result. Thus, it is normal to break up a calculation into two related pieces for the map and reduce functions to fulfill separately. For example, to find the maximum value in a data set, the map function can find the maximum value in each block of input data, and then the reduce function can find the single maximum value among all of the intermediate maxima.

This figure shows the Map phase of the mapreduce algorithm.

The Map phase of the mapreduce algorithmhas the following steps:

The Map phase of the mapreduce algorithm is complete when the map function processes each of the blocks of data in the input datastore. The result of this phase of the mapreduce algorithm is a KeyValueStore object that contains all of the key-value pairs added by the map function. After the Map phase, mapreduce prepares for the Reduce phase by grouping all the values in the KeyValueStore object by unique key.

Requirements for Map Function

mapreduce automatically calls the map function for each block of data in the input datastore. The map function must meet certain basic requirements to run properly during these automatic calls. These requirements collectively ensure the proper movement of data through the Map phase of the mapreduce algorithm.

The inputs to the map function are data, info,and intermKVStore:

In addition to these basic requirements for the map function,the key-value pairs added by the map function must also meet theseconditions:

Sample Map Functions

Here are a few illustrative map functions used in mapreduce examples.

function identityMapper(data, info, intermKVStore) % This mapper function simply copies the data and add them to the % intermKVStore as intermediate values. x = data.Value{:,:}; add(intermKVStore,'Identity', x);end

function maxArrivalDelayMapper (data, info, intermKVStore) partMax = max(data.ArrDelay); add(intermKVStore, 'PartialMaxArrivalDelay',partMax);end

function statsByGroupMapper(data, ~, intermKVStore, groupVarName) % Data is a n-by-3 table. Remove missing values first delays = data.ArrDelay; groups = data.(groupVarName); notNaN =~isnan(delays); groups = groups(notNaN); delays = delays(notNaN); % Find the unique group levels in this chunk [intermKeys,~,idx] = unique(groups, 'stable'); % Group delays by idx and apply @grpstatsfun function to each group intermVals = accumarray(idx,delays,size(intermKeys),@grpstatsfun); addmulti(intermKVStore,intermKeys,intermVals); function out = grpstatsfun(x) n = length(x); % count m = sum(x)/n; % mean v = sum((x-m).^2)/n; % variance s = sum((x-m).^3)/n; % skewness without normalization k = sum((x-m).^4)/n; % kurtosis without normalization out = {[n, m, v, s, k]}; endend