JavaScript Data Processing - Lists

There are two types of data sets commonly used in programs, lists and maps. Support for these two collection structures is provided in the language foundation of JavaScript - an array (Array) for a list, and a plain object for a map (property key-value pair mapping).

Today we only talk about arrays.

From the instance methods provided in the Array class, it can be seen that the array covers general list operations, including additions, deletions, modifications and investigations, and provides shift()/unshift() and push()/pop() like this The method makes the array have the basic functions of queue and stack.

In addition to daily CRUD, the most important thing is to traverse the list completely or partially to get the expected result. These traversal operations include

1. Traverse one by one: for , forEach() , map() etc.;
2. Filter/Filter: filter() , find() , findIndex() , indexOf() , etc;
3. Traversal calculation (reduction): reduce() , some() , every() , includes() , etc.

Most of the instance methods provided by the Array object for traversal receive a handler function to process each element during the traversal process. And the processing function usually has three parameters: (el, index, array) , which respectively represent the currently processed element, the index of the current element, and the currently processed array (ie, the original array). Of course, this is the majority, there are some exceptions, such as includes() is not the case, and the processing function of reduce will have an additional parameter representing the intermediate result. Needless to say, just check out MDN .

A simple traversal

We all know for syntax in JavaScript in addition to the basic for ( ; ; ) , also includes two kinds of for each traversal. One is for ... in to iterate over keys/indexes; the other is for ... of to iterate over values/elements. Neither of the two for each constructs can get the key/index and value/element at the same time, but the forEach() method can get it, which is the convenience of forEach() . However, to terminate the loop in the for each structure, you can use break , and in forEach() you can only terminate the loop through throw . Using throw to terminate the loop needs to be processed outside try ... catch , which is not flexible enough. Example:

 try {
    list.forEach(n => {
        console.log(n);
        if (n >= 3) { throw undefined; }
    });
} catch {
    console.log("The loop is broken");
}

If there is no try ... catch , the throw in it will directly interrupt the program.

Of course, there is an easier way. Note that both some() and every() both traverse the array until a qualifying/unqualifying element is encountered. Simply put, they are based on the return value of the handler function to determine whether to interrupt the traversal. For some() , it is to find a qualified element. If the processing function returns true , it will interrupt the traversal; and every() is just the opposite, it is to It is judged that each element meets the conditions, so as long as the return false is encountered, the traversal will be interrupted.

According to our understanding of general for loops and while loops, the conditions are true to loop, so it seems that every() is more customary. The above example is rewritten with every() :

 list.every(n => {
    console.log(n);
    return n < 3;
});

Use some() and every() to pay special attention: it does not need to accurately return the value of boolean type, only need to judge the true value (truthy) and the false value (falsy). The JavaScript function is equivalent to return undefined without an explicit return value, that is, returning a false value, and the effect is the same as return false .

For false values in JavaScript, see MDN - Falsy . All true values except false values.

Second, traverse the map

Sometimes we need to iterate over an array, producing another value and object based on the information provided by each of its elements, while the result is still placed in an array. The most common scenario of this kind of operation in front-end development is to process the list of model data obtained from the back-end into a list of view data required for front-end rendering. The normal operation is this:

 // 源数据
const source = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

// 创建目标数组容器
const target = [];
// 循环处理每一个源数据元素，并将结果添加到目标数组中
for (const n of source) {
    target.push({ id: n, label: `label${n}` });
}

// 消费目标数组
console.log(target);

map() is used to encapsulate such traversal, which can be used to process one-to-one element data mapping. The above example uses map() only one sentence is needed to replace the loop:

 const source = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const target = source.map(n => ({ id: n, label: `label${n}` }));
console.log(target);

In addition to reducing statements, using map() also turns several original statements into an expression, which can be flexibly used for upper and lower logical connections.

3. Dealing with multi-layer structures - expansion (flat and flatMap)

Expand, ie flat() operation can reduce a multidimensional array by one or more dimensions. for example

 const source = [1, 2, [3, 4], [5, [6, 7], 8, 9], 10];
console.log(source.flat());
// [ 1, 2, 3, 4, 5, [ 6, 7 ], 8, 9, 10 ]

This example is an array with three (albeit not neat) dimensions, using flat() reduces one dimension, resulting in two dimensions. flat() You can specify the number of dimensions to expand through parameters, here you only need to specify a value greater than or equal to 2 , it can flatten all elements into a one-dimensional array:

 const source = [1, 2, [3, 4], [5, [6, 7], 8, 9], 10];
console.log(source.flat(10));
// [ 1, 2, 3, 4,  5,  6, 7, 8, 9, 10 ]

With this thing, we will be more convenient when dealing with some sub-items. For example a common question:

There is a second-level menu data, I want to get a list of all menu items, what should I do? Data are as follows

 const data = [
 {
     label: "文件",
     items: [
         { label: "打开", id: 11 },
         { label: "保存", id: 12 },
         { label: "关闭", id: 13 }
     ]
 },
 {
     label: "帮助",
     items: [
         { label: "查看帮助", id: 91 },
         { label: "关于", id: 92 }
     ]
 }
];

what to do? No suspense should be handled using a double loop. But using map() and flat() can simplify the code:

 const allItems = data.map(({ items }) => items).flat();
//                   ^^^^                      ^^^^^

The first step map() map the elements of type { label, items } [...items] into an array of this form, and the mapping result is a two-dimensional array (illustration):

 [
    [...文件菜单项],
    [...帮助菜单项]
]

Then use flat() to flatten, and you get [...文件菜单项, ...帮助菜单项] , which is the expected result.

Usually we directly get a two-dimensional array to deal with very few cases. Generally, we need to first map() and then flat() , so JavaScript provides flatMap() for these two common combinational logics flatMap() method. To understand the function of flatMap() , it can be understood as first map(...) and then flat() . The above example can be changed to

 const allItems = data.flatMap(({ items }) => items);
//                   ^^^^^^^^

A two-layer structure of data is solved here. What if it is a multi-layer structure? The multi-layer structure is not an ordinary tree structure, and all sub-items can be processed by recursion flatMap() . The code is not provided first, please use your brain.

Fourth, filter

If we have a set of data and need to filter out the ones that meet certain conditions, we will use filtering, filter() . filter() Receive a processing function for judgment, and use the processing function to judge each element. If the result of the function's judgment on an element is true, the element will be retained; otherwise, it will not be included in the result. The result of filter() is a subset of the original array.

The usage of filter() is easy to understand. For example, the following example filters out numbers that are divisible by 3:

 const a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const r = a.filter(n => n % 3 === 0);
// [ 3, 6, 9 ]

Two points need to be emphasized:

First, if none of the elements meet the criteria, you get an empty array. neither null nor undefined , but [] ;

Second, if all elements are eligible , you get a new array with all the elements. Comparing it with the original array === or == will yield false .

Although filtering is simple, it should be used flexibly. For example, if you need to count the number of qualified data in a certain set of data, you usually think of traversal count. But we can also filter according to the specified conditions first, and then take the length of the result array.

5. Find

The difference between search and filter is: search is to find an element that meets the conditions, while filter is to find all. In terms of implementation effect, arr.filter(fn)[0] can achieve the search effect. Only filter() will traverse the entire array.

The professional find() will terminate the traversal immediately after finding the first eligible element, saving time and computing resources. From the results, find() can be regarded as a convenient implementation of filter()[0] (of course, the performance is also better), and its parameters (processing function) are the same as filter() .

The result of find() is the element found, or undefined if nothing was found. Therefore, when using find() , you must pay attention that the result may be undefined , and you should judge the validity before use. Of course, it's also easy to get involved in expressions if you combine the optional chaining operator ( ?. ) with the null coalescing operator ( ?? ) .

But sometimes, when we look for an element, we don't want to use it, we want to replace or delete it. At this time, it is difficult to get the element itself, and we need the index number more. It is easy to find the findIndex() method by checking MDN. Its usage is the same as find() except that the element index is returned instead of the element itself. If no matching element is found, findIndex() will return -1 .

Speaking findIndex() it is easy to think of indexOf() . The parameter of indexOf() is a value (or object), it will find the position of this value in the array and return it. Works fine for values of primitive types. But for object elements, be careful, take a look at the following example

 const m = { v: 1 };
const a = [m, { v: 2 }, { v: 3 }];

console.log(a.indexOf(m));          // 0
console.log(a.indexOf({ v: 1 }));   // -1

Also represented as { v: 1 } , but they are not really the same object!

By the way, some people like to use arr.indexOf(v) >= 0 to determine whether an element contains an element in the array. In fact, they might as well use the professional includes() method. includes() Returns a boolean value directly, and it allows to specify the position to start the search through the 2nd parameter.See MDN for details .

Then, if you want to judge whether there is a qualified element in the data according to a certain judgment method (function), should you use arr.find(fn) !== undefined to judge? In general, yes, but in special circumstances—

 // 查找判断是否包含假值
const a = [undefined, 1, 2, 3];
const hasFalsy = a.find(it => !it) !== undefined;  // false

Unfortunately, this result is wrong, it is visible to the naked eye that it does contain false values!

The correct way to find existence by condition is to use the some() method (mentioned earlier, forget?):

 const hasFalsy = a.some(it => !it);

6. Reduction

Reduction is a literal translation of reduce, and reduce() is also a method of arrays.

The reason for reduction is because sometimes the processing we need to do is not as simple as the one mentioned above, such as a common application - accumulation. Think about it, the previous processing method can only be accumulated by for or forEach() . Both of these methods require additional temporary variables and are not very friendly to functional programming. If you use reduce() , it is probably like this:

 const a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const sum = a.reduce((sum, n) => sum + n);

A little more complicated, if you want to separate the odd and even numbers in the array, put them in two arrays:

 const a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const [even, odd] = a.reduce(
    (acc, n) => {
        acc[n % 2].push(n);
        return acc;
    },
    [[], []]
);

Different from the above code, the second parameter [[], []] --- is given here when reduce() is used. This is an initial value that will be passed into the function as the first parameter when the handler is called for the first time, which is the acc parameter in the above example.

Then the attentive reader will ask why the first example does not need an initial value, what is the initial sum parameter in that case?

Here I have to say two features of reduce() :

1. reduce() The result of each processing, that is, the return value of the processing function, will be used as the first parameter of the next processing;
2. When calling reduce() , if the second parameter, the initial value, is given, it will be used as the first parameter in the first processing; but if no initial value is given, in the first processing, The first two elements in the array are used as the first two parameters passed into the handler function.

Seeing item 2, is there another question: what if there is only one element in the array? -- then the handler function is ignored, and this element is the result of reduce() .

So... what if the array is empty? Can this question be reduce() ask to cry--Is it okay to report an error? !

Learn reduce() and you will find that all the traversal processes mentioned above can be implemented with reduce() , after all, it is very flexible to use - but why? Besides, reduce() can only be interrupted by throw of course don't worry throw interrupt can't get the result, take the result as throw The object is thrown out, can't you get it from outside? hiahiahia~~

7. Intercept

Take a portion of the data, no doubt, using the slice() method. The two parameters of this method indicate the start and end points of the index to be intercepted, and the element corresponding to the end point index is not included. In mathematical language, this is a left-closed and right-open interval. It should be noted that the starting point must be less than the end point to be able to get elements, otherwise the result must be an empty array. This means that if you use arr.slice(-Infinity, Infinity) you can get all the elements - but who would do this, directly arr.slice(0) Isn't it fragrant (do not give the second parameter to indicate that it has been intercepted until the end) an element)?

In addition, slice() has two interesting features:

• Whether it is the starting point or the ending point, if a value beyond the array index range is given, it will not cause an error, it will take the intersection of the array index range and the specified range;
• If the given index is negative, such as -n , it will calculate the index according to arr.length - n . This makes it easier to get elements based on their ending positions.

The example will not be written, slice() the document is very clear, it is not difficult to understand.

Some people want to ask, the stream processing of many collections will have take() and skip() methods, which are used to intercept a certain number of elements at a specified position. Does JavaScript have it? slice() Isn't it? It is basically equivalent .skip().take() . It is basically equivalent because the parameter of take() represents a length, and the second parameter of slice() represents a position, so (the following .skip() .take() are pseudocode, just for illustration):

• arr.skip(m).take(n) equivalent to arr.slice(m, m + n)
• arr.skip(m) equivalent to arr.slice(m)
• arr.take(n) equivalent to arr.slice(0, n)

But while using slice() , don't forget that JavaScript's destructuring assignment syntax can also be used to simply intercept an array. for example

 const a = [1, 2, 3, 4, 5, 6];
const [,, ...rest] = a;
// rest = [3, 4, 5, 6]

It is the same as the result of a.slice(2) , but in comparison, slice() has more explicit semantics, which is better than the number of commas. But the destructuring syntax is handy in some cases, like when splitting commands and arguments in the CLI:

 const args = "git config --list --global".split(/\s+/);
const [exec, cmd, ...params] = args;
// exec: "git"
// cmd: "config"
// params: ["--list", "--global"]

Eight, create an array

Although this article mainly talks about traversal-based array operations, since it has already been mentioned slice() this non-traversal type of operation, may wish to mention creating an array by the way.

• Use [] to create an empty array, or an array with a known few elements, most commonly used;
• Use the spread operator ( ... ) on the iterable object to generate an array, such as [..."abc"] to get ["a", "b", "c"] ;
• Array(n) Create an array of the specified length, but note that although this array has a length, it has no elements and cannot be traversed. Want it to have elements -- look back --
• Array.from(Array(n)) , get an array of length n, all elements are undefined ;
• [...Array(n)] same result as above; [...Array(n).values()] the same;
• Array(n).fill(1024) to create an array of length n, all elements are 1024 . Of course, other element values can also be specified;
• The second parameter of Array.from is a mapper, so Array.from(Array(n), (_, i) => i) can create an array whose elements are from 0 to n - 1 ;
• Use [...Array(n).keys()] to create the same array as the previous one;
• ...

Now there is a problem, I want to create a 7x4 two-dimensional array, the default element is filled 0 , what should I do? That's not easy, so

 const matrix = Array(4).fill(Array(7).fill(0));
// [
//   [ 0, 0, 0, 0, 0, 0, 0 ],
//   [ 0, 0, 0, 0, 0, 0, 0 ],
//   [ 0, 0, 0, 0, 0, 0, 0 ],
//   [ 0, 0, 0, 0, 0, 0, 0 ]
// ]

It seems that there is nothing wrong, let's perform an operation and see how it works?

 matrix[0][4] = 4;
// [
//   [ 0, 0, 0, 0, 4, 0, 0 ],
//   [ 0, 0, 0, 0, 4, 0, 0 ],
//   [ 0, 0, 0, 0, 4, 0, 0 ],
//   [ 0, 0, 0, 0, 4, 0, 0 ]
// ]

All elements of the second level array with index 4 become 4 … Why?

Let's split the above initialization statement, and we may understand -

 const row = Array(7).fill(0);
const matrix = Array(4).fill(row);

You see, these 4 lines refer to an array, so no matter which one is changed, the output data of the 4 lines will be exactly the same (can the same array be different).

This is the most common pitfall when initializing multidimensional arrays. So Array(n).fill(v) Although it is easy to use, you must be careful. Here, if you use the mapped Array.from() it will be fine:

 const matrix = Array.from(
    Array(4),
    () => Array.from(Array(7), () => 0)
);

summary

Due to the dynamic nature of JavaScript, there is no need to define a lot of data types to represent different lists, just an array. Although there are still some limitations, it has been able to adapt to most application scenarios. This article mainly introduces the basic operations of arrays. For more details, please refer to the MDN - Array documentation. Next time I will talk about the basic operations of mapping tables (objects) and the joint application between arrays and objects. Regarding data processing in JavaScript, readers can also learn about Lodash , which provides a lot of tools.