Take you to understand two methods of linear programming: sparse matrix storage and preprocessing

Abstract: This article brings you sparse matrix storage and data preprocessing for linear programming.

This article is shared from the HUAWEI cloud community " linear programming-sparse matrix ", the original author: Bale10.

With the development of the AI era, it is inevitable that the scale of linear programming problems is getting bigger and bigger. In the face of large-scale linear programming problems, how to store data, save storage space to avoid waste of resources, and make data query, modification, addition and deletion convenient and quick, is a problem that needs to be solved urgently. This article brings you the sparse matrix storage and data preprocessing of linear programming.

sparse matrix

The size of the LP is usually determined by the size of the constraint matrix A. The elements of the matrix are usually stored in an 8-byte double type. Assuming that the matrix has m rows and n columns, it takes 8mn bytes to store A directly. If A has 10,000 rows and 20,000 columns (not particularly large-scale), then 1.6G of memory is needed to store A. On the one hand, the memory requirements are high, and on the other hand, the operation of matrix A is difficult. Large-scale LP usually contains a large number of zero elements, and the proportion of non-zero elements is very small. This property is called sparsity, that is, A is a sparse matrix.

Sparse matrix storage

The data structure design of sparse matrix should consider the following three factors:

Only non-zero elements are stored. A good sparse matrix data structure should only store the non-zero elements of A and not a large number of zero elements. There are three advantages to this. First, save memory so that large sparse matrices can be stored in memory. Secondly, if only non-zero memory can not be put down, it must rely on external memory, and the speed of accessing data from external memory is generally much slower than accessing data from memory. Therefore, when using external memory, we also Hope that there are only non-zero dollars left. Third, operations involving zero elements may not be executed, which significantly saves calculation time.
Generation of non-zero elements-fill in elements, a sparse matrix, in the process of Gaussian elimination (or LU decomposition), the original zero elements may become non-zero elements. This kind of non-zero elements that change from zero elements during the erasing process is called the filling element; the number of filling elements produced during the entire elimination process is called the filling amount. If a very sparse matrix produces a large number of input elements after the above elimination operation, the sparsity will disappear. Therefore, maintaining the sparsity of the matrix is the premise of using its sparsity. The algorithm needs to consider how to try to make as few input elements as possible in the elimination process. Sparse matrix is a dynamic data structure, especially when non-zero elements need to be inserted or deleted. Therefore, a good sparse matrix data structure must be easy to insert or delete.
The data structure of the sparse matrix is closely related to the elimination algorithm. When considering the data structure of the sparse matrix, the elimination algorithm must be considered at the same time. The data structure must be as easy as possible to implement the algorithm.

Linear table

The simplest sparse matrix storage scheme is a linear table. To be specific, let's take the following sparse matrix A_5 as an example:

Store the non-zero elements in the array CE in columns:

The row number of the corresponding non-zero element in CE is recorded in the array IROW:

In order to give the column number of each non-zero element, a pointer array ICFR is introduced, ICFR(j) represents the position of the first non-zero element in the jth column in CE.

The length of ICFR is N+1, ICFR(N+1) means that the last element is stored at the position of the last element in the $N$th column plus one, which is introduced for the convenience of calculating the number of non-zero elements in the last column. The amount of storage required for this storage scheme is 2NZ+N+1. Its advantages are small storage capacity and simple structure. The single disadvantage is that it is inconvenient to insert and delete. If a non-zero element is to be inserted, the non-zero element below it must be moved down by one position, which is very time-consuming.

In order to allow the insertion of non-zero elements in the program, it is usually necessary to specify a larger array. The above is to store A in columns, of course, it can also be stored in rows. The usual approach is to decompose A into LU first, and then store the lower triangular matrix L in columns and the upper triangular matrix U in rows.

Single list

In order to overcome the inconvenience of inserting and deleting linear lists, a linked list data structure is introduced. That is, add a linked list pointer array LNXT to the above linear table, LNXT(i) represents the position of the next non-zero element of the i-th non-zero element, and the number of non-zero elements in the column is recorded with the array ICNZ. This kind of linked list is called a singly linked list, also called a column linked list. Still take the above sparse matrix A_5 as an example, and store it in columns:

It can be seen from the above table structure that for each non-zero element of the sparse matrix, there is a triplet (row number, element, next element position) in the table, namely

The non-zero elements in each column are connected by a chain pointer $LNXT$. The last non-zero element in each column has a chain pointer of 0, which indicates the end of the chain. The pointer array $ICFR$ indicates the position of the head of each column.

The non-zero elements in the same column do not have to be stored next to each other, because each non-zero element uses $LNXT$ to indicate the position of the next element, so the next element can be placed anywhere in the table.
Insert and delete an element, no need to move other elements, only need to change the pointer.

The advantage of using a linked list is that it is easy to insert or delete; the disadvantage is that access to any element in the linked list needs to be found sequentially from the first non-zero element in a column, and all need indirect access, so it is slower than accessing a linear list.

Double linked list

The double-linked list is based on the singly-linked list by adding the ICOL(i) ith non-zero element column number, RNXT(i) the ith non-zero element row chain pointer, that is, the position of the next non-zero element of the row, and IRFR( i) The position of the first non-zero element in the i-th row in the linked list.

This double-linked list data structure is very convenient for dealing with sparse matrices with asymmetric structure. It has the advantage of a single-linked list, even if it is inserted and deleted.

The operation of inserting or deleting an element in a double-linked list is the same as that of a singly-linked list. Just note that inserting or deleting an element not only needs to modify the column chain and the empty cell chain, but also the row chain.

Designing the data structure of the sparse matrix is one of the keys to the sparse matrix algorithm. A good sparse matrix data structure should not only save storage, but also facilitate search, insertion and deletion, as well as the implementation of sparse matrix algorithms. Data structure has a fundamental impact on algorithm design. Usually, a sparse matrix is established with a linked list first, and a false Gaussian elimination is performed, the elements are inserted and filled, and the total number of non-zero elements is calculated, and then converted to a linear list for quick solution.

Data preprocessing

The scale of linear programming problems is getting bigger and bigger. Problems containing thousands of constraints and variables are very common. They are faced with data storage and processing efficiency problems. Large-scale problems are generally automatically generated by model support tools and often exist. For a large number of redundant constraints and variables, it is necessary to perform data preprocessing before the algorithm is solved.

main purpose

Reduce redundant constraints, reduce non-zero elements of the constraint matrix, increase sparsity, reduce problem scale, and save memory;
Improve the numerical characteristics of the model and increase the numerical stability of the problem;
If the problem is not solved, it is determined that the problem has no feasible solution or unbounded.

Main challenge

Improve the performance of complex preprocessing methods;
Determine the order of action and number of actions of different pretreatment methods;
Determine when the pretreatment will stop;
Design a parallel framework for preprocessing.