slides-cg-students

Column Generation - History • Ford and Fulkerson (1958) : A first suggestion to deal implicitly with the variables of a multicommodity flow problem. • Dantzig and Wolfe (1961) : Develop a strategy to extend a linear program columnwise • Gilmore and Gomory (1961, 1963) : First implementation as part of an efficient heuristic algorithm for solving the cutting stock problem • Desrosiers, Soumis and Desrochers (1984) : Embedding of column generation within a LP based B&B framework for solving a vehicle routing problem FG Simplex 3/38

Column Generation (CG) • A set of columns, a ∈ A ⊂ R m , c a ∈ R , b ∈ R m • Large-scale primal and dual problems: (P) max X a ∈ A c a x a X a ∈ A a x a ≤ b x a ≥ 0 a ∈ A (D) min π b π a ≥ c a a ∈ A π ≥ 0 • A too large : impossible (or impractical) to solve at once • Column Generation : select A 0 ⊆ A , solve Restricted Master Problems (RMP) (P A 0 ) max X a ∈ A 0 c a x a X a ∈ A 0 a x a ≤ b x a ≥ 0 a ∈ A 0 (D A 0 ) min π b π a ≤ c a a ∈ A 0 π ≥ 0 • primal feasible x ? and dual infeasible π ? FG Simplex 5/38

Column Generation Flowchart #1 (Minimization) FG Simplex 8/38

Branch-and-Price Methods Solve Restricted Master Problem (RMP) Solve Pricing Problem (PP) Solve Restricted Master Problem (RMP) Solve Pricing Problem (PP) Solve Restricted Master Problem (RMP) Solve Pricing Problem (PP) FG Simplex 11/38

The cutting stock problem • A paper company with a supply of large rolls of paper, of width W • Customer demand is for smaller widths of paper b i rolls of width w i ≤ W , i = 1 , 2 , ...... m need to be produced. • Smaller rolls are obtained by slicing large rolls • Example: a large roll of width 70 can be cut into 3 rolls of width w 1 = 17 and 1 roll of width w 2 = 15 , with a waste of 4. minimize the waste ! Cutting stock problem FG Simplex 14/38

Drawbacks of the Kantorovich Model However, the IP formulation ( P 1 ) is inefficient both from computational and theoretical point views. The main reason is that the linear program (LP) relaxation of ( P 1 ) is poor. Actually, the LP bound of ( P 1 ) is z ? LP = X k ∈K y k = X k ∈K m X i = 1 w i x k i W = m X i = 1 w i X k ∈K x k i W = m X i = 1 w i b i W (1) Question: Is there an alternative? FG Simplex 17/38

Gilmore - Gomory Model & Master Problem Cutting pattern p : described by the vector ( a p 1 , a p 2 , .... , a p m ) , where a p i represents the number of rolls of width w i obtained in cutting pattern p . Variables: λ p = # rolls to be cut according to cutting pattern p . [ex: write LP] Each column represents a cutting pattern. How many columns (cutting patterns) are there? It could be as many as m ! k !( m - k )! where k is the average number of items in each cutting patterns. Exponentially large!. FG Simplex 20/38

Cutting Pattern & Pricing Problem • p th pattern, can be represented by a column vector a p whose i th entry indicates how many rolls of width w i are produced by that pattern. • Pattern discussed earlier can be represented by the vector: ( 3 , 1 , 0 , ..., 0 ) • For a vector ( a p 1 ; a p 2 , ..., a p m ) to be a representation of a feasible pattern, its components must be nonnegative integers, and we must also have: m X i = 1 w i a p i ≤ W . FG Simplex 21/38

Initial Solution First step, set up column generation iteration starting point. Actually, the initial basis matrix can be constructed in a trivial way. For example, let W = 10 and the customer demand: small roll widths w 1 = 3 and w 2 = 2 . The following two choices are both valid: • Choice 1: a p 1 = 1 0 ; a p 2 = 0 1 • Choice 2: a p 1 = 3 0 ; a p 2 = 0 5 FG Simplex 22/38

Definition of the Restricted Master Problem P 0 ⊆ P a small subset of cutting patterns min X p ∈ P 0 λ p s.t. X p ∈ P 0 a p i λ p ≥ 0 i = 1 , 2 , ... m λ p ≥ 0 p ∈ P 0 Note: • m types of small roll customers requested • | P 0 | number of patterns generated so far • column generation ; tool for solving the linear programming relaxation. FG Simplex 23/38

How to Iterate? Suppose that we have a basis matrix B for the restricted master problem and an associated basic feasible solution, and that we wish to carry out the next iteration for the column generation. Because the coefficient of every variable λ p is 1, every component of the vector c B is equal to 1. Next, instead of computing the reduced cost c B = 1 - c B B - 1 a p associated with every column a p , we consider the problem of minimizing ( c B = 1 - c B B - 1 a p ) over all p . This is the same as maximizing c B B - 1 a p over all p . • If the maximum is ≤ 1 , all reduced costs are nonnegative and we have an optimal solution. • If the maximum is > 1 , column a p corresponding to a maximizing p has negative reduced cost and enters the basis. FG Simplex 24/38

Pricing Problem Recall that u = c B B - 1 max m X i = 1 u i a i s.t. [ex: write constraint] a i ≥ 0 i = 1 , 2 , ..... , m a i ∈ Z + i = 1 , 2 , ..... , m . FG Simplex 25/38

An Example Suppose for the paper company, a big roll of paper is W = 218 cm . The customers of the company want: • 44 small rolls of length 81cm • 3 small rolls of length 70cm • 48 rolls of length 68 cm That is, w =   81 70 68   , b =   44 3 48   FG Simplex 26/38

Initialization Step 1, to generate the initial basis matrix: a 1 =   1 0 0   , a 2 =   0 1 0   , a 3 =   0 0 1   First restricted master problem: min λ 1 + λ 2 + λ 3 s.t.   1 0 0 0 1 0 0 0 1     λ 1 λ 2 λ 3   ≥   44 3 48   λ p ≥ 0 p = 1 , 2 , ..... , 5 . FG Simplex 27/38

Solving the Restricted Master and Pricing Problems Solution of the restricted master problem ; u = c B B - 1 = ( 1 , 1 , 1 ) . Therefore, the first pricing problem is written max a 1 + a 2 + a 3 subject to: 81 a 1 + 70 a 2 + 68 a 3 ≤ 218 a i ≥ 0 i = 1 , 2 , 3 a i ∈ Z + i = 1 , 2 , 3 The optimal solution is   a 1 a 2 a 3   =   0 0 3   with optimal value 3 > 1 . FG Simplex 28/38

The Second Restricted Master Problem min λ 1 + λ 2 + λ 3 + λ 4 s.t.   1 0 0 0 0 1 0 0 0 0 1 3       λ 1 λ 2 λ 3 λ 4     ≥   44 3 48   λ p ≥ 0 p = 1 , 2 , ..... , 5 . After solving, we get u = ( 1 , 1 , 0 . 33 ) . FG Simplex 29/38

Solving the Second Pricing Problem By solving the restricted master problem, we can get u = ( 1 , 1 , 0 . 33 ) . Therefore, the second pricing problem is max a 1 + a 2 + 0 . 33 a 3 s.t. 81 a 1 + 70 a 2 + 68 a 3 ≤ 218 a i ≥ 0 i = 1 , 2 , 3 a i ∈ Z + i = 1 , 2 , 3 The optimal solution is   a 1 a 2 a 3   =   0 3 0   with optimal value 3 > 1 . FG Simplex 30/38

The Third Restricted Master Problem min λ 1 + λ 2 + λ 3 + λ 4 + λ 5 s.t.   1 0 0 0 0 0 1 0 0 3 0 0 1 3 0         λ 1 λ 2 λ 3 λ 4 λ 5       ≥   44 3 48   λ p ≥ 0 p = 1 , 2 , ..... , 5 . After solving the above program, we get u = ( 1 , 0 . 33 , 0 . 33 ) . FG Simplex 31/38

Solving the Third Pricing Problem By solving the restricted master problem, we can get u = ( 1 , 0 . 33 , 0 . 33 ) . Therefore, the second pricing problem is max a 1 + 0 . 33 a 2 + 0 . 33 a 3 s.t. 81 a 1 + 70 a 2 + 68 a 3 ≤ 218 a i ≥ 0 i = 1 , 2 , 3 a i ∈ Z + i = 1 , 2 , 3 The optimal solution is   a 1 a 2 a 3   =   2 0 0   with optimal value 2 > 1 . FG Simplex 32/38

Restricted Master Problem # 4 min λ 1 + λ 2 + λ 3 + λ 4 + λ 5 + λ 6 subject to:   1 0 0 0 0 2 0 1 0 0 3 0 0 0 1 3 0 0           λ 1 λ 2 λ 3 λ 4 λ 5 λ 6         ≥   44 3 48   λ p ≥ 0 p = 1 , 2 , ..... , 6 After solving the above program, we get u = ( 0 . 5 , 0 . 33 , 0 . 33 ) . FG Simplex 33/38

Pricing Problem # 4 By solving the restricted master problem, we can get u = ( 0 . 5 , 0 . 33 , 0 . 33 ) . Therefore, the second pricing problem is max 0 . 5 a 1 + 0 . 33 a 2 + 0 . 33 a 3 s.t. 81 a 1 + 70 a 2 + 68 a 3 ≤ 218 a i ≥ 0 i = 1 , 2 , 3 a i ∈ Z + i = 1 , 2 , 3 The optimal solution is   a 1 a 2 a 3   =   1 0 2   with optimal value 1 . 16 > 1 . FG Simplex 34/38

Restricted Master Problem # 5 min λ 1 + λ 2 + λ 3 + λ 4 + λ 5 + λ 6 + λ 7 s.t.   1 0 0 0 0 2 1 0 1 0 0 3 0 0 0 0 1 3 0 0 2             λ 1 λ 2 λ 3 λ 4 λ 5 λ 6 λ 7           ≥   44 3 48   λ p ≥ 0 p = 1 , 2 , ..... , 7 . After solving the above program, we get u = ( 5 , 0 . 33 , 0 . 25 ) . FG Simplex 35/38

Pricing Problem # 5 Restricted master problem # 5 ; u = ( 0 . 5 , 0 . 33 , 0 . 25 ) . Therefore, pricing problem # 5 is max 0 . 5 a 1 + 0 . 33 a 2 + 0 . 25 a 3 s.t. 81 a 1 + 70 a 2 + 68 a 3 ≤ 218 a i ≥ 0 i = 1 , 2 , 3 a i ∈ Z + i = 1 , 2 , 3 The optimal solution is   a 1 a 2 a 3   =   2 0 0   with optimal value 1 ; optimality condition is satisfied. FG Simplex 36/38

Optimal Solution The optimal solution of the cutting stock problem is: 1   0 3 0   + 10   2 0 0   + 24   1 0 2   with optimal value 35. Question: if the initial matrix basis is : a 1 =   0 3 0   ; a 2 =   2 0 0   ; a 3 =   0 0 3   How many iterations you need to take before reaching an LP optimal solution? FG Simplex 37/38

Column Generation - Take aways When to use Column Generation? • Compact formulation of a ILP cannot solved efficiently , e.g. due to • weak LP relaxation . Can be tightened by column generation formulation. • Symmetric structure , → poor performance of B&B . Column generation formulation allows us to get ride of symmetries. • Existence of a "natural decomposition". FG Simplex 38/38

Column Generation - Why it works • Master Problem , → Very often, a general ILP problem • Pricing Problem , → An ILP problem with a special structure • Knapsack problem (e.g., the pricing problem of the cutting stock problem). • Shortest path problem. • Maximum flow problem. • Any combinatorial problem. The success of applying column generation often relies on efficient algorithms to solve the pricing problem. • Moreover, column generation , → decomposition: Natural interpretation often allows us to take care easily of additional constraints. FG Simplex 39/38