A relaxation of an integer polytope.

CS 599: Rounding Techniques in Approximation Algorithms (Fall 2024)

Course Information

Instructor: Nathan Klein

Syllabus: Link

Lectures: Tuesday and Thursday, 3:30 - 4:45pm in CDS 701

Office Hours: Tuesday 1:30 - 3:30, Thursday 2:30 - 3:30, and by appointment

Prerequisites: Strong undergraduate-level knowledge of algorithms, linear algebra, and probability. Some familiarity with linear programming and at least one of CS 530, 531, or 537 is recommended but not required. Motivated, mathematically mature undergraduate students who have excelled in CS 237 and CS 330 are also welcome.

Grading: Four homework sets (15% each, lowest dropped), scribe one lecture (10%), participation and (very basic) quizzes (15%), and a final project (30%)

Cow: Point set from Mathematica

Overview

We will survey a simple but powerful framework for designing approximation algorithms known as "Relax and Round." Given a (possibly NP-Hard) discrete optimization problem, this framework first relaxes it into a polynomial time solvable one over a continuous domain. It then solves this easier problem, whose solution can have fractional coordinates: for example, it could assign a variable to be half true and half false in a SAT formula. Below, in pink is the convex hull of the feasible solutions of a discrete optimization problem. In purple is a relaxation of it.

A relaxation of an integer polytope.

In the final step, we round this fractional solution (above in blue) to an integer one (above in red). Our goal will be to find a rounding procedure that, like a good translator, finds an integer solution that approximately preserves the key properties of the fractional one such as cost. There are many beautiful methods known for performing this rounding step, which will be our main focus. We will discuss many of them, such as the use of integral polytopes, iterative rounding, iterative relaxation, and independent and dependent randomized rounding. The course will emphasize graph problems as well as results from the past decade with exciting open directions.

Homework

Related Textbooks and Materials

Related Courses

Course Schedule

Weeks 1 and 2: Intro and Independent Randomized Rounding

Lecture 1, Tuesday 9/3: Admin, Introduction to Approximation Algorithms and Rounding [Notes] We will use Vertex Cover to review the basics of approximation algorithms, linear programming, integrality gaps, and the relax and round framework. Related lectures: [Chakrabarty].
Lecture 2, Thursday 9/5: Independent Randomized Rounding for Congestion Minimization [Notes] After a recap of relax and round using a more polyhedral lens, we will review basic probability and Chernoff bounds. Then we will discuss using independent randomized rounding for the congestion minimization problem. Related lectures: [Lee] [Chakrabarty], also see Chapter 5.11 of the Williamson-Shmoys textbook and this paper by Raghavan-Thompson.
Lecture 3, Tuesday 9/10: Independent Randomized Rounding for \(k\)-edge-connectivity [Notes] We will review basic graph concepts like edge connectivity and minimum cuts, as well as Karger's bound on the number of \(\alpha\)-near minimum cuts (see this paper for the version we did in class as well as a more precise bound). We will then discuss Karger's algorithm for finding \(k\)-edge-connected graphs of approximate minimum cost.
Lecture 4, Thursday 9/12: Overflow, Quiz 1, Open Questions, and Work Session [Flashcards] We will finish the previous lecture and give time for questions on what we have covered so far. Then we will have a short quiz on the flashcards posted for this lecture, after which some open questions will be discussed. Time permitting, there will be some group work on the homework. Some topics mentioned in class:

Weeks 3, 4, and 5: Dependent Randomized Rounding

Lecture 5, Tuesday 9/17: Randomized Pipage Rounding [Notes] We will first prove the integrality of a natural formulation of the spanning tree polytope and introduce Randomized Pipage Rounding. Related lectures on pipage rounding: [Chakrabarty] [Oveis Gharan].
Lecture 6, Thursday 9/19: Randomized Pipage Rounding and an Improved Approximation for \(k\)-edge-connectivity [Notes] We will continue analyzing the Randomized Pipage Rounding technique and use it to approximate the \(k\)-edge-connected multi-subgraph problem (see this paper).
Lecture 7, Tuesday 9/24: Thin Trees and an \(O(\log n/\log \log n)\) Approximation for ATSP [Notes] We will use randomized pipage rounding to approximate the Asymmetric Traveling Salesperson Problem (ATSP) via thin trees. Here is the original paper which uses max entropy sampling. Related lectures: [Oveis Gharan].
Lecture 8, Thursday 9/26: Splitting Off and Approximating Prize-Collecting Problems [Notes] We will first review the splitting-off theorems of Lovász and Mader and then use it to approximate the Prize-Collecting TSP and Prize-Collecting Steiner Tree problems. See this paper (see Section 5 for a proof of the decomposition lemma, a key tool we will use in this lecture) as well as this one, both on Prize-Collecting TSP. Here is the most relevant paper on splitting off (Frank 1992). Here is a more recent paper on splitting off, and here are some lecture notes related to splitting off.
Lecture 9, Tuesday 10/1: Overflow, Quiz 2, Open Questions, and Work Session [Flashcards] [HTML] We will finish the previous lecture and give time for questions on what we have covered so far. Then we will have a short quiz on the flashcards posted for this lecture, after which some open questions will be discussed. Papers related to the open problems discussed:
Class Canceled on Thursday 10/3 Take a look at this book to prepare for next week.

Weeks 6, 7, and 8: Sparsity, Iterative Rounding, and Relaxation

Lecture 10, Tuesday 10/8: Convex Geometry, Sparsity, and Unrelated Parallel Machines [Notes] We will go over some basics of convex geometry, in particular showing the equivalence of a vertex, an extreme point, and the unique solution to a subset of tight constraints. We will discuss how this leads to "sparsity," the phenomenon that extreme points can often be shown to have few non-zero entries, and apply it to approximating the problem of minimizing the makespan on unrelated parallel machines (here is the original paper by [Lenstra, Shmoys, and Tardos]). Related lectures: [Chakrabarty] [Chekuri/Ene]
Lecture 11, Thursday 10/10: Iterative Rounding – Jain's Approximation for Survivable Network Design [Notes] We will introduce the idea of iterative rounding and prove a 2-approximation for the survivable network design problem. See Jain's original paper here and a paper by [Chekuri and Rukkanchanunt] that gives a simplified proof. Also see this paper by [Nagarajan, Ravi, and Singh]. See Chapter 10 of the iterative rounding book. Related lecture notes: [Chekuri] [Svensson and Newman]
No class on Tuesday 10/15: BU is on a Monday schedule Relax! (And round?)
Lecture 12, Thursday 10/17: Iterative Relaxation for Discrepancy [Notes] We will recap iterative rounding and finish the proof of Jain's 2-approximation. We will then introduce the idea of iterative relaxation and prove the Beck-Fiala discrepancy theorem. Since this is a very short topic, we will likely also start discussing the next lecture. See Chapter 13 of the iterative rounding book. Related lectures: [Rothvoss] [Nikolov]. Original paper: [Beck-Fiala]
Lecture 13, Tuesday 10/22: Iterative Relaxation for Bounded Degree Matroids [Notes] We will use iterative relaxation to obtain minimum cost bounded degree spanning trees and more generally bounded degree matroid bases. The original paper on bounded degree spanning trees and its generalization to matriods. Here is the earlier paper for the unweighted case. Finally, see Chapter 5 of the iterative rounding book. Related lectures: [Vondrak]
Lecture 14, Thursday 10/24: Overflow, Quiz 3, Open Questions, and Work Session [Flashcards] [HTML] We will finish the previous lecture and give time for questions on what we have covered on iterative rounding and relaxation. After a short quiz, some open questions will be discussed. Time permitting, there will be some group work on the homework. Problems discussed:
  • Unrelated parallel machines: open to beat a factor of 2. Some progress on the restricted assignment case here and a follow-up here
  • It is open to beat a factor of 2 for survivable network design, including the special case of 2-ECSS as previously discussed. One special case is the Tree Augmentation Problem, on which there has been recent exciting progress: see here.
  • Beck-Fiala: open to beat \(2t-\log^*(t)\) (see this paper). The conjectured bound is \(O(\sqrt{t})\) but it would be a major result to get \(1.999t\).
  • Open problem on bounded degree matroids.

Weeks 9 and 10: Iterative Randomized Rounding

Lecture 15, Tuesday 10/29: Iterative Randomized Rounding and Isotropy [Notes] We will start to discuss a recent work by Bansal on iterative randomized rounding which combines the techniques we have learned about over the past two units. After motivating this new framework, we will discuss isotropic distributions. See this talk by Bansal.
Lecture 16, Thursday 10/31: Sub-Isotropic Rounding [Notes] We will continue discussing the work by Bansal and in particular the sub-isotropic rounding algorithm.
Lecture 17, Tuesday 11/5: Martingales [Notes] We will introduce martingales and show a crucial lemma for our analysis of this algorithm.
Lecture 18, Thursday 11/7: Sub-Isotropic Updates and Open Questions We will use the last lecture to show that sub-isotropic updates imply concentration. Afterwards, we will discuss some open questions related to sub-isotropic rounding.

Week 11: Guest Lecture (at workshop)

Tuesday 11/12: No Class - Recommended Talk to Watch This talk will recap the last week and give more applications of iterative randomized rounding.
Lecture 19, Thursday 11/14: Guest Lecture by Kira Goldner on Rounding LPs Online for Contention Resolution [Kira's Notes] See this paper about rounding a fractional point in a matroid polytope online to construct an independent set. This style of rounding is known as an online contention resolution scheme.

Weeks 12 and 13: Beyond Linear Programs

Lecture 20, Tuesday 11/19: Semidefinite Programming [Notes] The Goemans-Williamson algorithm for Max Cut. Related lectures: [Chakrabarty] [Bun] [O'Donnell]. Here is a [Video by O'Donnell] with a similar framing for the separation oracle. See Section 2.3.3 of [Monique] for an implementation of the separation oracle using Gaussian elimination.
Lecture 21, Thursday 11/21: Sherali-Adams and SoS Hierarchies (Part 1) We will discuss how to systematically strengthen any linear program.
Lecture 22, Tuesday 11/26: Sherali-Adams and SoS Hierarchies (Part 2) More on LP hierarchies and some applications.
Thursday 11/28: No class, Fall break Time to relax

Week 14 and 15: A Requested Topic and Final Projects

Lecture 23, Tuesday 12/3: TBD by Vote We will vote on a recent paper to talk about in this lecture.
Thursday, 12/5: Final Project Presentations Lectures from your classmates.
Tuesday, 12/10: Final Project Presentations Lectures from your classmates and the end of the course!