Lecture 10: Fourier Concentration of DNFs¶
Topics: Mansour's Theorem, Switching Lemma, AC$^0$
O'Donnell Chapter: 4.4
Based on lecture notes by: Lucas Ericsson
Notebook by: Gabriel Taboada
Key Results¶
| Theorem | Statement |
|---|---|
| Thm 10.2 | Size-$s$ DNF has $\text{Inf}[f] \leq O(\log s)$ |
| Switching Lemma | $\Pr[\text{DT-depth}(f_{J,z}) > k] \leq (5pw)^k$ |
| LMN Lemma | Width-$w$ DNFs are $\varepsilon$-concentrated up to degree $O(w \log \frac{1}{\varepsilon})$ |
| Mansour | Width-$w$ DNFs have $\varepsilon$-weight on $\leq w^{O(w \log 1/\varepsilon)}$ coefficients |
# Install/upgrade boofun (required for Colab)
!pip install --upgrade boofun -q
import boofun as bf
print(f"BooFun version: {bf.__version__}")
[notice] A new release of pip is available: 25.2 -> 26.0 [notice] To update, run: /Library/Developer/CommandLineTools/usr/bin/python3 -m pip install --upgrade pip
/Users/gabrieltaboada/dev/Boofun/boofun/src/boofun/core/errormodels.py:21: UserWarning: uncertainties library not available - some error models disabled
warnings.warn("uncertainties library not available - some error models disabled")
/Users/gabrieltaboada/dev/Boofun/boofun/src/boofun/quantum/__init__.py:22: UserWarning: Qiskit not available - quantum features limited
warnings.warn("Qiskit not available - quantum features limited")
BooFun version: 1.1.1
import numpy as np
import boofun as bf
from boofun.analysis.restrictions import (
random_restriction, apply_restriction, switching_lemma_probability
)
from boofun.analysis.complexity import decision_tree_depth
import warnings
warnings.filterwarnings('ignore')
1. DNFs vs Parity: Spectral Contrast¶
Key insight: DNFs have Fourier weight concentrated on low degrees, while Parity has ALL weight on degree $n$.
def print_spectral_weights(f, name, max_bars=30):
"""Print spectral weight by degree."""
weights = f.spectral_weight_by_degree()
total_inf = f.total_influence()
print(f"\n{name} (n={f.n_vars}, Inf[f]={total_inf:.4f})")
print("-" * 50)
max_w = max(weights.values()) if weights else 1
for k in sorted(weights.keys()):
w = weights[k]
bar_len = int((w / max_w) * max_bars) if max_w > 0 else 0
if w > 0.001: # Only show significant weights
print(f" k={k}: {w:.4f} {'#' * bar_len}")
n = 9
print("SPECTRAL CONTRAST: DNF vs Parity")
print("=" * 50)
# Tribes is a canonical DNF (OR of ANDs)
tribes = bf.tribes(3, n) # 3 tribes of 3 vars each
print_spectral_weights(tribes, "Tribes (DNF)")
# Parity has ALL weight on highest degree
parity = bf.parity(n)
print_spectral_weights(parity, "Parity")
print("\n-> Tribes (DNF): weight on low degrees")
print("-> Parity: ALL weight on degree n (NOT in AC^0!)")
SPECTRAL CONTRAST: DNF vs Parity ================================================== Tribes (DNF) (n=9, Inf[f]=1.7227) -------------------------------------------------- k=0: 0.1155 ######### k=1: 0.3297 ############################ k=2: 0.3499 ############################## k=3: 0.1507 ############ k=4: 0.0349 ## k=5: 0.0151 # k=6: 0.0035 Parity (n=9, Inf[f]=9.0000) -------------------------------------------------- k=9: 1.0000 ############################## -> Tribes (DNF): weight on low degrees -> Parity: ALL weight on degree n (NOT in AC^0!)
2. Theorem 10.2: Small DNFs Have Small Influence¶
Theorem 10.2: If $f$ is a size-$s$ DNF, then $\text{Inf}[f] \leq O(\log s)$.
This is proven using random restrictions (Lemma 9.9).
print("Influence Bound for DNFs")
print("=" * 50)
# Compare DNF influence to log(size) bound
for k in [2, 3, 4]:
# k tribes of k vars each = k terms, each of width k
tribes_k = bf.tribes(k, k*k)
n_terms = k # number of tribes (terms)
inf = tribes_k.total_influence()
bound = np.log2(n_terms) if n_terms > 1 else 1
print(f"Tribes({k},{k*k}): {n_terms} terms, n={tribes_k.n_vars}")
print(f" Inf[f] = {inf:.4f}")
print(f" O(log s) ~ {bound:.4f}")
print()
Influence Bound for DNFs ================================================== Tribes(2,4): 2 terms, n=4 Inf[f] = 1.5000 O(log s) ~ 1.0000 Tribes(3,9): 3 terms, n=9 Inf[f] = 1.7227 O(log s) ~ 1.5850 Tribes(4,16): 4 terms, n=16 Inf[f] = 1.6479 O(log s) ~ 2.0000
3. Hastad's Switching Lemma¶
Lemma 10.5 (Switching Lemma): For width-$w$ DNF $f$ and $(J,z) \sim R_p$:
$$\Pr[\text{DT-depth}(f_{J,z}) > k] \leq (5pw)^k$$
Key implication: With $p = \frac{1}{10w}$, the RHS $\leq \frac{1}{2^k}$.
print("Verifying the Switching Lemma")
print("=" * 60)
# Use Tribes as our width-3 DNF (each tribe is an AND of 3 variables)
width = 3
f = bf.tribes(3, 9) # 3 tribes of 3 vars
# Test with p = 1/(10*width) as suggested in lecture
p = 1 / (10 * width)
print(f"Width w = {width}, p = 1/(10w) = {p:.4f}")
print()
# Empirically measure Pr[DT-depth > k]
rng = np.random.default_rng(42)
num_samples = 500
depth_counts = {}
for _ in range(num_samples):
rho = random_restriction(f.n_vars, p, rng)
f_rho = apply_restriction(f, rho)
n_free = f_rho.n_vars if f_rho.n_vars else 0
if n_free > 0:
depth = decision_tree_depth(f_rho)
else:
depth = 0 # Constant function
depth_counts[depth] = depth_counts.get(depth, 0) + 1
print(f"{'k':<5} {'Pr[depth>k]':<15} {'(5pw)^k bound':<15} {'Check':<10}")
print("-" * 50)
for k in range(5):
# Empirical probability
prob_exceed = sum(c for d, c in depth_counts.items() if d > k) / num_samples
# Theoretical bound
bound = switching_lemma_probability(width, p, k)
check = "OK" if prob_exceed <= bound + 0.1 else "FAIL" # Allow small margin
print(f"{k:<5} {prob_exceed:<15.4f} {bound:<15.4f} {check:<10}")
print(f"\n-> Most restrictions reduce depth significantly!")
Verifying the Switching Lemma ============================================================ Width w = 3, p = 1/(10w) = 0.0333 k Pr[depth>k] (5pw)^k bound Check -------------------------------------------------- 0 0.0580 1.0000 OK 1 0.0060 0.5000 OK 2 0.0000 0.2500 OK 3 0.0000 0.1250 OK 4 0.0000 0.0625 OK -> Most restrictions reduce depth significantly!
4. Parity is NOT in AC$^0$¶
Key fact: AC$^0$ circuits have concentrated Fourier spectrum. Parity has ALL weight on degree $n$.
Therefore: Parity cannot be computed by polynomial-size constant-depth circuits.
print("Parity vs AC^0 Functions")
print("=" * 50)
# Compare spectral concentration
n = 8
functions = {
"Tribes (AC^0)": bf.tribes(2, n),
"Majority": bf.majority(n+1), # n+1 to make it odd
"AND (AC^0)": bf.AND(n),
"Parity (NOT AC^0)": bf.parity(n),
}
from boofun.analysis import SpectralAnalyzer
print(f"{'Function':<20} {'90% weight by deg':<20} {'99% weight by deg':<20}")
print("-" * 60)
for name, f in functions.items():
analyzer = SpectralAnalyzer(f)
deg_90, deg_99 = None, None
for d in range(f.n_vars + 1):
conc = analyzer.spectral_concentration(d)
if conc >= 0.9 and deg_90 is None:
deg_90 = d
if conc >= 0.99 and deg_99 is None:
deg_99 = d
break
print(f"{name:<20} {deg_90 or f.n_vars:<20} {deg_99 or f.n_vars:<20}")
print("\n-> AC^0 functions concentrate on low degrees.")
print("-> Parity has ALL weight at degree n = not learnable from low-degree!")
Parity vs AC^0 Functions ================================================== Function 90% weight by deg 99% weight by deg ------------------------------------------------------------ Tribes (AC^0) 3 5 Majority 7 9 AND (AC^0) 8 3 Parity (NOT AC^0) 8 8 -> AC^0 functions concentrate on low degrees. -> Parity has ALL weight at degree n = not learnable from low-degree!
Summary¶
| Function Class | Fourier Concentration | Learnable? |
|---|---|---|
| Width-$w$ DNF | Degree $O(w \log 1/\varepsilon)$ | Yes (quasi-poly time) |
| Size-$s$ DNF | $\text{Inf}[f] \leq O(\log s)$ | Yes |
| AC$^0$ | Low-degree concentrated | Yes |
| Parity | ALL weight on degree $n$ | No (not in AC$^0$) |
Key Formulas¶
- Switching Lemma: $\Pr[\text{DT-depth}(f_{J,z}) > k] \leq (5pw)^k$
- LMN Concentration: $W^{\geq 10wk}[f] \leq 2 \cdot (1/2)^k$
boofun API¶
from boofun.analysis.restrictions import switching_lemma_probability
from boofun.analysis.complexity import decision_tree_depth
switching_lemma_probability(width, p, k) # (5pw)^k bound
decision_tree_depth(f) # Optimal DT depth
f.spectral_weight_by_degree() # W_k[f] for each k