Open In Colab

Lecture 10: Fourier Concentration of DNFs¶

Topics: Mansour's Theorem, Switching Lemma, AC$^0$

O'Donnell Chapter: 4.4
Based on lecture notes by: Lucas Ericsson
Notebook by: Gabriel Taboada


Key Results¶

Theorem Statement
Thm 10.2 Size-$s$ DNF has $\text{Inf}[f] \leq O(\log s)$
Switching Lemma $\Pr[\text{DT-depth}(f_{J,z}) > k] \leq (5pw)^k$
LMN Lemma Width-$w$ DNFs are $\varepsilon$-concentrated up to degree $O(w \log \frac{1}{\varepsilon})$
Mansour Width-$w$ DNFs have $\varepsilon$-weight on $\leq w^{O(w \log 1/\varepsilon)}$ coefficients
In [1]:
# Install/upgrade boofun (required for Colab)
!pip install --upgrade boofun -q

import boofun as bf
print(f"BooFun version: {bf.__version__}")
[notice] A new release of pip is available: 25.2 -> 26.0
[notice] To update, run: /Library/Developer/CommandLineTools/usr/bin/python3 -m pip install --upgrade pip
/Users/gabrieltaboada/dev/Boofun/boofun/src/boofun/core/errormodels.py:21: UserWarning: uncertainties library not available - some error models disabled
  warnings.warn("uncertainties library not available - some error models disabled")
/Users/gabrieltaboada/dev/Boofun/boofun/src/boofun/quantum/__init__.py:22: UserWarning: Qiskit not available - quantum features limited
  warnings.warn("Qiskit not available - quantum features limited")
BooFun version: 1.1.1
In [2]:
import numpy as np
import boofun as bf
from boofun.analysis.restrictions import (
    random_restriction, apply_restriction, switching_lemma_probability
)
from boofun.analysis.complexity import decision_tree_depth

import warnings
warnings.filterwarnings('ignore')

1. DNFs vs Parity: Spectral Contrast¶

Key insight: DNFs have Fourier weight concentrated on low degrees, while Parity has ALL weight on degree $n$.

In [3]:
def print_spectral_weights(f, name, max_bars=30):
    """Print spectral weight by degree."""
    weights = f.spectral_weight_by_degree()
    total_inf = f.total_influence()
    print(f"\n{name} (n={f.n_vars}, Inf[f]={total_inf:.4f})")
    print("-" * 50)
    max_w = max(weights.values()) if weights else 1
    for k in sorted(weights.keys()):
        w = weights[k]
        bar_len = int((w / max_w) * max_bars) if max_w > 0 else 0
        if w > 0.001:  # Only show significant weights
            print(f"  k={k}: {w:.4f} {'#' * bar_len}")

n = 9
print("SPECTRAL CONTRAST: DNF vs Parity")
print("=" * 50)

# Tribes is a canonical DNF (OR of ANDs)
tribes = bf.tribes(3, n)  # 3 tribes of 3 vars each
print_spectral_weights(tribes, "Tribes (DNF)")

# Parity has ALL weight on highest degree
parity = bf.parity(n)
print_spectral_weights(parity, "Parity")

print("\n-> Tribes (DNF): weight on low degrees")
print("-> Parity: ALL weight on degree n (NOT in AC^0!)")
SPECTRAL CONTRAST: DNF vs Parity
==================================================

Tribes (DNF) (n=9, Inf[f]=1.7227)
--------------------------------------------------
  k=0: 0.1155 #########
  k=1: 0.3297 ############################
  k=2: 0.3499 ##############################
  k=3: 0.1507 ############
  k=4: 0.0349 ##
  k=5: 0.0151 #
  k=6: 0.0035 

Parity (n=9, Inf[f]=9.0000)
--------------------------------------------------
  k=9: 1.0000 ##############################

-> Tribes (DNF): weight on low degrees
-> Parity: ALL weight on degree n (NOT in AC^0!)

2. Theorem 10.2: Small DNFs Have Small Influence¶

Theorem 10.2: If $f$ is a size-$s$ DNF, then $\text{Inf}[f] \leq O(\log s)$.

This is proven using random restrictions (Lemma 9.9).

In [4]:
print("Influence Bound for DNFs")
print("=" * 50)

# Compare DNF influence to log(size) bound
for k in [2, 3, 4]:
    # k tribes of k vars each = k terms, each of width k
    tribes_k = bf.tribes(k, k*k)
    n_terms = k  # number of tribes (terms)
    inf = tribes_k.total_influence()
    bound = np.log2(n_terms) if n_terms > 1 else 1
    
    print(f"Tribes({k},{k*k}): {n_terms} terms, n={tribes_k.n_vars}")
    print(f"  Inf[f] = {inf:.4f}")
    print(f"  O(log s) ~ {bound:.4f}")
    print()
Influence Bound for DNFs
==================================================
Tribes(2,4): 2 terms, n=4
  Inf[f] = 1.5000
  O(log s) ~ 1.0000

Tribes(3,9): 3 terms, n=9
  Inf[f] = 1.7227
  O(log s) ~ 1.5850

Tribes(4,16): 4 terms, n=16
  Inf[f] = 1.6479
  O(log s) ~ 2.0000


3. Hastad's Switching Lemma¶

Lemma 10.5 (Switching Lemma): For width-$w$ DNF $f$ and $(J,z) \sim R_p$:

$$\Pr[\text{DT-depth}(f_{J,z}) > k] \leq (5pw)^k$$

Key implication: With $p = \frac{1}{10w}$, the RHS $\leq \frac{1}{2^k}$.

In [5]:
print("Verifying the Switching Lemma")
print("=" * 60)

# Use Tribes as our width-3 DNF (each tribe is an AND of 3 variables)
width = 3
f = bf.tribes(3, 9)  # 3 tribes of 3 vars

# Test with p = 1/(10*width) as suggested in lecture
p = 1 / (10 * width)
print(f"Width w = {width}, p = 1/(10w) = {p:.4f}")
print()

# Empirically measure Pr[DT-depth > k]
rng = np.random.default_rng(42)
num_samples = 500
depth_counts = {}

for _ in range(num_samples):
    rho = random_restriction(f.n_vars, p, rng)
    f_rho = apply_restriction(f, rho)
    n_free = f_rho.n_vars if f_rho.n_vars else 0
    if n_free > 0:
        depth = decision_tree_depth(f_rho)
    else:
        depth = 0  # Constant function
    depth_counts[depth] = depth_counts.get(depth, 0) + 1

print(f"{'k':<5} {'Pr[depth>k]':<15} {'(5pw)^k bound':<15} {'Check':<10}")
print("-" * 50)

for k in range(5):
    # Empirical probability
    prob_exceed = sum(c for d, c in depth_counts.items() if d > k) / num_samples
    # Theoretical bound
    bound = switching_lemma_probability(width, p, k)
    check = "OK" if prob_exceed <= bound + 0.1 else "FAIL"  # Allow small margin
    print(f"{k:<5} {prob_exceed:<15.4f} {bound:<15.4f} {check:<10}")

print(f"\n-> Most restrictions reduce depth significantly!")
Verifying the Switching Lemma
============================================================
Width w = 3, p = 1/(10w) = 0.0333

k     Pr[depth>k]     (5pw)^k bound   Check     
--------------------------------------------------
0     0.0580          1.0000          OK        
1     0.0060          0.5000          OK        
2     0.0000          0.2500          OK        
3     0.0000          0.1250          OK        
4     0.0000          0.0625          OK        

-> Most restrictions reduce depth significantly!

4. Parity is NOT in AC$^0$¶

Key fact: AC$^0$ circuits have concentrated Fourier spectrum. Parity has ALL weight on degree $n$.

Therefore: Parity cannot be computed by polynomial-size constant-depth circuits.

In [6]:
print("Parity vs AC^0 Functions")
print("=" * 50)

# Compare spectral concentration
n = 8
functions = {
    "Tribes (AC^0)": bf.tribes(2, n),
    "Majority": bf.majority(n+1),  # n+1 to make it odd
    "AND (AC^0)": bf.AND(n),
    "Parity (NOT AC^0)": bf.parity(n),
}

from boofun.analysis import SpectralAnalyzer

print(f"{'Function':<20} {'90% weight by deg':<20} {'99% weight by deg':<20}")
print("-" * 60)

for name, f in functions.items():
    analyzer = SpectralAnalyzer(f)
    deg_90, deg_99 = None, None
    for d in range(f.n_vars + 1):
        conc = analyzer.spectral_concentration(d)
        if conc >= 0.9 and deg_90 is None:
            deg_90 = d
        if conc >= 0.99 and deg_99 is None:
            deg_99 = d
            break
    print(f"{name:<20} {deg_90 or f.n_vars:<20} {deg_99 or f.n_vars:<20}")

print("\n-> AC^0 functions concentrate on low degrees.")
print("-> Parity has ALL weight at degree n = not learnable from low-degree!")
Parity vs AC^0 Functions
==================================================
Function             90% weight by deg    99% weight by deg   
------------------------------------------------------------
Tribes (AC^0)        3                    5                   
Majority             7                    9                   
AND (AC^0)           8                    3                   
Parity (NOT AC^0)    8                    8                   

-> AC^0 functions concentrate on low degrees.
-> Parity has ALL weight at degree n = not learnable from low-degree!

Summary¶

Function Class Fourier Concentration Learnable?
Width-$w$ DNF Degree $O(w \log 1/\varepsilon)$ Yes (quasi-poly time)
Size-$s$ DNF $\text{Inf}[f] \leq O(\log s)$ Yes
AC$^0$ Low-degree concentrated Yes
Parity ALL weight on degree $n$ No (not in AC$^0$)

Key Formulas¶

  • Switching Lemma: $\Pr[\text{DT-depth}(f_{J,z}) > k] \leq (5pw)^k$
  • LMN Concentration: $W^{\geq 10wk}[f] \leq 2 \cdot (1/2)^k$

boofun API¶

from boofun.analysis.restrictions import switching_lemma_probability
from boofun.analysis.complexity import decision_tree_depth

switching_lemma_probability(width, p, k)  # (5pw)^k bound
decision_tree_depth(f)                    # Optimal DT depth
f.spectral_weight_by_degree()             # W_k[f] for each k