10 Dimensionality Reduction

Tip

🆕 Դասերն անցկացված են։ Սլայդերը, նշումներով PDF-երը և տեսանյութերը ներքևում են՝ երկու դասախոսություն ([36] PCA և SVD, [37] t-SNE և UMAP)։

🎲 Random

TBD

📚 Նյութը

Երկու դասախոսություն․ առաջինը՝ ինչ է տալիս չափողականության իջեցումը և երբ որ մեթոդին դիմել, երկրորդը՝ ինչպես է UMAP-ն իրականում աշխատում։ Սլայդերը ml/10_dimensionality_reduction/ պանակում։

📝 Թեմայի վերաբերյալ հարցաշար (Google Form): TBD

References:

🏡 Տնային

Important

Հանձնարարվող պրոջեկտը Project 3-ն է — սարքիր քո սեփական Google Photos-ը՝ քո նկարներով։ Project 1-ը լավ տաքացում է, Project 2-ը՝ «hard mode» ամենահավակնոտների համար․ երկուսն էլ կամավոր են։

Project 1 — Eigenfaces: a face in a handful of numbers 🧀🧀

A 64×64 face is 4096 numbers, but faces live near a much smaller subspace. PCA finds that subspace, and its components — drawn back as images — look like ghostly faces (“eigenfaces”). In this project you’ll compress faces, visualize them in 2-D, and finally recognize them, all with the tools from the lecture.

Setup. Data: sklearn.datasets.fetch_olivetti_faces (40 people × 10 images, 64×64, values in [0, 1]). One reproducible notebook, seed 509, ending with a 3–5 sentence conclusion. Save results to an out/ subfolder.

Tasks.

  1. Load the faces, show a few, and flatten to (400, 4096).
  2. Fit PCA; display the mean face and the top eigenfaces (components reshaped to 64×64).
  3. Plot the scree and cumulative explained variance; pick k for ~95%.
  4. Reconstruct a face at several k (e.g. 10 / 50 / 150) and plot reconstruction error vs k. How few numbers still look like the person?
  5. Visualize face space: embed in 2-D three ways — PCA, t-SNE, and UMAP — colored by person (use ~10 people so it stays readable). Which one separates identities best?
  6. Recognizer: make a stratified train/test split, fit PCA on the training faces, and classify test faces by nearest neighbor in PCA space. Plot accuracy vs number of components, and compare to nearest neighbor on the raw pixels.
  7. (Callback) Run k-means in PCA space and compare the clusters to the true identities with the adjusted Rand index.

Bonus: whiten the PCA (whiten=True); add a few of your own photos; vary the train/test split.

Project 2 — When eigenfaces meet the real world 🧀🧀🧀

Project 1 works. It works because Olivetti is a laboratory dataset: every photo is centered, cropped identically, lit the same way, taken against the same wall, on the same day. Change any of that and the method falls over.

This project is about making it fail, understanding exactly why, and then fixing it. You are not following a recipe here — you are diagnosing a broken model. The write-up matters as much as the accuracy number.

Setup. Data: sklearn.datasets.fetch_lfw_people — Labeled Faces in the Wild, real photographs scraped from news articles. Start with min_faces_per_person=70, resize=0.4 (≈1288 images, 7 people). Seed 509, one reproducible notebook, results to out/.

Note

Heads up on the download. fetch_lfw_people pulls ~200 MB the first time and caches it in ~/scikit_learn_data/. Do this once, early, not five minutes before the deadline.

Part A — Break it

  1. Run exactly your Project 1 pipeline on LFW: PCA, then nearest neighbor in PCA space. Report accuracy next to your Olivetti number.
  2. Do not explain the gap yet. Just show it, and show a confusion matrix and per-person accuracy. Which person is hardest?
  3. Compute the majority-class baseline — this subset is badly imbalanced, and one person accounts for a large share of it. Report your accuracy next to that baseline. An accuracy that sounds respectable can be much less impressive than it looks.

Part B — Diagnose it

  1. Display the top 10 eigenfaces for LFW next to your Olivetti ones. They look different. Describe how.
  2. Test the suspicion directly: compute each image’s mean brightness and correlate it with each of the first five PC scores. Report the five correlations, together with how much variance each of those components explains. One of them will not be subtle.
  3. Make the argument visual: colour your 2-D PCA scatter by brightness, then colour the same scatter by person. Which one looks organized?
Tip

This is the lecture’s “when NOT to reduce” slide, happening to you. PCA ranks directions by variance, with no knowledge of y. Across web photos, illumination and pose vary far more than identity does — so the fat, high-variance axes are lighting, and identity is hiding in thin ones PCA is happy to throw away.

Part C — Fix it

Try all three, measure each one, and report them in a single table. At least one of them will disappoint you — that is deliberate, and reporting it honestly is part of the task.

  1. Drop the leading components. Rebuild the recognizer on components d:k instead of 0:k — the classic trick from the Fisherfaces paper. Sweep d from 0 to about 8 and plot accuracy against it.
  2. Normalize per image (standardize each face, or histogram-equalize it) before PCA, so brightness cannot become a direction at all. Does this help as much as task 7? If not, work out why — the two fixes sound equivalent but are not.
  3. Use the labels. Fit LDA (LinearDiscriminantAnalysis) on top of the PCA scores — this is Fisherfaces. Where PCA ranks directions by raw variance, LDA maximizes between-class over within-class scatter, so it is explicitly hunting the directions that separate people.
  4. Now combine your best two fixes. Does the combination beat either alone? If it does not, explain what that tells you about what LDA was already doing.
  5. State plainly which fix did the real work, with the numbers side by side.

Part D — Be honest about it

  1. Report your best pipeline’s confusion matrix and per-person accuracy. Then actually look at the misclassified images and name the failure modes you can see — profile shots? sunglasses? motion blur? someone photographed mostly at a single event, against a single backdrop?
  2. Check whether your gains are spread evenly. Did the rare people improve, or did the model just get better at the majority class? A per-person accuracy table answers this and a single number does not.
  3. One paragraph: would you deploy this? If not, what would you need instead? (You do not have to build it. You have to know what it would take.)

Bonus (optional). Add ~10 photos of yourself, cropped and resized to match LFW’s geometry, as an eighth person; report whether the recognizer picks you up and what it takes. Skip this if you would rather not put your face in a homework repo — the project is complete without it.

Important

What is actually being graded: Part B and Part D. Anyone can call PCA and LinearDiscriminantAnalysis. Far fewer people can look at a model that reports 65% and explain precisely which property of the data produced that number.

Project 3 — Build your own Google Photos 🧀🧀

Your phone does this silently every day: thousands of photos, no labels, and yet “beach”, “food” and “screenshots of documents” somehow become groups. In this project you build that yourself — on your own pictures.

Clustering raw pixels gives you colors, not objects — that was the lesson of the clustering chapter’s image projects. This is a dimensionality-reduction project because the representation is the whole story: swap the pixels for 512 learned numbers, change nothing else, and watch what happens.

Step 0. Gather your photos into one folder — trip photos, screenshots, pictures of cheeses; a few hundred is a good amount, and nothing sensitive is required. There is no prepared dataset: everything runs on your own collection, which also means no two submissions can come out alike.

The tool: CLIP. OpenAI’s CLIP (2021, open-sourced — from the times when OpenAI and the word “open” still had something in common) turns a picture into an embedding: a vector of 512 numbers, where photos with similar content get nearby vectors. Treat it as a black box for now — the neural-networks block explains how it works. The one property to remember: text lives in the same space as images.

Given code — you are not asked to write the tooling, only to use it. The ML (clustering, choosing k, the projections, the analysis) is yours; the engineering is provided:

  • py_src/embed_my_photos.py — point it at your folder, get back an npz with the embeddings and small thumbnails. One-time pip install torch transformers; CPU is fine — a few hundred photos take minutes, though the ~600 MB model download can take a while on a slow connection. iPhone HEIC photos work too if you also pip install pillow-heif.
  • py_src/photo_map.py — make_photo_map(xy, jpegs, ...) turns your 2-D points + the thumbnails into a self-contained interactive hover-the-photo HTML map in one function call (jpegs_from_npz unpacks the thumbnails).

Tasks.

  1. First cluster the raw pixels: shrink the images, flatten them to vectors, k-means. Look at the resulting groups — what actually got clustered?
  2. Now the same on the embeddings (from the given script), everything else unchanged. Compare the groups by eye — this contrast is the whole project.
  3. (Optional, but recommended) hand-label a few dozen of your photos into categories — now you have “true” labels and can score both representations with ARI / AMI, turning the task-1-vs-2 comparison into numbers.
  4. Choose k with the elbow and the silhouette (the silhouette needs no labels) and justify the choice.
  5. Project to 2D (t-SNE or UMAP; add PCA for comparison) and make the normal matplotlib scatter, colored by your clusters. Then hand the same 2-D points to the provided make_photo_map(...) and open your interactive map — this is your Google Photos moment.
  6. Find the cluster that acts as a junk drawer (the everything-in-one-place group) and say why one always appears.

Requirements. One reproducible notebook, seed 509, and a hand-written 3–5-sentence conclusion. torch is needed only for the embedding step; everything else is numpy / scikit-learn / matplotlib (+ umap-learn). Keep the code simple; use LLMs sparingly and review every line you take from one — the decisions (which k, which method, why) are the part that gets graded.

Extra, for the curious: CLIP embeds text into the same space as images. Take a list of words (“sea”, “food”, “cat”, “document”, …), embed them too (transformers’ get_text_features is the door), and name each cluster automatically by its centre’s nearest word.

Bonus: hover along a boundary between two clusters and collect the genuinely ambiguous photos; try k well above your chosen one and see what splits first.

Reference walkthrough — the same pipeline on a 92-photo demo set (11 categories fetched from the web: Khustup, Sevan, shawarma, khorovats, khachkars…), embedded with the same given scripts, plus a CLIP playground: searching photos with words, and vector arithmetic like mean(sevan) − mean(khustup) ≈ "lake". 37_image_clusters_solution.ipynb (download) · view on GitHub — useful as a reference, but your submission runs on your photos, so copying it wholesale cannot produce your project.

Flag Counter