Probing Sentiment in Frozen DistilBERT
Linear Decodability Across Pretrained and Random-Initialized Layers
A logistic-regression probe on frozen DistilBERT reaches 85.9% test accuracy on IMDB sentiment at the final layer. The result means only that sentiment labels are linearly decodable from the activations. It does not show that DistilBERT uses that signal when making a prediction.
What a linear probe measures
At layer \(\ell\), the token activations are averaged into one vector \(h_\ell(x) \in \mathbb{R}^{768}\) for each review. A logistic-regression probe predicts
\[ \hat{y} = \sigma\!\left(w^\top h_\ell(x) + b\right). \]
DistilBERT remains frozen; only \(w\) and \(b\) are trained. High held-out accuracy means that a hyperplane separates many positive reviews from negative ones in that representation. The linear restriction keeps the decoder simple, but it can still exploit any feature correlated with the labels, including dataset artifacts. Linear probes were introduced as a way to track separability across network depth[1].
Experiment
DistilBERT[2] provides seven representations. Layer 0 is the embedding output, followed by the outputs of six transformer blocks. Token vectors are mean-pooled for every layer because the layer-0 \([\text{CLS}]\) vector is constant across reviews. The dataset contains 25,000 training and 25,000 test reviews, balanced between positive and negative sentiment[3].
Each layer receives an L2-regularized logistic-regression probe. The regularization strength is selected on a held-out training slice and then reused across layers. A second DistilBERT with the same architecture and tokenizer but random weights provides a control. Both models remain frozen.
Each probe is fitted on five bootstrap resamples of the training data. This measures sensitivity to the probe's training sample, not to model initialization; the random-initialized curve still represents a single initialization.
Why random embeddings work
The random layer-0 baseline is not empty. Let \(c\) be a review's token-count vector, \(E\) the random embedding matrix, and \(T\) the number of tokens in the review. Mean pooling produces \(c^\top E/T\), so the probe logit is
\[ \frac{1}{T}c^\top Ew. \]
This is a constrained linear bag-of-words classifier. Random embeddings give each token a stable code, and the supervised probe can assign sentiment weights through those codes. The random curve therefore retains information about token identity; it does not isolate a causal contribution from the architecture or provide an additive breakdown of accuracy.
Results
The pretrained curve rises from 82.2% at the embedding layer to 85.9% at the final layer. The random-initialized curve falls from 77.4% to 75.1%. The pretrained checkpoint is 4.8 percentage points higher at layer 0 and 10.8 points higher at layer 6. For this random initialization, passing through six untrained transformer blocks reduces linear separability by 2.3 points.
The figure supports a descriptive conclusion. Pretraining is associated with more linearly separable sentiment features, and that separation increases modestly with depth. It does not show why the curves move or how much each stage causally contributes.
Limitations
Probe accuracy measures decodability, not whether the model uses the decoded feature[5][6]. The results also depend on mean pooling, regularization, and the input distribution. Multiple random-model seeds are needed to characterize the random baseline, and a count-vector or TF-IDF probe would provide a cleaner lexical comparison. A fine-tuned run under the same split and evaluation protocol is required before using the dashed reference as a quantitative comparison.