{"id":201620,"date":"2025-12-24T13:49:13","date_gmt":"2025-12-24T13:49:13","guid":{"rendered":"https:\/\/www.newsbeep.com\/il\/201620\/"},"modified":"2025-12-24T13:49:13","modified_gmt":"2025-12-24T13:49:13","slug":"the-machine-learning-advent-calendar-day-23-1d-cnn-for-text-in-excel","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/il\/201620\/","title":{"rendered":"The Machine Learning \u201cAdvent Calendar\u201d Day 23: 1D CNN for Text in Excel"},"content":{"rendered":"

were first introduced for images, and for images they are often easy to understand.<\/p>\n

A filter slides over pixels and detects edges, shapes, or textures. You can read this article I wrote earlier<\/a> to understand how CNNs work for images with Excel.<\/p>\n

For text, the idea is the same.<\/p>\n

Instead of pixels, we slide filters over words.<\/p>\n

Instead of visual patterns, we detect linguistic patterns.<\/p>\n

And many important patterns in text are very local. Let\u2019s take these very simple examples:<\/p>\n

\u201cgood\u201d is positive<\/p>\n

\u201cbad\u201d is negative<\/p>\n

\u201cnot good\u201d is negative<\/p>\n

\u201cnot bad\u201d is often positive<\/p>\n

In my previous article<\/a>, we saw how to represent words as numbers using embeddings.<\/p>\n

We also saw a key limitation: when we used a global average, word order was completely ignored.<\/p>\n

From the model\u2019s point of view, \u201cnot good\u201d and \u201cgood not\u201d looked exactly the same.<\/p>\n

So the next challenge is clear: we want the model to take word order into account.<\/p>\n

A 1D Convolutional Neural Network is a natural tool for this, because it scans a sentence with small sliding windows and reacts when it recognizes familiar local patterns.<\/p>\n

1. Understanding a 1D CNN for Text: Architecture and Depth<\/p>\n

1.1. Building a 1D CNN for text in Excel<\/p>\n

In this article, we build a 1D CNN architecture in Excel with the following components:<\/p>\n

Embedding dictionary
We use a 2-dimensional embedding. Because one dimension is not enough for this task.
One dimension encodes sentiment, and the second dimension encodes negation.<\/p>\n

Conv1D layer
This is the core component of a CNN architecture.
It consists of filters that slide across the sentence with a window length of 2 words. We choose 2 words to be simple.<\/p>\n

ReLU and global max pooling
These steps keep only the strongest matches detected by the filters.
We will also discuss the fact that ReLU is optional.<\/p>\n

Logistic regression
This is the final classification layer, which combines the detected patterns into a probability.<\/p>\n

\"\"1D CNN in Excel \u2013 all images by author<\/p>\n

This pipeline corresponds to a standard CNN text classifier.
The only difference here is that we explicitly write and visualize the forward pass in Excel.<\/p>\n

1.2. What \u201cdeep learning\u201d means in this architecture<\/p>\n

Before going further, let us take a step back.
Yes, I know, I do this often, but having a global view of models really helps to understand them.<\/p>\n

The definition of deep learning is often blurred.
For many people, deep learning simply means \u201cmany layers\u201d.<\/p>\n

Here, I will take a slightly different point of view.<\/p>\n

What really characterizes deep learning is not the number of layers, but the depth of the transformation applied to the input data.<\/p>\n

With this definition:<\/p>\n

Even a model with a single convolution layer can be considered deep learning,<\/p>\n

because the input is transformed into a more structured and abstract representation.<\/p>\n

On the other hand, taking raw input data, applying one-hot encoding, and stacking many fully connected layers does not necessarily make a model deep in a meaningful sense.
In theory, if we don\u2019t have any transformation, one layer is enough.<\/p>\n

In CNNs, the presence of multiple layers has a very concrete motivation.<\/p>\n

Consider a sentence like:<\/p>\n

This movie is not very good<\/p>\n

With a single convolution layer and a small window, we can detect simple local patterns such as: \u201cvery + good\u201d<\/p>\n

But we cannot yet detect higher-level patterns such as: \u201cnot + (very good)\u201d<\/p>\n

This is why CNNs are often stacked:<\/p>\n

the first layer detects simple local patterns,<\/p>\n

the second layer combines them into more complex ones.<\/p>\n

In this article, we deliberately focus on one convolution layer.
This makes every step visible and easy to understand in Excel, while keeping the logic identical to deeper CNN architectures.<\/p>\n

2. Turning words into embeddings<\/p>\n

Let us start with some simple words.\u00a0 We will try to detect negation, so we will use these terms, with other words (that we will not model)<\/p>\n

\u201cgood\u201d<\/p>\n

\u201cbad\u201d<\/p>\n

\u201cnot good\u201d<\/p>\n

\u201cnot bad\u201d<\/p>\n

We keep the representation intentionally small so that every step is visible.<\/p>\n

We will only use a dictionary of three words : good, bad and not.<\/p>\n

All other words will have 0 as embeddings.<\/p>\n

2.1 Why one dimension is not enough<\/p>\n

In a previous article on sentiment detection, we used a single dimension.
That worked for \u201cgood\u201d versus \u201cbad\u201d.<\/p>\n

But now we want to handle negation.<\/p>\n

One dimension can only represent one concept well.
So we need two dimensions:<\/p>\n

senti: sentiment polarity<\/p>\n

neg: negation marker<\/p>\n

2.2 The embedding dictionary<\/p>\n

Each word becomes a 2D vector:<\/p>\n

good \u2192 (senti = +1, neg = 0)<\/p>\n

bad \u2192 (senti = -1, neg = 0)<\/p>\n

not \u2192 (senti = 0, neg = +1)<\/p>\n

any other word \u2192 (0, 0)<\/p>\n

\"\"<\/p>\n

This is not how real embeddings look. Real embeddings are learned, high-dimensional, and not directly interpretable.<\/p>\n

But for understanding how Conv1D works, this toy embedding is perfect.<\/p>\n

In Excel, this is just a lookup table.
In a real neural network, this embedding matrix would be trainable.<\/p>\n

\"\"<\/p>\n

3. Conv1D filters as sliding pattern detectors<\/p>\n

Now we arrive at the core idea of a 1D CNN.<\/p>\n

A Conv1D filter is nothing mysterious. It is just a small set of weights plus a bias that slides over the sentence.<\/p>\n

Because:<\/p>\n

each word embedding has 2 values (senti, neg)<\/p>\n

our window contains 2 words<\/p>\n

each filter has:<\/p>\n

4 weights (2 dimensions \u00d7 2 positions)<\/p>\n

1 bias<\/p>\n

That is all.<\/p>\n

You can think of a filter as repeatedly asking the same question at every position:<\/p>\n

\u201cDo these two neighboring words match a pattern I care about?\u201d<\/p>\n

3.1 Sliding windows: how Conv1D sees a sentence<\/p>\n

Consider this sentence:<\/p>\n

it is not bad at all<\/p>\n

We choose a window size of 2 words.<\/p>\n

That means the model looks at every adjacent pair:<\/p>\n

(it, is)<\/p>\n

(is, not)<\/p>\n

(not, bad)<\/p>\n

(bad, at)<\/p>\n

(at, all)<\/p>\n

Important point:
The filters slide everywhere, even when both words are neutral (all zeros).<\/p>\n

\"\"<\/p>\n

3.2 Four intuitive filters<\/p>\n

To make the behavior easy to understand, we use four filters.<\/p>\n

\"\"<\/p>\n

Filter 1 \u2013 \u201cI see GOOD\u201d<\/p>\n

This filter looks only at the sentiment of the current word.<\/p>\n

Plain-text equation for one window:<\/p>\n

z = senti(current_word)<\/p>\n

If the word is \u201cgood\u201d, z = 1
If the word is \u201cbad\u201d, z = -1
If the word is neutral, z = 0<\/p>\n

After ReLU, negative values become 0. But it is optional.<\/p>\n

Filter 2 \u2013 \u201cI see BAD\u201d<\/p>\n

This one is symmetric.<\/p>\n

z = -senti(current_word)<\/p>\n

So:<\/p>\n

\u201cbad\u201d \u2192 z = 1<\/p>\n

\u201cgood\u201d \u2192 z = -1 \u2192 ReLU \u2192 0<\/p>\n

Filter 3 \u2013 \u201cI see NOT GOOD\u201d<\/p>\n

This filter looks at two things at the same time:<\/p>\n

neg(previous_word)<\/p>\n

senti(current_word)<\/p>\n

Equation:<\/p>\n

z = neg(previous_word) + senti(current_word) \u2013 1<\/p>\n

Why the \u201c-1\u201d?
It acts like a threshold so that both conditions must be true.<\/p>\n

Results:<\/p>\n

\u201cnot good\u201d \u2192 1 + 1 \u2013 1 = 1 \u2192 activated<\/p>\n

\u201cis good\u201d \u2192 0 + 1 \u2013 1 = 0 \u2192 not activated<\/p>\n

\u201cnot bad\u201d \u2192 1 \u2013 1 \u2013 1 = -1 \u2192 ReLU \u2192 0<\/p>\n

Filter 4 \u2013 \u201cI see NOT BAD\u201d<\/p>\n

Same idea, slightly different sign:<\/p>\n

z = neg(previous_word) + (-senti(current_word)) \u2013 1<\/p>\n

Results:<\/p>\n

\u201cnot bad\u201d \u2192 1 + 1 \u2013 1 = 1<\/p>\n

\u201cnot good\u201d \u2192 1 \u2013 1 \u2013 1 = -1 \u2192 0<\/p>\n

This is a very important intuition:<\/p>\n

A CNN filter can behave like a local logical rule, learned from data.<\/p>\n

3.3 Final result of sliding windows<\/p>\n

Here is the final results of these 4 filters.<\/p>\n

\"\"<\/p>\n

4. ReLU and max pooling: from local to global<\/p>\n

4.1 ReLU<\/p>\n

After computing z for every window, we apply ReLU:<\/p>\n

ReLU(z) = max(0, z)<\/p>\n

Meaning:<\/p>\n

negative evidence is ignored<\/p>\n

positive evidence is kept<\/p>\n

Each filter becomes a presence detector.<\/p>\n

By the way, it is an activation function in the Neural network. So a Neural network is not that difficult after all.<\/p>\n

\"\"<\/p>\n

4.2 Global Max pooling<\/p>\n

Then comes global max pooling.<\/p>\n

For each filter, we keep only:<\/p>\n

max activation over all windows<\/p>\n

Interpretation:
\u201cI do not care where the pattern appears, only whether it appears strongly somewhere.\u201d<\/p>\n

At this point, the whole sentence is summarized by 4 numbers:<\/p>\n

strongest \u201cgood\u201d signal<\/p>\n

strongest \u201cbad\u201d signal<\/p>\n

strongest \u201cnot good\u201d signal<\/p>\n

strongest \u201cnot bad\u201d signal<\/p>\n

\"\"<\/p>\n

4.3 What happens if we remove ReLU?<\/p>\n

Without ReLU:<\/p>\n

negative values stay negative<\/p>\n

max pooling may select negative values<\/p>\n

This mixes two ideas:<\/p>\n

absence of a pattern<\/p>\n

opposite of a pattern<\/p>\n

The filter stops being a clean detector and becomes a signed score.<\/p>\n

The model could still work mathematically, but interpretation becomes harder.<\/p>\n

5. The final layer is logistic regression<\/p>\n

Now we combine these signals.<\/p>\n

We compute a score using a linear combination:<\/p>\n

score = 2 \u00d7 F_good \u2013 2 \u00d7 F_bad \u2013 3 \u00d7 F_not_good \u2013 3 \u00d7 F_not_bad \u2013 bias<\/p>\n

\"\"<\/p>\n

Then we convert the score into a probability:<\/p>\n

probability = 1 \/ (1 + exp(-score))<\/p>\n

That is exactly logistic regression.<\/p>\n

So yes:<\/p>\n

the CNN extracts features: this step can be considered as feature engineering, right?<\/p>\n

logistic regression makes the final decisions, it is a classic machine learning model we know well<\/p>\n

\"\"<\/p>\n

6. Full examples with sliding filters<\/p>\n

Example 1<\/p>\n

\u201cit is bad, so it is not good at all\u201d<\/p>\n

The sentence contains:<\/p>\n

After max pooling:<\/p>\n

F_good = 1 (because \u201cgood\u201d exists)<\/p>\n

F_bad = 1<\/p>\n

F_not_good = 1<\/p>\n

F_not_bad = 0<\/p>\n

Final score becomes strongly negative.
Prediction: negative sentiment.<\/p>\n

\"\"<\/p>\n

Example 2<\/p>\n

\u201cit is good. yes, not bad.\u201d<\/p>\n

The sentence contains:<\/p>\n

After max pooling:<\/p>\n

F_good = 1<\/p>\n

F_bad = 1 (because the word \u201cbad\u201d appears)<\/p>\n

F_not_good = 0<\/p>\n

F_not_bad = 1<\/p>\n

The final linear layer learns that \u201cnot bad\u201d should outweigh \u201cbad\u201d.<\/p>\n

Prediction: positive sentiment.<\/p>\n

This also shows something important: max pooling keeps all strong signals.
The final layer decides how to combine them.<\/p>\n

\"\"<\/p>\n

Exemple 3 with A limitation that explains why CNNs get deeper<\/p>\n

Try this sentence:<\/p>\n

\u201cit is not very bad\u201d<\/p>\n

With a window of size 2, the model sees:<\/p>\n

It never sees (not, bad), so the \u201cnot bad\u201d filter never fires.<\/p>\n

It explains why real models use:<\/p>\n

larger windows<\/p>\n

multiple convolution layers<\/p>\n

or other architectures for longer dependencies<\/p>\n

\"\"<\/p>\n

Conclusion<\/p>\n

The strength of Excel is visibility.<\/p>\n

You can see:<\/p>\n

the embedding dictionary<\/p>\n

all filter weights and biases<\/p>\n

every sliding window<\/p>\n

every ReLU activation<\/p>\n

the max pooling result<\/p>\n

the logistic regression parameters<\/p>\n

Training is simply the process of adjusting these numbers.<\/p>\n

Once you see that, CNNs stop being mysterious.<\/p>\n

They become what they really are: structured, trainable pattern detectors that slide over data.<\/p>\n","protected":false},"excerpt":{"rendered":"were first introduced for images, and for images they are often easy to understand. A filter slides over…\n","protected":false},"author":2,"featured_media":201621,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[345,343,344,15394,10716,2158,85,46,1359,125],"class_list":["post-201620","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-cnn","tag-data-science","tag-deep-learning","tag-il","tag-israel","tag-machine-learning","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/201620","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=201620"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/201620\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/201621"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=201620"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=201620"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=201620"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}