Update: Backpropagation in matrix form could be found here

Intro

In the field of deep learning, optimizing the network involves a crucial process of continuously updating the weights and bias items. This is achieved by implementing the gradient descent method, which progressively minimizes the loss function. At the heart of this process lies the backpropagation algorithm, which facilitates efficient computation of gradients across the network

To better understand this concept, let us recall the formula for gradient descent. In this formula, we utilize the symbol $\theta$ to represent all the learnable parameters of the model, $J$ to represent the cost or loss function, and $\alpha$ to denote the learning rate. Thus, we can express the updating process as:

Introduction

Recently, I review the machine learning course of Andrew ng in Coursera. Surprisingly, I can still learn a lot, so I decided to write some posts👍.

To talk about linear regression, we must first have a basic understanding of what is machine learning. What is machine learning? abstractly speaking, machine learning is learning a function: $$ f(input) = output $$ where $f$ refers to the specific machine learning model. Machine learning is a methodology for automatically mining the relationship between input and output. Sometimes we find it hard to define a specific algorithm to solve some problems, and this is where machine learning shines, we can let it learn and summarize some patterns from data and make predictions. This is also where it differs from traditional algorithms (binary search, recursive, etc.). One has to admit that machine learning is fascinating by definition, and it seems to provide a viable framework for solving all intractable problems. It just so happens that many real-life problems are so hard that solving them with traditional algorithms is impossible.

Intro

Today I want to talk about the unpacking operators(* and **) in python.

Basic usage

We use * for numeric data types to indicate we want to do multiplication. However, we can also apply * to iterable objects¹, which means we want to unpack all the elements inside them.

📒The built-in iterable objects: list, tuple, set, and dict

Starred assignment/expression

In the release of python 3.0, it is shipped with powerful iterable unpacking operations², which is called the starred assignment/expression(or parallel assignment).

Intro

Today I’m going to talk about a new feature introduced in Python 3.8: the Walrus operator（:=）, which is a much-debated feature, but it’s finally passed and released 🤔

In Python, an assignment statement (=) is not an expression but a statement. Walrus operator is expression though. The difference between statement and expression can be simply understood as: expression always returns a value, while statement does not return a value.

The difference between the two can be seen in the following code:

Intro

Today I want to talk about the new feature bring in Python 3.10 – Pattern matching 🎉

Those who have learned C language must be familiar with the following switch statement:

Intro

If you’ve been using dynamic languages like Python, Javascript, etc., you probably won’t notice the difference between stack and heap. Because these languages have garbage collectors (GCs) that will automatically manage memory for you, you just need to program at a high level without considering the details. The bad news is that GC is not a cost-free design. No matter how well designed a GC algorithm is, the performance of the code will be degraded to some extent. If you have been in programming for a long time, you may have heard something like “the recursion explodes the stack”. You may or may not click on the search engine to understand the difference between stack and heap. Chances are you just clicked into this article :)

Intro

In the process of learning Vim, the most enlightening sentence for me is :

Vim is a kind of programming language

I once tried to learn Vim a long time ago. However, the keystroke combinations are very difficult to memorize in my opinion. So I later gave up learning Vim and switched to a normal text editor. My view of Vim changed when I took this course: Missing semester. I started to regard Vim as a kind of programming language rather than just a text editor. Then I find that the combinations of various operations of Vim are the syntax of programming language.

What are the semantic actions

When the parser processes the input code, it not only determines whether the syntax is correct but also performs some useful actions. These actions are called semantic actions. In fact, it is a piece of code, which is generally embedded in the rules of the grammar file. Then when the parser applies this specific rule, the code you set will be executed. From another perspective, semantic actions are actually “triggers”, and the trigger condition is that the parser applies the corresponding rules.

Intro

Today I coded the Leetcode 169. Majority Element again. I vaguely remember what the optimal solution is called Boyer-Moore Majority Voting Algorithm. However, I have no idea what is except for its name. So I plan to systematically learn the principle of this algorithm and summarize it to write this blog. I once heard that:

If you want to master something, teach it :)

So, I’m here today to share this algorithm with you, and try to teach you this method in plain language, so let’s get started :)