Intro

I like Python’s tuple, which allows us to quickly bundle together values of different types as a single entity and manage them in an intuitive and easy-to-use way. However, I find that once the tuple has many fields, I’m forced to add a comment to indicate the meaning of each field. For example,

t = (3, 4, 3.5)  # (x, y, value)

Then, I can jump to this definition and check the comment to see each field’s meaning. But indeed, this brings a lot of inconvenience, once the code becomes longer it’s not easy to locate that specific line of code.

Intro

Recently I started learning the GitHub Actions, a feature provided by GitHub that can be used to automate a series of steps. In the process of software development, the most common use of this feature may be the building process. For static-typed programming languages such as C/C++, we are usually required to write the build scripts. The build process involves environment preparation, dependencies download, and build execution. However, automating software builds with GitHub Actions is not the focus of this post. As I was learning this feature, I thought about how I could put it into practice and realized I could use it to automate the build and deployment of my Hugo blog :)

Tail call & Tail-recursion

What’s LoRA

Since the era of LLM(large language model) arrived, fine-tuning LLM has become a challenge because the LLM models are extremely large, making it difficult to perform full fine-tuning. There are mainly two approaches: freeze the entire LLM and perform prompt tuning or In-context Learning; freeze the entire LLM but inserting trainable modules. Today, I will introduce the LoRA(Low-Rank Adaptation), which corresponds to the latter technical approach. This is a work proposed by the Microsoft team¹

Intro

Occasionally, you may want to get the next/prev lexicographical permutation of a sequence. How would you do that? If you are a C++ programmer, you are probably familiar with the next_permutation¹ and prev_permutation² APIs. However, Python does not provide the counterparts. So the topic today is how to do this in Python. Since the solutions of prev lexicographical permutation and the next lexicographical permutation are very similar, let us focus on the next lexicographical permutation problem.

What is the bag-of-word model?

In NLP, we need to represent each document as a vector because machine learning can only accept input as numbers. That is, we want to find a magic function that: $$ f(\text{document}) = vector $$

Today’s topic is bag-of-word(BoW) model, which can transform a document into a vector representation.

💡 Although the BoW model is outdated in 2023, I still encourage you to learn from the history and think about some essential problems:

Intro

You may have difficulties when trying to calculate the partial derivatives in machine learning like me. Even though I found a good reference cookbook that could be used to derive the gradients, I still got confused. Today, I want to share a practical technique I recently learned from this video: when calculating partial derivatives in machine learning, you can treat everything as if it were a scalar and then make the shapes match