My stream of consciousness on the web and how I approached building kennyvh.com...

Building personal websites

The web is home to millions of websites. According to [internetlivestats](https://www.internetlivestats.com/total-number-of-websites/), the very first website went live in August 1991 and now there are almost ***2 billion*** websites online at the time of writing this post.

![total-number-websites](/images/uploads/total-number-websites.png)

I wrote a short python script to scrape the internetlivestats and plot the data below:

````python
# %% [markdown]
# this script reads the total number of websites from
# https://www.internetlivestats.com/total-number-of-websites/
# and plots it.
#
# to run it, you will need to install matplotlib, pandas, and lxml. using `pip`
# you can:
#
# ```
# pip install matplotlib pandas lxml
# ```
#
# %%
from datetime import datetime

import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

# %%
# NOTE: lxml does not accept HTTPS
URL = "http://www.internetlivestats.com/total-number-of-websites/"
df = pd.read_html(URL)[0]

# %%
df["Year (June)"] = df["Year (June)"].str.extract(r"(\d{4})")
df["Websites"] = df["Websites"].str.extract(r"(\d*)")

df = df.dropna(axis=0, subset=["Year (June)", "Websites"])

df["Year (June)"] = pd.to_numeric(df["Year (June)"]).astype(int)
df["Websites"] = pd.to_numeric(df["Websites"]).astype(int)
df = df.sort_values("Year (June)").reset_index()

# %%
plt.style.use("dark_background")
loc = mpl.ticker.MultipleLocator(base=3.0)

ax = sns.barplot(x="Year (June)", y="Websites", data=df)

ax.set(ylabel="Number of websites", xlabel="Year")
ax.xaxis.set_major_locator(loc)
ax.get_yaxis().set_major_formatter(plt.matplotlib.ticker.StrMethodFormatter("{x:,.0f}"))
ax.yaxis.grid(color="gray", linestyle="dashed")
ax.set_axisbelow(True)

now = datetime.now().strftime("%Y/%m/%d")
plt.suptitle(f"Total number of websites by year ({now})", fontsize=16)
plt.tight_layout()
plt.savefig("total-number-of-websites.png")

# %%
````

The thing that I love the most about the internet is how it allows people to share and publish information that is accessible to anyone in the world with an internet connection. Having a personal website is like having your own little corner of the internet to share with whomever you wish.

However, most people I know can count the number of websites they visit daily on both hands. Why is that? Unfortunately, the internet has been reduced down to a handful of big sites that are quite literally designed to [drive attention and create addictive experiences](https://blog.mozilla.org/internetcitizen/2018/07/27/attention-addiction/).

## kennyvh.com

I made <https://www.kennyvh.com/> for a few reasons:

* to carve out my own personal corner of the internet
* to experiment and play around on the web
* to practice reflecting and writing

So far, it has been a blast to do so. I've probably rewritten my site like 3-4x by now and I've learned a lot and had fun each time. Highly recommend.

### Technical decisions

For now, I've settled with Vercel's, [Next.JS](https://nextjs.org/). Prior to this, I've used plain HTML, CSS, and JS, I've used [create-react-app](https://reactjs.org/docs/create-a-new-react-app.html), and I've tried [GatsbyJS](https://www.gatsbyjs.com/) as well.

There were a few requirements I had in mind when building my site:

#### Technology is fun

I wanted to use fun technology to build my site (who doesn't like fun stuff?). There are a *ton* of awesome things going on in the web world that I'm super excited about. React is awesome and Next.JS is seriously the piece that the React ecosystem was missing IMO. The tooling and developer experience (DX) is unmatched. This is such a refreshing breath of fresh air coming from the embedded world where the DX is basically non-existent and the ecosystem is dominated by sh*tty vendor-specific tools.

I'm particularly excited to be using NextJS since it seems like they're pioneering some awesome things such as [React server components](https://vercel.com/blog/everything-about-react-server-components). IMO Gatsby was awesome as well, however I felt like using GraphQL for shuffling data around was a little hacky and unintuitive.

#### Needs to be fast

I initially went with full vanilla HTML, CSS, & Javascript because I wanted the site to be fast, responsive on mobile, lightweight, and I wanted my site to work without JS. I was vehemently against these "fad" frameworks.

However, after giving Gatsby a try, I was amazed by the DX. I was able to get a site up and running in <10 minutes just by following their docs, that's how great and easy it was. I quickly found that modern front-end tooling enables you to do so much with so little effort. You get static site generation, lazy loading images, routing, etc. for almost *free*. I scored pretty much the same in the [Google lighthouse score](https://github.com/hkennyv/kennyvh.com/tree/gatsby-stop) using Gatsby as I did with vanilla HTML, CSS, and JS. Amazing.

#### Needs to be low maintenance

This is kind of ironic in a post talking about front-end frameworks. It is basically a meme at this point where everyone jokes about a new javascript framework coming out every week. I used to be in this camp as well, but I've come to accept and embrace the rapid innovation happening in the front-end space. There's a time and a place for old and boring tech and similarly there's a place for the amazing stuff that is happening in the front-end space.

Honestly though, the principles behind Gatsby and NextJS more or less just build static HTML (there is SSR as well, which is a different story) and deploy it. Simple and easy. You can easily drop some static HTML behind apache, nginx, or whatever your webserver of choice is. However, with the amazing (and free) services like [Netlify](https://www.netlify.com/), [Vercel](https://vercel.com/), [Cloudflare pages](https://pages.cloudflare.com/), or even [Github pages](https://pages.github.com/), CI and deployment couldn't be easier.

I'm sure this won't be the last time I rewrite my website. This process can sound like quite a lot, but I would highly encourage anyone who is interested in web technology to explore and experiment with all of the awesome stuff going on in this space.

Feel free to checkout the source for my site [here](https://github.com/hkennyv/kennyvh.com).

Cheers,

Kenny

Training GPT2 on >40k of my own facebook messages...

having fun with with GPT2

So this weekend, I finally found the time to sit down and play around a little bit with GPT-2 and work on making my own chat bot based on over 40K messages from my facebook messenger. The ultimate goal is to incorporate my bot into some kind of API or twitter bot.

All my work is done in this [repository](https://github.com/hkennyv/meowbot).

## GPT-2

For those who aren't familiar with GPT-2, it is a state-of-the-art, large-scale, unsupervised [language model](https://openai.com/blog/better-language-models/) made by [OpenAI](https://github.com/openai/gpt-2). It is capable of generating coherent text paragraphs\*, rudimentary reading comprehension, and more!

\* most of the time ;) 

#### GPT-2 Use Cases

There are a lot of use cases for a language model such as GPT-2 such as:

- summarizing large bodies of text
- creating a customer support chat bot (one that doesn't suck)
- creating a chat-bot that converses with people (could help fight against depression & loneliness)
- ...and tons more!

However, one of the things I learned in like 7th grade science class was "with great power comes great responsibility". This is particularly true when considering such a powerful modern technological advancement. We need to evaluate the question "Should we be doing this?" in addition to "Can we do this?". So, considering the ethics of such a technology, we can clearly see what kind issues may arise as a result:

- generating misleading information (IMO, one of the most important issues that has been significantly magnified by the popularity of social media)
- impersonation of other people (GPT-2, combined with deep-faked videos are terrifying but are probably already happening)
- invasion of privacy - _\*puts on tin-foil hat\*_ - a _really_ good nlp model, of a person for example, could potentially reveal a lot about a person that they aren't comfortable sharing. how scary is it that a bot can possibly know more about you, than _you_ just from analyzing the sentiments in your language and speech.

If any of this really interests you, there's a whole subdiscipline dedicated to it called [natural language processing](https://en.wikipedia.org/wiki/Natural_language_processing) (or NLP for short). Check it out!

## Creating my own chat bot

I'm sure that I am barely scratching the surface of the capabilities of this bot, but I've been wanting to play around with it for a while and have come up with the novel idea of creating a bot that talks like one of my friends.

#### The method

In order to do this, I will be building off of the pre-trained GPT2-small model and training it on about 40k of my own facebook messages using a technique called [transfer learning](https://en.wikipedia.org/wiki/Transfer_learning).

Transfer learning is a technique that allows you to overcome some of the largest barriers to accessible deep learning: having a ton of data and having a ton of compute (GPT-2 was trained on >8 million web pages and costs [roughly $50K USD to train](https://blog.usejournal.com/opengpt-2-we-replicated-gpt-2-because-you-can-too-45e34e6d36dc)). Specifically, I'll be fine-tuning the model using facebook messages.

I can highly recommend using [Google colab](https://colab.research.google.com/notebooks/intro.ipynb#recent=true) for the fine-tuning process. It's free and I was able to train quite a bit using a VM with a Nvidia Tesla T4 GPU. I even almost considered purchasing colab pro for a month... oh well, maybe in the future.

The full method with more instruction is available on my [repository](https://github.com/hkennyv/meowbot).

#### Results

The results were actually super funny. I trained about 4 models based on 3 different people so far with various success (all in one day). It took me about 30-60 minutes per model to train using the Tesla T4 on colab.

Here's the results from a model where I gave it the prompt: `"I love to eat"`:

```
i love to eat healthier by using low calories..*sigh*don't eat meat and dairy products ths not?& the doctor has given us bad info & misinformed people the wrong info%& and our bodies are constantly attacked by undeserved fat even though we do have health problems & we have terrible immune issues? :'(

==========
i love to eat these burritos once a month :'(

==========
i love to eat dif,

==========
i love to eat with ya mf...

==========
i love to eat breakfast with john & billy :'(
```

What really stuck out to me about this model is that it 1) captured some names (changed them to keep anonymity), 2) it captured some abbreviations that I've used when speaking with this friend. "mf" is short for "my friend" :P, and 3) it even uses slang like "eat with ya".

Here's another one on one of my poorer performing models:

```
I love to eat you know
can I skip just desserts right?

==========
I love to see
 - it's my first time building nanoblocks baha

==========
I love to sing
And can't sing Korean songs right?
 (You got me all the tickets??
```

This one produces a lot of garbage, likely because GPT2 is more geared for larger paragraphs and the facebook messages I trained it with were probably, on average, less than 10-15 words per message. Still funny, nonetheless...

And this one was probably the most interesting (and slightly unsettling):

```
my deepest, darkest secret is that of my life, because I couldn't even risk it.
Not even in high-school, when my family was wiped out by hurricane harvey, my brothers, me, and even the music we made together would have all disappeared.
I couldn't imagine being in a position to change it, even if it was a sign of weakness.

==========
my deepest, darkest secret is that I only slept 3 hours last night - 
and this morning I woke up completely.
I would have never imagined I had the power to do anything with my body. 

==========
my deepest, darkest secret is to admit it to your bud, even if it's ur deepest desire.
It's not uncommon to find libido issues after observing someone in a private area. 

==========
my deepest, darkest secret is that of my friend, John. 
Even now, as I sit here in his hall, wondering where I might have gone so long without you. 
But I am a man ready to shed himself of you.
You must have a new friend, someone to save me, someone to listen to my deepest thoughts,
a kindred spirit to understand my condition, and a responsibility to live up to your principles.
Even now, as I sit here in your hall, you mustering up a bouquet of hope for our brothers and friends in need.
But all I can say is I am a man ready to shed himself of you.
I cannot take the lashes off my lashes, and drag my humanity out of you.
You must share in my every act with as much will power as you can get.  
Even now, as you say to me, you must hide in plain sight from me.  
Even now, as you joke with me about raising the price of tobacco products,
and say that you would have done more if there was a tax on tobacco products, yet you would have done more with less.
Even now, as you and I laugh at the irony in front of my face, think about how we
==========
my deepest, darkest secret is that I never realized that I had the power to make myself happy or even
happy with myself. That's what made me feel abandoned and unwanted all along.
```

This model was trained on a much larger set of messages (~30k messages). It's very amusing because I can definitely hear my friend's tone and see some of his sentiments baked into these responses. That last one though... _yikes_, hits a little too close to home lol.

## Conclusion

All-in-all, it was a really productive day and I learned quite a bit. It was really fun to get my hands dirty and building a model. I think I'll continue to play around with it and figure out where to deploy some of these :)


 



Benchmarking the M1

In my last post, I talked about [upgrading my laptop](https://www.kennyvh.com/posts/upgrading-my-laptop). I'm about a month in and, so far, I'm extremely satisfied with it and would recommend it to *most* people.

I used the qualifier "most", because although this machine has worked delightfully for me, there are a few things that could make it less than ideal for some use-cases that I won't fully go into detail here.

## My use-case

I'll be focusing on my use-case in this post, so what do I actually use my laptop for?

* web browsing
* checking emails
* discord
* recently, [TFT](https://teamfighttactics.leagueoflegends.com/)
* dj'ing
* blogging
* development & programming

Well, most of these things work out of the box. Some of these things work *to some extent*. Luckily for me, none of them don't work at all (although the process to get them set up is less than ideal for many people).

### Surfing the web

Web browsing for the most part, is *extremely* fast. My M1 MBA scores a higher [browser bench](https://browserbench.org/) than my desktop (which I will show in the benchmark section below). In fact, due to the M1 using the ARM architecture, there seems to be an instruction in the ARM ISA that [accelerates javascript](https://news.ycombinator.com/item?id=25233554). This is really great actually since a lot of tools these days offer an SPA version in the browser (not saying this is great, but this is the way things seem to be moving). MS office offers both a web version as well as an ARM native version, so no qualms there either.

### Gaming

I don't play too many games nowadays, I'm mostly in it so I can talk to my friends and hangout in the voice chat. Discord works without a hitch on my M1, which is mostly expected since it's built on top of [electron](https://www.electronjs.org/). TFT also runs quite well on the M1, which is a pleasant surprise as well as some other games like [tabletop simulator](https://store.steampowered.com/app/286160/Tabletop_Simulator/). I have some benchmarks posted in the [benchmarks](#benchmarks) section.

### DJ'ing

I've also recently picked up a [DDJ-400 controller](https://www.pioneerdj.com/en-us/product/controller/ddj-400/black/overview/) that has been a blast to play around with. It uses [rekordbox](https://rekordbox.com/en/), which also works without a hitch on my M1.

### Development

Lastly, programming and developing things is where the most friction is with the M1 (as expected when transitioning to a new architecture). So, depending on what kind of development you do, the M1 may or may not be ready for you yet. I do a broad range of development that is quite literally, "full stack" from embedded pcb design (using KiCAD) to writing embedded software (c/c++/rust) and a good amount of web development too (python/js).

Again, some of these things work out of the box. Some of them require a bit of effort to get setup. In my experience:

At the time, when I set up brew, they were recommending to run it through Rosetta. I didn't want to install and run all my packages in x86 only to reinstall them again when an ARM port came out, so I went ahead and went with the not-recommended native install. This went mostly smoothly, the main difference was manualling installing brew to `/opt/homebrew/` instead of `/usr/local` like on x86 mac os.

From there, when setting up a new machine, I normally just clone my [dotfiles](https://github.com/hkennyv/dotfiles) and run `brew bundle --file $HOME/.khuynh/Brewfile` and it installs all of my casks & formulae.

The rust toolchain didn't work out of the box, but it wasn't too difficult to get setup with. The "rustup-init" brew formulae failed to install, so I simply installed it manually and installed the nightly rust toolchain, which had M1 support. Easy!

I haven't done too much C/C++ development on my laptop yet, but I don't anticipate it to be too difficult. I've been mostly playing around with some ESP32 and STM32 dev boards I have. For these, there are many toolchain options, so I'm not too worried. I use PIC at work and expect there will be some difficulty using proprietary vendor toolchains though.

The javascript/node.js environment and toolchain installed without any issues through brew. I normally use VSCode insiders, so they had released an ARM native version already, so I was good to go!

Lastly, I do mostly data engineering stuff in Python, so the numpy/pandas/scipy stack is important to me. Python itself installed without an issue but problems arise when trying to install some of the scientific packages such as numpy and tensorflow. Since these heavily rely on clibs such as openblas, it took a lot of work to get these to install properly. It's still annoying to do since I make heavy use of virtualenvs, I have to build numpy from source and manually set the `OPENBLAS`env var to point to my installation each time and even then pandas and matplotlib have some issues working with it. I haven't even tried to do any deep learning stuff since I expect even more dependency issues with it. So, unfortunately, I would not recommend someone who uses the scientific python stack heavily to snag an M1 just yet. There are some key issues, however, that you can probably track to see when these issue are resolved: [numpy#18160](https://github.com/numpy/numpy/issues/18160), [numpy#17807](https://github.com/numpy/numpy/issues/17807), [scipy#13409](https://github.com/scipy/scipy/issues/13409).

**EDIT:** For anyone reading this currently, I've been using most of the python scientific stack natively with no issues (haven't tried the deep learning frameworks yet, mostly skl). What I've found to work is the following (assuming you have `brew` installed):

**Installing system dependencies**

```bash
brew install openblas
```

**Installing packages (order matters)**

```bash
pip install cython pybind11
OPENBLAS="$(brew --prefix openblas)" pip install --no-use-pep517 numpy
OPENBLAS=$(brew --prefix openblas) CFLAGS="-falign-functions=8 ${CFLAGS}" pip install --no-use-pep517 scipy
pip install matplotlib pandas scikit-learn
```

## Benchmarks

That being said, let's see some numbers. I'll be comparing 3 machines:

* 2020 M1 Macbook Air 13" 16GB
* mid-2012 Macbook Pro 13" i5 16GB
* Ryzen 2600X, RX580, 24GB (hackintosh + windows)

The benchmarks I'll be running are designed to be more-or-less things that I would *actually* do. So, with that being said, the benchmarks will be:

* cinebench r23 (single core and multi core)
* compiling sqlite (single-job and multi-job)
* running one of my crappy advent of code (AOC) rust programs
* TFT FPS

In addition, I'm publishing all of my benchmarking data in a [gist](https://gist.github.com/hkennyv/0f041cb93552d74a5a546ceb585e1bc7). You can also view the analysis for this [here](https://github.com/hkennyv/kennyvh.com/blob/main/content/notebooks/benchmarking-the-m1.ipynb).

### Cinebench R23 benchmark

![benchmark-cinebench-sc](/images/uploads/benchmarks-cinebench-sc.png "cinebench single core benchmark")

![benchmark-cinebench-mc](/images/uploads/benchmarks-cinebench-mc.png "cinebench multi core benchmark")

### Compiling sqlite benchmark

![benchmark-sqlite1](/images/uploads/benchmarks-sqlite-1.png "sqlite single-job benchmark")

![benchmark-sqlite2](/images/uploads/benchmarks-sqlite-2.png "sqlite multi-job benchmark")

### Rust AOC benchmark

`TODO`

### TFT benchmark

`TODO`

Upgrading my laptop


#### Out with the old...

The time has finally come to retire my beloved daily driver laptop, my
**Mid-2012 Apple Macbook Pro i5 13"**. This laptop was by far the best laptop I've
owned and the best part is, I bought it in 2016 for \$300 from an old coworker!
This was a popular model since it was the last model that was (mostly)
user-servicable. I took full advantage of that in upgrading the RAM to 16GB,
swapping out the 500GB HDD for a 500GB SSD, and even removing the SuperDrive
for this handy [SATA HDD tray](https://smile.amazon.com/gp/product/B0058AH2US/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1)
with another 500GB HDD. This made for a fairly snappy machine, at least for a
laptop from 2012.

Sadly, after years of use, some parts of the laptop were wearing and its
internals were starting to show their age. I started to keep an eye out for a
new laptop, some things that I was particularly interested in were:

- USB-C - an absolute must for me. I have two [Lenovo P27h-10 27" monitors](lenovo.com/us/en/accessories-and-monitors/monitors/professional/P27h-10A16270QP127inch-Monitor-HDMI/p/61AFGAR1US)
  at home and I would _love_ to utilize the unified displayport/usb-c pd
  feature. Plus, less cables is always a win for me.
- Higher resolution screen - the Mid-2012 MBP 13" is rocking a 1280x800
  resolution, and while that was probably pretty good in 2012, not having a
  1080p screen (at minimum) in 2020 kind of sucks. Not to mention, my screen
  was actually starting to give out and the left fifth of the screen would
  occasionally go all black w/ green lines
- More powerful processor - the Mid-2012 MBP boasts a dual core, 2nd gen core
  i5 processor. For most things I do, it's not even that big of a deal, but
  having a more modern CPU would be nice :p

I had also considered a [Lenovo Thinkpad T480](https://www.lenovo.com/us/en/laptops/thinkpad/thinkpad-t-series/ThinkPad-T480/p/22TP2TT4800)
for the excellent linux support, decent build quality, and RAM and storage
expandability. However, I decided to stick inside the Apple ecosystem in the
end.

#### In with the new...

The new kid on the block is the Apple's [M1](https://en.wikipedia.org/wiki/Apple_M1)
SOC. Funny enough, I actually had _just_ purchased an Intel 2020 Macbook Pro
i5 13 at the beginning of November, fully aware the M1 was coming out. I had
opted for the Intel 2020 MBP because I was expecting the M1 to be fairly
lackluster and didn't want to deal with any of the first-gen transitional
issues that would come with switching to ARM.

However, after watching the keynote and seeing Apple make some extremely
bold claims in their ["One More Thing" keynote](https://www.youtube.com/watch?v=5AwdkGKmZ0I),
I was both skeptical and very interested at the same time. As an embedded
systems engineer, I'm always super excited to see some innovation in the
hardware space (one of the reasons I've disliked intel's anti-consumer practices
and stagnating products for a while and been super bullish on AMD since zen1
first released in 2016). However, I still wanted to be very cautious with
first-party benchmarks, especially such bold ones as Apple had claimed.

Not too long after, third-party benchmarks flooded youtube & I watched almost
all of them before finally returning my 2020 Intel MBP and pulling the trigger
on a 2020 M1 MBA (with the help of a _very generous_ employee discount from one
of my friends, thanks SM!).

#### Comparison

Here's a table comparing the specs and costs between the two machines:

|               | 2020 Intel MBP 13"                                                 | 2020 M1 MBA 13"                       |
| ------------- | ------------------------------------------------------------------ | ------------------------------------- |
| **Processor** | 10th gen Intel Core i5 (4c)                                        | Apple M1 CPU (8c)                     |
| **Graphics**  | Intel Iris Plus integrated graphics                                | Apple M1 SOC 8-Core GPU               |
| **Storage**   | 512GB SSD                                                          | 512GB SSD                             |
| **RAM**       | 16GB 3733MHz LPDDR4X                                               | 16GB Unified Memory                   |
| **I/O**       | 4 Thunderbolt ports                                                | 2 Thunderbolt / USB 4 ports           |
| **Price\***   | **\$1799.00** (although I purchased an open-box one for \$1349.00) | **\$1449.00** (\$1091.00 w/ discount) |

\* before tax prices

So, in the end, I spent ~\$300 less than I intended to, yay! And for what, I
would argue, is a superior machine (for my needs). So far though, I am happy
with my purchase and will try to continue to write about my experiences with
it.

I think I will do another post with some benchmarks of this new machine and
compare it to what I was getting with my old Mid-2012 MBP for funsies.

Cheers!


Obligatory first post


# welcome to my blog

I've long blogged privately (and will continue to do so), but would love to
have a forum to do some public blogging as well (more formally than twitter at
least...).

I haven't decided yet if I want this to be a strictly technical blog or if I
want to do some personal blogging on it as well. I'm learning towards the latter
where I just dump a bunch of crap here.

I think i've changed my website at least 4-5x in the past year, so it's always
going to be a WIP.

That being said, it's a great excuse for me to play around with new
technologies. I've really been enjoying the serverless stuff lately.
There's something super satisfying about pushing code and watching your build
finish on netlify or vercel. Throw in a headless CMS, and it's almost magic
that you can have a front-end up and a pseudo back-end up without managing
any servers at all.

cheers to writing much more!


Howdy! 👋