DEV Community: Josh Hawkins

Using Hubot to Mention Everyone in GroupMe

Josh Hawkins — Sat, 28 Sep 2019 01:07:18 +0000

this post was originally written on 2016/10/11 for my personal blog on hawkins.is

hawkins / groupme-at-all

A Hubot chat bot to mention every user in a GroupMe channel all at once.

Unofficial GroupMe @all

@all is a third-party GroupMe chat bot built on Hubot. It was configured to be deployed on Heroku to get you up and running as quick as possible.

This project is not sponsored by, endorsed by, or managed by GroupMe. This is entirely a third-party project and is 100% open sourced. Use at your own risk.

For questions or concerns, please contact the repository owner, not GroupMe.

Features

Mention (tag) everyone in your group at once!

Blacklist users who don't need notifications.

Deploy to heroku easily!

Using

In practice, users can simply write @all anywhere in their message to tag everyone in the group. The bot will repeat their message and tag everyone in the group in their repeated message.

You can also control a blacklist/whitelist via chat commands as well. This is a bit more advanced, so please review the source code yourself if you're…

View on GitHub

If you're like me, you love @everyone in Slack, and your group desperately needs that feature.

Maybe there are free cookies on the Quad on campus; you've gotta tell your friends! Why can't you just say "@all of you get out to the quad for free cookies!"?!

After following along with this blog, you can!

How it works

We'll be writing a Hubot script using the hubot-groupme adapter to deploy on Heroku that will listen for messages with "@all" in them in a given GroupMe room. We'll deploy our bot for free* on Heroku.

Free for the first 550 (or 1000 if you verify a credit card) hours per month

Then, we can tag everyone in the group (minus a blacklist) by saying anything with "@all" in the message.

Note: You can host the bot on your own server instead, but I'm writing this for anyone to be able to handle. If you know how to host bots / scripts on your own machine, you shouldn't have any trouble translating the few Heroku-specific instructions. Moving on...

How it works (technical)

If you're the type of guy that throws the manuals away when you buy a product, scroll past this section.

Hubot uses regular expressions to match messages in a chat room. Groupme-at-all listens for /.*@all.*/i to act. It will construct a message to send in an HTTP POST request to the GroupMe v3 API.

Note: GroupMe's API is not well-documented. Multiple features are completely or partially ignored. In fact, mentions are not documented at all! So tagging, as far as I can determine, is handled similarly to an image or emoji attachment, which maps a User ID to a string of characters, which appear as bold text to users.

So after receiving the command, the bot maps every user's ID to an individual character in the message for a tag, since some GroupMe versions seem to require a tag of length >= 1.

Next, the bot stringify's the message object and sends the HTTP POST request, logging the results.

Check out the actual code used to do all this:

robot.hear /(.*)@all(.*)/i, (res) ->
  """@all command"""
  text = res.match[0]
  users = robot.brain.users()
  if text.length < users.length
    text = "Please check the GroupMe, everyone."
  message =
    'text': text,
    'bot_id': bot_id,
    'attachments': [
      "loci": [],
      "type": "mentions",
      "user_ids": []
    ]
  i = 0
  for user, values of users
    if user in blacklist
      continue
    message.attachments[0].loci.push([i, i+1])
    message.attachments[0].user_ids.push(user)
    i += 1
  json = JSON.stringify(message)
  options =
    agent: false
    host: "api.groupme.com"
    path: "/v3/bots/post"
    port: 443
    method: "POST"
    headers:
      'Content-Length': json.length
      'Content-Type': 'application/json'
      'X-Access-Token': token
  req = https.request options, (response) ->
    data = ''
    response.on 'data', (chunk) -> data += chunk
    response.on 'end', ->
      console.log "[GROUPME RESPONSE] #{response.statusCode} #{data}"
  req.end(json)

How to install groupme-at-all

Installing this is easy, but there's a few steps...

Create a GroupMe Bot

GroupMe has native bot integration - they're kind of just an API token for your scripts to use, but that's all Hubot needs.

Navigate to the GroupMe dev site and log in with your GroupMe account
Click create a new bot
Choose the group your bot will live in
Name your bot (something like "All" will do)
Provide a callback URL (doesn't really matter for us, but has to be unique)
Provide a URL for an image for your bot to use (my favorite is All brand laundry detergent)
Click submit

Now you'll be lead to a page with your GroupMe bot's ID, ROOM_ID, and TOKEN. Copy these guys into a note, you'll need them again in a moment.

Setup Hubot

Follow the instructions at Hubot Getting Started to install and wrap your head around Hubot. Just remember to use the adapter groupme.

It's easy, so just get the general idea after you install it.

Setup @all

Now you're ready for the fun stuff! I've already written the code for you, so all you need to do is...

Fork my groupme-at-all repository to your own GitHub account. (Bonus points if you "Star" it ;) )
Log in (or create an account) to Heroku
Create a new app on Heroku
Choose to deploy from your GitHub and select the repo
Configure environment variables (see below)

Configure Environment Variables

Hubot is centered around environment variables for accessing sensitive information.

Load your app's settings and click "Reveal Config Vars"
Add the config variables and the appropriate values from your GroupMe Bot you made earlier:
- HUBOT_GROUPME_BOT_ID
- HUBOT_GROUPME_ROOM_ID
- HUBOT_GROUPME_TOKEN

Test Your Bot

At this point, you should be good to go!

To start the bot, start your Heroku app (or run ./bin/hubot -a groupme from your repo's root directory if you're not using Heroku).

Give it a moment to start up, and then use your regular GroupMe account (phone, desktop, etc) to say "Testing @all!" in your group. You should see your bot reply "Testing @all" in bold text since it tagged everyone in your group!

Success!

Hooray!! You can finally tell EVERYONE about Free Chips & Queso day at Moe's!

Your bot's good to go now, but there's some optional configuration you may prefer to do still, such as keep your bot awake.

(Optional) Further Config

Here's what else the bot can do:

Set up a ping bot to keep your Heroku instance alive
Handle a blacklist to not mention specific users by ID

Begone, sleep!

Speaking of writing blog posts at 2am... your bot would love to be an insomniac. Otherwise, he'll fall asleep after only an hour only to be awoken manually later. So, if you're using Heroku, you've got a few options...

Pay Heroku for a better plan. If you plan to do some serious work here, I'd suggest this. If it's just for your Sunday Brunch club, maybe not.
Or set up a ping site like Pingdom or Uptime Robot to hit your Heroku App's URL periodically. This is the easiest option if you don't want to code anything.
Or set up heroku-keep-alive. I've had some issues with this one, but some people like it!
Or finally make a simple setInterval in your app. Something as easy as var http = require("http"); setInterval(() => (http.get("http://your-app-name-goes-here.herokuapp.com"), 300000)); to ping your Heroku app every 5 minutes will do.

Remember, if you're on a free Heroku plan, you're limited to 550 free hours, or 1000 if you verify a credit card. If you host this bad boy on your bot already should be a fully-functioning insomniac. Congrats!

Blacklist

Groupme-at-all uses a redis server controlled by Hubot to keep a persistent blacklist. This can be a tricky to get set up, so I'll leave this mostly as an excercise for you to understand.

In a nutshell tho, here's what you'll do...

Create a redis server
Configure hubot to use this server
Say "blacklist @so-and-so" to blacklist "so-and-so" in GroupMe
Cry that people don't want to be notified about free food trucks outside the office

But, you don't exactly need a redis server. The only issue is, that without it, your blacklist will reset every time your bot sleeps. That's pretty useless, but if your bot never sleeps then spend this time writing your own post on how computers are everything a college student ever dreamed of being in finals week.

Leveraging Travis-CI for Continuous Deployment to Publish Compiled Binaries to GitHub

Josh Hawkins — Thu, 25 Apr 2019 03:02:32 +0000

Recently I wrote a binary called "Watchdog" in Rust on my Mac that I wanted to take with me to Linux systems I frequent, but I couldn't bring a Rust compiler to these systems for unrelated reasons. This meant I had to cross-compile my application and ship just the binary.

Now, I'm taking a few liberties in this post to showcase this as a proof-of-concept, but most notably of these is that I am not handling dynamically-linked libraries. I'm shipping a standalone binary, that is to say the one file I build has everything it needs to run, assuming you run it on the correct platform. Or, so I naively hope - it can definitely be more complicated than that.

What is cross-compilation?

Cross-compilation is compiling your code on one platform (for me, Mac OS) to run on another platform (for me, Linux). If you're more familiar with languages like Ruby, JavaScript, and Python, this might not make a lot of sense since your code already runs on different platforms most likely. But for compiled languages like Rust, C++, C, etc., we have to compile it to code that can run on certain platforms. That's what we're doing here, we're compiling my Rust code on my Mac to run on another computer that runs Linux.

You can configure cross-compilation on your local computer, but it can be difficult, and often has some big quirks. I've dealt with this recently when cross-compiling a lambda function in rust for mac->linux. In particular, getting OpenSSL to behave was an absolute nightmare and killed my motivation for months. Eventually, I kicked the problem to the curb and said I'll bring in Docker just to work around it. Your mileage may vary.

So instead of xcompiling locally, we'll leverage Travis CI to do this natively for free. Technically it's not cross-compiling, since Travis will compile it in a Linux VM, but the accomplished goal is the same - building my application for platforms I can't otherwise. So, if you'd like to chew me out for click-baiting, I welcome it - I just couldn't think of a better way to word what we're doing here!

Where Is Your Code?

You can see all the code I worked on for this post on my GitHub project!

hawkins / watchdog

⚠️ Watch filesystem for changes and then run a command

Watchdog

⚠️ Watch filesystem for changes and then run a command

Great for automatically running make test or similar commands in response to file changes.

Usage

TODO: This isn't stable yet, but here's the output for --help at 0.2:

USAGE:
    watchdog [FLAGS] [OPTIONS] <COMMAND> [-- <PATH>...]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information
    -v, --verbose    Enables verbose output

OPTIONS:
    -g, --glob <GLOB>    Glob used for matching files

ARGS:
    <COMMAND>    Command ran on response to changes
    <PATH>...    File path(s) used for matching files

Goals

Easy, inuitive way to select files to watch
- Regular expressions (#1)
- Globs
  - Both via your shell and via Rust internals, pick your poison!
- Explicit file paths
Sensible GNU make interop (#4)
Simple, out-of-the-way API
- It's a simple problem. Therefore, keep the solution simple, too, stupid.

View on GitHub

Initial Travis Config

Travis lets us configure it via code, so my initial .travis.yml file before I set up "cross-compilation" looked like this:

language: rust
rust:
- stable
- beta
- nightly
matrix:
  allow_failures:
  - rust: nightly
  fast_finish: true
os:
- linux
- osx
deploy:
  # This part's not very relevant to this blog post, we'll add a new deploy step though, so I'll keep it as an example of what you might already have
  provider: cargo
  skip_cleanup: true
  on:
    tags: true
    condition: "$TRAVIS_RUST_VERSION = stable && $TRAVIS_OS_NAME = linux"
    branch: master

If you're not familiar with Travis, I'll give you a brief summary of what this config does.

We run a "matrix" build (multiple platforms and rust versions) to test both stable rust versions and nightly builds of their rust compiler on different platforms.
Nightly builds can fail, since those compilers aren't final and may have bugs, so don't mark these build fails as an overall failure if that's all that went wrong
When I tag a release in git, publish this as a new version of the crate (rust package) in the global repository so other rust users can grab the latest version just released

Note that there's no talk of publishing binaries yet - because that's the new part we'll add today!

Travis Releases Provider

GitHub already has a sweet releases feature, and I happen to be using it for Watchdog, so I'd like to publish my built binaries there so users can find it easily if they, like me on my linux machines, can't access a Rust compiler.

Travis CI is super rad, so naturally they already have swell support for GitHub Releases! Lucky us!

To use this provider, we'll have to:

Create a new personal access token on GitHub for Travis to use to upload the binaries to your releases: https://github.com/settings/tokens
Add them to your Travis project:
- In your terminal, run these commands:
- cd <Your project>
- gem install travis (you'll of course need ruby + rubygems, which Mac OS comes with)
- travis encrypt (then follow the provided instructions)
Add the below code to your .travis.yml:

deploy:
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file: target/release/watchdog # where `watchdog` is your built rust binary
    api_token:
      secure: # this part from your `travis encrypt` command

Testing Travis Changes

I'm no Travis expert, so maybe a commenter can help us both learn a better way to test this, but here's how I test changes to my travis config:

Commit the change in git
- To save your sanity, comment out the branch: master part so you can do this on a throw-away branch
Tag the commit
Push to GitHub (remembering to push with tags, i.e. git push --tags!)

Travis will then run your new build.

Connecting the Dots

The above config won't quite work for us though, so we have to be a little more clever about what we send to GitHub.

If you added that last section like I suggested, you'll notice that target/release/watchdog doesn't exist. This is because by default, Travis builds in Debug mode (notice the absence of Rust's required --release field for release mode builds in the "Default script" setting, implying it's Debug):

We want to publish our binary in Release mode, so let's modify our yaml to build in Release mode by adding this key to the root of the yaml:

script: cargo build --verbose --release; cargo test --verbose

Sweet. Now if you tested it, you should see that Travis is a little slower (release builds take a bit more work!) but your file should be uploaded to the GitHub release! 🎉

But Wait, There's More!

Notice how we build for two platforms, but only see one binary in the release? And we can't tell what platform its built for because its just named watchdog? That's no good, if I downloaded that on my linux box, I have no idea if I'm downloading a linux binary or a mac binary.

Why is there only one?

Well, admitting that I didn't encounter this myself (so your exact error may vary) because I did this out of order, these Travis builds run out of order and concurrently, so your linux build may upload a file named "watchdog", only to be overwritten by your mac build when it uploads a different file named "watchdog". So, we have a race condition.

How can we get both binaries to upload and not overwrite each other?

Well, we can get two birds with one stone here, by also indicating in the file name whether it is built for linux or mac. The idea is we have to give the files on different builds different names so we don't overwrite them, so we might as well specify which platform they're built for to be clearer to users too.

Renaming Your Binary

This was the fun part for me, tracking down how to rename binaries so we could upload multiple to GitHub and specify their platform. It only reinforced my love for Travis when I learned that it provided all the building blocks I needed:

a before_deploy script step to run whatever I want after we run the script step yet before we run the deploy step
a collection of environment variables to tell all sorts of things about our builder, like its platform

With that in mind, we can just connect the dots to rename the binary before we upload it in the deploy step:

before_deploy:
  - mv target/release/watchdog "target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME"

But, if we want to support having multiple deploy steps (publishing to crates.io AND GitHub releases, for instance), this will actually cause an error and fail the whole build. That's because this before_deploy script gets called once for each deploy step. That actually burned me and caused my "final" build to fail, because the second time it was called, mv threw an error because target/release/watchdog didn't exist anymore. So, to account for that, you'll need a little bash-fu to update your yaml once more:

before_deploy:
 - "if [[ -f target/release/watchdog ]]; then mv target/release/watchdog \"target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME\"; fi"

What this modification does is use bash to only run mv if the file exists, so we won't error out and fail the whole Travis build if we have multiple deploy steps. Anyways....

Rad, now our binary is named something like watchdog-v0.2.0-linux! Now we can adjust our deploy step to find this new name pattern:

deploy:
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file_glob: true # <-- note this new field
    file: target/release/watchdog-*  # <-- note the `-*`
    api_token:
      secure: # this part from your `travis encrypt` command

The key here is to use globbing to find the new binary, since we can't hardcode the name anymore. We have to enable that with file_glob: true, then we can add -* to the end of our file name to find the new name pattern easily!

Testing this now should show a GitHub Release with your two platform's binaries both uploaded, finally. Way to go! 🎉

Travis	GitHub

Putting it All Together

Here's the final Travis config we used:

language: rust
rust:
- stable
- beta
- nightly
matrix:
  allow_failures:
  - rust: nightly
  fast_finish: true
os:
- linux
- osx
before_deploy:
  - mv target/release/watchdog "target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME"
deploy:
  - provider: cargo
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable && $TRAVIS_OS_NAME = linux"
      branch: master
    token:
      secure: # ;)
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file:
      - target/release/watchdog-*
    api_key:
      secure: # ;)

And that's it! Now we've got a Travis config that allows us to build and publish binaries for any platform Travis supports whenever we tag a release.

You can check out my release I cut to demonstrate this here: https://github.com/hawkins/watchdog/releases/tag/v0.2.5

Once more, you can review all this in action with a real Rust project on my GitHub. And if you have any questions, feel free to leave either a comment on this blog post, and issue on the below repo, or shoot me an email!

hawkins / watchdog

⚠️ Watch filesystem for changes and then run a command

Watchdog

⚠️ Watch filesystem for changes and then run a command

Great for automatically running make test or similar commands in response to file changes.

Usage

TODO: This isn't stable yet, but here's the output for --help at 0.2:

USAGE:
    watchdog [FLAGS] [OPTIONS] <COMMAND> [-- <PATH>...]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information
    -v, --verbose    Enables verbose output

OPTIONS:
    -g, --glob <GLOB>    Glob used for matching files

ARGS:
    <COMMAND>    Command ran on response to changes
    <PATH>...    File path(s) used for matching files

Goals

Easy, inuitive way to select files to watch
- Regular expressions (#1)
- Globs
  - Both via your shell and via Rust internals, pick your poison!
- Explicit file paths
Sensible GNU make interop (#4)
Simple, out-of-the-way API
- It's a simple problem. Therefore, keep the solution simple, too, stupid.

View on GitHub

The Story of How I wrote A CLI for Your Bookmarks using Larder.io

Josh Hawkins — Sun, 28 Oct 2018 18:06:48 +0000

The story starts when I began using Larder recently, and while I loved the mobile app and browser integrations, I really just wanted to use it from the command line too. I checked the roadmap and found that a CLI was suggested back in 2016, but there had been no other motion on that effort. A quick glance revealed they had a nice and simple API, so I popped open a terminal and got to work.

I've been writing more and more Ruby lately, and I've been loving it! I thought this would be a fun time to become a publisher of ruby gems and not just a consumer, so I set out to write the project in Ruby. (Also, I heard great things about a few key Ruby libraries that would make this effortless, but we'll get to that.) It wasn't long before I had a first commit that printed out info on a user based on a provided token from a yaml file. Not half an hour later, I had added lard folders and pretty-printed them too. I'll post pictures of the output later, but for now, you can see this on the project's readme:

hawkins / lard

📗 A third-party command line interface for larder.io

Lard

📗 A third-party command line interface for larder.io

Note: This project is a work-in-progress, and as such, may not be fully functional quite yet. Note this also means patch versions in the 0.0.z release family can be major breaking changes. Until we reach a 0.1.0, upgrade at your own risk!

Installation

To install, simply run gem install lard. Then, you can run the binary by calling lard.

You'll need to log in before you can do much, so follow instructions in lard login to get started.

Contributing

I welcome contributions of all kinds! Feel free to open an issue to discuss problems, feature requests, smoothie flavors, or code changes you'd like to make. This project is meant to be useful to every Larder user, so I'd love to hear from you!

If you want to run the code locally, follow these steps:

Clone the repo
Make…

View on GitHub

It started simply:

# lib/lard.rb
require 'httparty'
class Lard
  include HTTParty
  base_uri 'https://larder.io'
  def initialize(token)
    @options = {
      headers: {
        'Authorization' => "Token #{token}"
      }
    }
  end
  def user
    res = self.class.get '/api/1/@me/user', @options
    JSON.parse res.body, symbolize_names: true
  end
end

# ...
# bin/lard
#!/usr/bin/ruby
require 'yaml'
require './lib/lard'

config = YAML.load_file 'lard.yml'
lard = Lard.new config['token']
puts lard.user.inspect

But hold up a second - am I really calling it "Lard?" What, am I 5 years old? Well... sort of. I actually have some decent reasons though, so bear with me:

I'm not associated with Larder, so this is third party and can't use their name as my own
It's a command line app, so the less typing, the better - short names rule!
It's a Larder client first and foremost, though, so the name should be memorable
And, again, I'm five years old, so I think it's just silly enough that its other merits stand strong to make it a good name

I could tell this was going to be fun, and not too stressful, so I was sold on keeping the project and making it feature-complete with regards to what their API provided as quickly as possible. So, to make sure I wouldn't step on anyone behind Larder's toes, I shot them a tweet asking if it would be alright to open-source my work (my repo was private, at the time). // Detect dark theme var iframe = document.getElementById('tweet-1054206203839070210-604'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1054206203839070210&theme=dark" }
They very kindly agreed... // Detect dark theme var iframe = document.getElementById('tweet-1054207395268685826-90'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1054207395268685826&theme=dark" } so I was off to the races!

I wrote a little bit more code to fetch bookmarks from a folder, broke my ruby install by mucking about with rvm and rbenv and finding they had collided on my mac, and finally went to bed.

The next morning I woke up early to work on the project and fix my ruby install (exciting, right!?) and finish fetching bookmarks. Eventually I ran off to work, but when I came home, I spent all evening fixing the ruby install to compile with openssl as needed, and got work late that night since I was too hyped to sleep. I only coded an hour that night, but I got a lot done as I was able to pretty-print out an entire folder of bookmarks, paginating with the Larder API and all.

The next day, I decided it was time to quit writing my own CLI kit and adopt such a wildly popular one, Thor. It was super nice, and this was where I really started learning in this project. I had already learned how to make basic HTTP requests with net/http and HTTParty, but now I was learning a speedy and nice way to write command line interfaces in Ruby without breaking a sweat.

I also almost admitted I was doing all this on a mac by committing the .DS_Store file. 🤫

Then, I published Lard 0.0.1, the first version of the gem! Woohoo!! SPOILER ALERT: I didn't know it at the time, but gem install lard didn't bring along Lard's dependencies, so don't bother installing this particular version...

Finally I made a hackjob at posting a bookmark (which turns out you can post and edit at the same endpoint, it matches based on the URL you provide). But it didn't work yet - I was seeing an HTTP error 301: Redirection, and my POST request was getting an error back:

{"detail":"Method \"GET\" not allowed."}

But I was sending a POST request! I spent a long time debugging this. After a while I cracked open wireshark and saw that I was encountering a 301: Redirection and sending a GET request to the redirected URL. Now that was odd. I didn't know why that would happen. Normally HTTP requests follow through 300's like they weren't even there, right?

Well, not this request. It was late, I was tired, and I just wanted this to work so I could start using the CLI myself and not just developing it. I decided to get some air, went on a walk, picked up a milkshake to ruin whatever few calories I'd managed to burn on the walk. I chatted with my friend and compiler partner Cameron about the issues, and found the solution on my walk home. I was so burned out I didn't want to write the code for it though, so I opened an issue for it and went to bed.

Remove use of HTTParty #4

hawkins posted on Oct 26, 2018

Honestly, this lib has been something of a headache with its less-than-stellar docs and presumptuous api to say the least, so I've boiled the HTTP calls down to something pretty simple such that we can hopefully remove httparty pretty easily.

On top of all that, I'd like to get back support for ruby < 2, I think I can justify replacing httparty with net/http or another lib that's a bit lower level and with lower dependencies.

Concerns:

3xx codes
- If I remember right, POST https://larder.io/api/1/@me/* hit 301's to somewhere, so I need whatever interface I scrap together to follow redirects and maintain HTTP verbs throughout redirections.
Easily infer errors
- res[:error] is nice - I don't want to lose that semantic response if possible

View on GitHub

The next morning, I was cooled off and wrote the patch to fix httparty (yes, it was one stupid function call to change an option otherwise not exposed and one that seems contrary to HTTP 300 semantics). Now I don't want to trash talk this library - other than this gripe, it was super nice. But this hobby project should be fun, and I don't want headaches - I get enough of those living with chronic migraines! So just know that if you use HTTParty, that's perfectly fine, but this very minor gripe was enough to give it a bad taste for me. I have the luxury of making very simple requests in this project, so going back to net/http where I know exactly what I'm getting isn't a big deal at all.

I kept using HTTParty for now, but I knew it wouldn't be here to stay much longer. I wanted to do fun things, features, not refactoring. So later that night, wrote lard tags, lard search, and lard login.

I tweeted back to Larder now that I had something to show, and they were excited! // Detect dark theme var iframe = document.getElementById('tweet-1055299299922362370-380'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1055299299922362370&theme=dark" }

// Detect dark theme var iframe = document.getElementById('tweet-1055303873638744064-132'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1055303873638744064&theme=dark" }

That got me way too hyped, and I really needed to go to bed, so I kept tagging and publishing new versions between features hoping that would make me stop and rest, but alas:

Bump 0.0.4  … committed 4 days ago

This should've been 2 at this point but I kept thinking I'd stop and go
to bed, but here we are. A few bugs and features later, and this is it.

then only 8 minutes later:

Correct nil check  … committed 4 days ago 

I said I was done for the night, but I have no self control when it
comes to side projects

Finally after this, I published version 0.0.5 (still not installing gem dependencies with it, mind you) and went to sleep.

The gang at Larder told me the API limits folder requests to 200 items per request, so I applied that in my code as soon as I got a chance to the next evening. Then I did lots of refactoring type things to make working in the project feel less-icky, starting with error messages. Then I learned about the magic that is rubocop,and I desperately questioned how I ever wrote large scale ruby projects without it. Wow! Nicest linter I've seen in a long time. My pet language, Druid, has a lot to learn from this for it's "friendly compiler" effort...

Most excitingly, I set up Travis CI! If you're not familiar with Travis, Travis CI is a super awesome and free continuous integration platform that integrates insanely well with GitHub and practically every popular language and software packaging platform like RubyGems. Perfect for running tests, linting, and deploying updates to your package automatically when you open a PR or tag a release! Check out Lard's status page on Travis where you can see the latest builds it has ran, what triggered them, and whether tests are passing/failing!

The next day, October 26th (6 days straight working on Lard!), Josh Sharp from Larder opened an issue which caught me by surprise:

Missing dependencies? #7

joshsharp posted on Oct 27, 2018

Heya, I had a go at installing and running Lard yesterday, and I ran into some issues. I'm not a Ruby person, but I think the issue here was that its dependencies were not being installed when I installed the gem.

The first time I ran it, I got an issue about httparty missing, which I figured out how to install as a gem myself 😇 but since then I'm still getting an error:

hades:~$ lard
Traceback (most recent call last):
        6: from /home/hades/.rvm/gems/ruby-2.5.3/bin/ruby_executable_hooks:24:in `<main>'
        5: from /home/hades/.rvm/gems/ruby-2.5.3/bin/ruby_executable_hooks:24:in `eval'
        4: from /home/hades/.rvm/gems/ruby-2.5.3/bin/lard:23:in `<main>'
        3: from /home/hades/.rvm/gems/ruby-2.5.3/bin/lard:23:in `load'
        2: from /home/hades/.rvm/gems/ruby-2.5.3/gems/lard-0.0.5/bin/lard:3:in `<top (required)>'
        1: from /home/hades/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
/home/hades/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- paint (LoadError)

If I'm doing something wrong, please let me know! Otherwise hope this helps you fix an issue.

View on GitHub

Woah, one of the two devs from Larder tried my project? Awesome!! And it failed catastrophically, OOOOOPS!! I was so embarrassed, but I was excited because he'd clearly found an issue that I could learn from! That's why I love open sourcing everything I do, because someone is bound to have a different experience, and I LOVE learning from these opportunities!

Right away, I had a theory:

Comment for #7

hawkins commented on Oct 27, 2018

Hey thanks so much for trying it out and reporting an issue! 🎉

This is actually my first published ruby gem, so I'm guessing I didn't mark the Gemfile which lists our dependencies (httparty, paint, thor) to be included in the published gem or something to that affect. Oops! Will look into it tonight, thanks a bunch!

In the mean time, if you'd like to keep toying with the app, you might be able to get away with this:

gem install httparty
gem install paint
gem install thor

This will make sure you've got the dependencies installed system-wide, but really, gem install lard should have covered that, so I'll figure out how to cross the T's here. Thanks again!

View on GitHub

... and I learned two things at once:

I should do some very basic googling before responding to issues. Even if I really just want to fix the user's issue as quick as possible (with the gem install * commands I gave him), I might be able to give a more insightful and easier solution after a quick search
Sure enough, I was right about how Gem dependencies don't come from Gemfile:

Comment for #7

hawkins commented on Oct 27, 2018

I should shut my mouth more - very quick google search revealed this is a rookie gem publisher mistake! 😅 Fixed it with lard@0.0.6, so feel free to run gem uninstall lard and gem install lard to grab the latest version and its dependencies and we can pretend like this never happened. 😉

View on GitHub

So I patched it and deployed it as 0.0.6 so Josh could try again later.

Then, more refactoring. I decided to split the CLI from the lib so that the CLI is easier to work with and other packages can leverage lard programatically in the future. Pretty straight forward stuff, just turning LardHTTP from a module to a class, maintaining some state, and leveraging that library from the CLI.

What I didn't know about this was how it could work simultaneously in production and development, though. Since this is my first foray into publishing a Ruby Gem, I've never had to link libraries from the file system, I could always just say "Hey Gem, find that thing for me" and be on my way. Not this time!

I first kept require 'lard' in the code that would be deployed to production. This works, because when someone runs gem install lard, Gem will know where lard is, so require 'lard' will resolve successfully. In development, though, lard is not installed, we just have its source files. So at the time, I would manually change this line to require '../lib/lard' (or use require_relative) and change it back to require 'lard' before committing, so production code would not be looking relatively, but development code would. Gross.

After some digging, I eventually found that leaving require 'lard' in the code is the right move, but that I can run the development binary bin/lard with a command line argument to link the lard library from the source code by using ruby bin/lard -Ilib. This looks familiar to you if you know GCC, but here's how it works:

We run the ruby binary
We pass it bin/lard, a ruby file from my source code that contains the CLI code (and the infamous require 'lard' call at the top of its file)
We specify the -Ilib argument to map the lib folder (where lib/lard.rb exists) as an include folder for this runtime
Ruby processes require 'lard'
Gem is unable to resolve 'lard' from its normal folder, and sees that it is not relative
Gem finds that the lib folder was linked, though, and that it contains a lard.rb file - so this require is resolved to this file!

This is great. Now, I can just run that command during development and be perfectly comfortable to not commit broken require statements. Woohoo!

Now technically day 7, even tho it's 12 AM on a Friday night/ Saturday morning, I cleaned up the lard user command to pretty-print user details finally.

At this point, the library was technically complete in everything I wanted it to do at a minimum to be a viable command line interface for Larder!

So I celebrated by going back to sleep finally and didn't get back at it until the next evening (same day, next night). I'm a little bit nocturnal on the weekends, if you haven't noticed...

This is where I really made Travis shine - by configuring it to automatically deploy new gem versions when I tag a release in git. Yaha! (I also took some time to go back and do this to Prettier Webpack Plugin, like I should have done years ago) Hey, why not, shameless plug:

hawkins / prettier-webpack-plugin

Process your Webpack dependencies with Prettier

Prettier Webpack Plugin

Automatically process your source files with Prettier when bundling via Webpack.

How it works

This plugin reads all file dependencies in your dependency graph If a file is found with a matching extension, the file is processed by Prettier and overwritten.

You can provide options Prettier by specifying them when creating the plugin.

Looking for a loader?

It's in its early stages, but you can find a loader version of this plugin here: prettier-webpack-loader

Installation

Note, for Webpack 4 support, install prettier-webpack-plugin@1. For Webpack < 4, install prettier-webpack-plugin@0

Simply run npm install --save-dev prettier prettier-webpack-plugin or yarn add --dev prettier prettier-webpack-plugin to install.

Then, in your Webpack config files, add the lines:

var PrettierPlugin = require("prettier-webpack-plugin");

module.exports = {
  // ... config settings here ...
  plugins: [
    new PrettierPlugin()
  ],
};

Why?

Keeping style consistent between…

View on GitHub

Then, after some linter warnings, I finally got around to removing httparty. Whew! It felt great to have a little bit more control over what's happening, and to know I don't have to read the library author's mind to know how to use it anymore. Alright alright, sorry, let's move on.

Continuing the trend of "I'll sleep when it's done!", I kept on with Day #8 at midnight. The next issue I tackled was another interesting learning point for publishing Ruby Gems. Check it out:

How can we ensure the User-Agent gets updated with every new version? #11

todo[bot] posted on Oct 28, 2018

https://github.com/hawkins/lard/blob/295e5d9cf7b86de00f6376f9b2627bd11b44f9ac/lib/lard.rb#L100-L112

This issue was generated by todo based on a `TODO` comment in 295e5d9cf7b86de00f6376f9b2627bd11b44f9ac when #10 was merged. cc @hawkins.

View on GitHub

I wanted to add a User-Agent to all network traffic coming from Lard to indicate it was coming from Lard. Trouble is, like the whole require 'lard' headache up above, I didn't want to have to make code changes here to be in sync with the gem version itself manually. That would be silly! Computers are smarter than me, surely they can do this themselves. My first thought was a rake task that would complain if I didn't change it manually. So when I tag a release, Travis would find that rake screams about the release not updating the User-Agent, so I at least get forced to change it without publishing an accidentally-wrong User-Agent.

Okay, but that doesn't fix the root issue. I thought back to how gemspec's are actually written in Ruby and leverage the Gem library. So I thought "maybe Gem can tell me what version of 'lard' I'm using?"

Sure enough, it can, with Gem.loaded_specs['lard'].version. Bingo! But... wait. No, no dice. That only works in production, when gems are loaded from the local gem repository. Not in development, when they're linked virtually. Oof... well, okay. I thought I could add a constant to the library and pull that into lard.gemspec then, since gemspec's are just Ruby code after all. So I tried it... and after realizing I neglected to mark lib/lard.rb as requirable and wasting 0.0.7 as a test for this, I found that this version style would otherwise work!

In the process, I learned that RubyGems offers a pattern of pre-release versions, which are meant for shipping possibly-broken code like what I just wrote. Awesome! I just fixed two problems with one PR!

Finally, I decided it was time for proper tests. Now I love testing, especially when automated via Travis, but I've been struggling with how to test a program like this properly for a few reasons:

How do we test effectively when our library relies so heavily on external network connections?
What about our CLI, what do we test? The final binary, or the modules of it?
How do we avoid testing the CLI kit's underlying capabilities (Thor)?

After brainstorming that at 1am (always a good idea!), I opened an issue with some thoughts and got to work:

Add tests #15

hawkins posted on Oct 28, 2018

I waited a while on this because I wasn't sure how best to test it, but here's a few ideas I'd like to see about implementing:

test Lard lib functions but mock the http responses
test LardCLI functions like print_*
test bin/lard executable if I can come up with a way to put an API token on Travis.

View on GitHub

So to end out the night, I installed RSpec and wrote some basic tests to get the ball rolling.


# spec/lard_spec.rb
require 'lard'

RSpec.describe Lard, '#authorized' do
  it 'returns false to indicate user is unauthorized' do
    l = Lard.new
    expect(l.authorized).to be false
  end

  it 'returns true to indicate user has a token' do
    # Though we can't actually guarantee the token is authorized
    # without an http request
    l = Lard.new 'token'
    expect(l.authorized).to be true
  end
end

RSpec.describe Lard, '#api_url_prefix' do
  it 'returns the Larder API\'s Base URL' do
    l = Lard.new
    url = l.api_url_prefix
    expect(url).to be_an_instance_of String
    expect(url).to be_an_start_with 'https://'
  end
end

RSpec.describe Lard, '#get' do
  it 'will fail if not authorized' do
    l = Lard.new
    expect(l.authorized).to be false
    expect { l.get 'user' }.to raise_error(RuntimeError)
  end

  it 'can fetch user' do
    l = Lard.new 'token'
    expect(l.get('user')).to be_an_instance_of Hash
    expect(l.get('user')[:links]).to be_an_instance_of Integer
  end
end

# spec/spec_helper.rb
require 'webmock/rspec'
WebMock.disable_net_connect!(allow_localhost: true)

RSpec.configure do |config|
  # Handle web request mocking
  config.before(:each) do
    user_body = { 'id' => 1,
                  'username' => 'username',
                  'first_name' => 'greatest',
                  'last_name' => 'ever',
                  'timezone' => 'America/New_York',
                  'avatar' => nil,
                  'date_joined' => '2016-01-21T03:03:33Z',
                  'links' => 500,
                  'is_trial' => true }
    stub_request(:get, 'https://larder.io/api/1/@me/user/')
      .with(
        headers: {
          'Accept' => '*/*',
          'Authorization' => 'Token token',
          'Host' => 'larder.io',
          'User-Agent' => ['Lard/0.0.8', 'Ruby']
        }
      )
      .to_return(status: 200, body: user_body.to_json)
  end
# ... the rest of `rspec --init` output
end

Recap: What I learned

I learned a ton in just 8 short days of writing an MVP command line interface for bookmarks management while working full time on other projects, so it's hard to sum it all up, but I've sat here for an hour now writing a silly story no one will ever read, so I might as well sum it up so I can regale my grand-children with the escapades of my youth:

Indie devs rock! The two behind Larder, Josh Sharp and Belle Cooper made an awesome product and were super supportive through the whole process. I can't wait to make Lard as awesome as possible, even if I'm the only user, just as a tribute to Larder itself.
Language version management is hard. My mac came with Ruby installed, but it's sort of known that you shouldn't use it - so I installed rbenv some time ago. Well, I didn't remember that I had Ruby installed on my mac when I started this, so I grabbed rvm this time. Whoops! That put me in quite a pickle, but I learned how rvm links Ruby and its dependent libraries (having a non-system dependent OpenSSL being an issue for me) and managed to get that solved.
Gemfile != gemspec. Almost every Ruby project I've used had a Gemfile that installed everything you need. This was my first Ruby gem I wrote myself, so I had to learn how to specify runtime dependencies on the Gem itself, as well as development dependencies that can belong in the Gemfile. Still a little unclear as to how to reduce duplication between them, but that's a story for another day.
Rubocop rules. I've always loved linters, especially auto-formatters like Prettier, but Rubocop is the best of both worlds in my opinion.
Don't reach for the libraries until you know you need them. I generally don't, but in hobby projects like this, sometimes I reach for the big guns just to move quickly so I can have fun and not do drudge-work. This time, that gave me a big headache, so I've got more ammo to support rolling my own infrastructure on the next project. (Like we're doing over at Druid!)
Ruby Gems are super smart. From linking development dependencies, making your gemfile just plain Ruby so you can avoid repeating yourself by importing some constants from your code to your spec, I was super pleased with working with Ruby Gems all around. Writing them and publishing them is a breeze! Especially with Travis.
Oh, Travis CI freaking rules. I already knew this, but setting up auto deployment was the real hot stuff on it this time.
Don't over-think your testing struggles. I could have set up testing way sooner, mocking HTTP requests is very much a solved problem when it comes to ruby testing, as my brief time with RSpec and Webmock showed last night. I wish I had set up testing sooner, but I had a lot to learn and iterate on so I put it off too long.

What's next for Lard?

Well, first of all, TESTS! I'm absolutely getting proper testing in place as soon as I sit down to code again on Lard. But then, exciting new features:

Improving lard bookmark to make it more dynamic and flexible to how you want to describe or change your bookmarks
Aliasing commands with Thor (so lard s means lard search, etc)
Working with your bookmarks offline, so you can make changes and upload them to Larder servers next chance you're online. Or, work entirely offline, if that's your workflow. (But Larder rules, so Lard will always be a Larder client first and foremost)

Conclusion

Writing Lard was a blast, and I'm stoked on how it will help my workflow and how much I learned about Ruby and its ecosystem. I'm excited to see how far I can take the project, and I hope I convinced at least one reader out there somewhere to either Try out Lard and break my code or to make their own project writeup someday!

If you want to try Lard, feel free to run gem install lard and lard help to get started. Or pop over to my GitHub to check it out and say hello!

hawkins / lard

📗 A third-party command line interface for larder.io

Lard

📗 A third-party command line interface for larder.io

Installation

To install, simply run gem install lard. Then, you can run the binary by calling lard.

You'll need to log in before you can do much, so follow instructions in lard login to get started.

Contributing

If you want to run the code locally, follow these steps:

Clone the repo
Make…

View on GitHub

Alright, well I've spent entirely too much time in front of screens this week, so it's time I get out and enjoy the last part of this weekend. Happy weekend, everybody, and thanks for reading!

Why is an IDE for Interpreted Languages "Overkill?"

Josh Hawkins — Sun, 28 Oct 2018 14:55:39 +0000

So often I hear the words "an IDE is overkill for Python/Ruby/JavaScript, just use Vim/Emacs/Nano/IDLE."

Why is that? I write interpreted languages as much as the next guy, but in my experience, I

still want a debugger
still want tag definitions for demystifying framework "magic"
still want types
despise "print statement debugging"

OK, so your fancy vim setup has ctags/cscope for tag definitions and you "know your code so well you don't need types." What about in 6 months when a bug crops up? Still remember what a is? i? j? temp?

So the root of my question is, where did this mantra that you don't need IDEs for interpreted languages come about? You don't need an IDE for any language, but what about compiled languages makes them so much more desirable?

(Don't Fear) The Regex: A Practical Introduction to Regular Expressions

Josh Hawkins — Wed, 22 Mar 2017 05:17:36 +0000

Have you ever used strings before? That's right, those "arrays of characters" we all know and love? Unless you code only in C, I'd bet you have - and maybe even a lot.

But what about using a lot of strings? Or using strings that your program didn't generate? Maybe you're reading an email, parsing command line arguments, or reading human instructions, and you just need a more structured way to handle this.

Sure, you could iterate over each word or element in the strings. And you'll probably understand the code you used to do so. But for large applications, that can become very overwhelming - and very expensive to maintain.

Enter: The Regular Expression

Without getting too deep into computer science, let's define a Regular Expression real quick.

Regular expressions are the grammars describing a Regular language
Regular languages are a form of formal language that can be described by a finite state machine

There are a number of better explanations on regular languages out there, so if you're not happy yet, just google for a couple minutes.

Enter: The Regex

If it hasn't already, here's where it gets funky... I draw a distinction between what programming languages call "regular expressions" and what computer science calls a regular expression.

Computer Science Regular Expression - a grammar to describe a regular language
Programming Language Regular Expression - a grammar to describe, at most, a context-sensitive language

Context-sensitive languages are a good deal more complex and useful, so we'll call a programming language regular expression "regex" now to solidify the distinction that its languages are not regular.

Learning to Write Regexs

Regexs are described between //'s and match strings if they fit the 'pattern' defined between the two //'s. For instance, /Hi/ matches "Hi", so we can check a string to see if it is (or has, more on that later) "Hi" in the string using a regular expression.

We match characters in a string with regular expressions by typing them normally. For instance, /Hello World/ will match the string "Hello World".

We could simplify this to match any word by adding a little regex magic: \w matches any "word" made of only letters: \w will match any one word (if only letters).

We can similarly match numbers with \d.

Example 1

Great, so we can perform string equality or see if strings fit some simple pattern now. So what? Can they be more useful?

You bet! Let's say we wrote an IRC chat bot that listens for someone to say "Josh". Our bot basically scans each message someone says in the channel until we get a match. Then, the bot will respond "Woah, I hope you aren't talking bad about my pal Josh!" Because Josh's only friends are robots.

...

Our bot will use the pattern /Josh/ to match the strings.

Suddenly, some named Eli stumbles along: "Eli: Josh, do you really need that much caffeine?"

Our bot will kick in gear and scan the message with /Josh/ and find one match! So he replies, and Eli is sufficiently creeped out. Mission accomplished!

Or was it?

What if our bot was more intelligent? What if the bot addressed whoever spoke by name? Something like "Woah, I hope you aren't bad-mouthing my buddy Josh, Eli."

Quantifiers (Repeating Characters)

0 or Many

We can do that... but we've got to learn a few things to get there. First off, Quantifiers (for Repeating characters).

We can use * to match 0 or many characters before it. For instance, /a*/ matches "aaaaa" BUT ALSO "". That's right, it will match the empty string.

* serves to match something optional, because the character it matches doesn't have to exist. But it can. And it can exist many, many times (theoretically infinitely many times).

We can match "Josh" with /Josh/, but we could also match "JJJJJJJJJosh" and "osh" with /J*osh/.

1 or Many

+ can be used to match 1 or many characters. It effectively works the same way as * does, except the character existing is no longer optional. We have to have at least one of those characters to match now.

So, we can match "JJJJosh" and "Josh" with /J+osh/ but not "osh".

Wildcards

Great, we can match a lot more interesting features now. Maybe someone screaming "Joooosh" if they're really mad at me...

But what if they're so mad that they slam their face on the keyboard? How do we match "afuhiudgosigs" if we don't know how pointy their nose is?

With Wildcards!

Wildcards allow you to match ANYTHING. Their syntax is .. (Yes, just a period. Period.). You'll probably use this a lot, so don't confuse it for matching the end of a sentence.

We can use this to match "Joooafhuaisggsh" by combining our knowledge of repeating characters and wildcards in this regex: /Jo+.*sh/. To be clear, this will match 1 "J", 1 or more "o", 0 or many wildcards, and 1 "s" and 1 "h". Those five blocks lead us to what we call...

Character Groups

Character Groups are the blocks of characters that appear in order in a string. When you use a * or +, you're actually matching many of the last character group, not just the last character.

This is useful to understand in its own right, but combined with repeating characters, can be very powerful. To do this, we can define our own character group by using parenthesis (that's these guys).

Let's say we want to repeat "Jos" but not "h". So "JosJosJosJosJosh" will match. We can do this with the regex /(Jos)+h/ Easy, right?

But finally... back to our example, how can we get Eli's name in the IRC chat message he sent?

Character groups are also a means of remembering parts of the string. This way we can add parts of a string to variables in our programming code when we see a string that fits the pattern.

To do this, typically you'll do something like \1 to match the first specified group.

For instance, /(.+) \1/ is a special one. Here we look at a group of random characters 1 or more times, have a space afterwards, and then repeat the exact same characters again. So this regex will match the string "abc abc" but not "abc def" even though "def" would match (.*) independently.

Remembering matches is very powerful, and it will probably boil down to the most useful feature of programming with regular expressions.

Example 2

Whew... finally ready to continue with our IRC bot. Let's use what we learned to see who was talking smack.

If we want to capture the sender's name when they say "Josh", our regex can look like this: /(\w+): .*Josh.*/ and we can save the match as a variable in our programming language for our reply.

That's just 1 or more letters followed by ": ", a wildcard for 0 or many characters, the string Josh, and a wildcard for 0 or many characters.

Note: /.*word.*/ is a simple way to match a string containing "word" that may or may not have other things around it.

In Python, that regex might look like this:

import re
pattern = re.compile(ur'(\w+): .*Josh.*')  # Our regex
string = u"Eli: Josh go move your laundry" # Our string

matches = re.match(pattern, string)        # Test the string

who = matches.group(1)                     # Get who said the message
print(who)                                 # "Eli"

Notice we used .group(1) just like we'd use \1 in the regex pattern. Nothing new here, aside from using the regex in Python.

Beginning and End

Until now, we've actually allowed matching strings to occur in any part of the string. For intsance, /(Jos)+h/ will match any string containing the Jos-repeating-h anywhere in the stringg.

What if we wanted the string begin with Jos-repeating-h? We can specify this with /^(Jos)+h/, where ^ matches the start of the string.

Similarly, $ can be used to match the end of the string.

So if we want our pattern to match strings containg Jos-repeating-h from beginning to end, we can alter it to look like this: /^(Jos)+h$/.

Character Options

But maybe you're writing a regex for a sandwich order. You don't know if the customer wants white or wheat bread, but you'll accept either. How do you add choice in a regex? With Character Options!

Character Options allow you to specify a set of possible values for a group. Syntax for this is (white|wheat) in the context of our sandwhich, where either "white" or "wheat" would be accepted.

You could also use the [brackets] to specify options in another way. Each character is an option here, instead of the total string of characters. I.e., "b", "r", "s", "t", e", "k", "c", "r" would each be accepted individually. But this could be handy for more complicated groups, as you can substitute a character for a more complicated expression inside a Character Group here.

Modifiers

We talk about regex's with /slash marks/, right? We know what goes in the middle, but what goes on the sides?

Plot twist, nothing.

... goes on the left. The right side, however, has some very, very useful stuff. It's almost a shame we ignored it for so long!

Modifiers modify the rules with which the regular expressions are applied.

Here's a list of the most common modifiers (from Regex101.com):

Modifier	Nickname	Description
g	global	All matches (don't return on first match)
m	multi-line	Causes ^ and $ to match the begin/end of each line (not only begin/end of string)
i	insensitive	Case insensitive match (ignores case of [a-zA-Z])
x	extended	Spaces and text after a # in the pattern are ignored
X	extra	A \ followed by a letter with no special meaning is faulted
s	single line	Dot matches newline characters
u	unicode	Pattern strings are treated as UTF-16. Also causes escape sequences to match unicode characters
U	ungreedy	The match becomes lazy by default. Now a `?` following a quantifier makes it greedy
A	anchored	Pattern is forced to ^
J	duplicate	allow duplicate subpattern names

For instance, until now, all of our examples have been case-sensitive. That means, capitalizing or lower-casing any one character would make that string no longer match the pattern. We can make our patterns case-insensitive with the i modifier.

Maybe Eli was so mad at me that he spammed the chat with a Mix OF casE CHArACters. Never fear, i is here! We can match his "I hAate LiVing witH JOSH!!!" rage with /i ha+te living with josh!+/i. Now it's easier to read and more powerful and useful. Awesome!

I'll leave the rest of the modifiers for you to play with on your own, but I bet you'll find igm to be your most used in general.

What's next?

Hopefully this article has shown you another useful way to interact with strings a little bit more intelligently. I've hardly even scratched the surface of regexs, but you already know how to use regular expressions for some minor tasks now.

There's an overwhelming number of symbols / tokens to use in your regexs. Typically you'll stumble on them from Stack Overflow searches, or you'll guess them from previous experience (\n is the new line character, for instance). You've more or less got what you need for now, but there's plenty still to learn.

You can get a full list of tokens and test your regexs extensively here. I still use this website almost every time I write regexs, because the testing tool is remarkably helpful and powerful. It even generates code for you if you're not sure how to do it in your programming language yet.

If this was a cakewalk for you, check out regex crossword puzzles. They'll really get you thinking with regex!

Hi, I'm Josh Hawkins

Josh Hawkins — Wed, 22 Mar 2017 00:43:50 +0000

I have been coding for over 11 years.

You can find me on Twitter as @hawkinjs, and GitHub / NPM as @hawkins

I live in the south, currently, but I'm looking to move soon.

I mostly program in these languages: JavaScript, Python, C.

I am currently learning more about full stack web development. If you'd like to talk to someone about full stack, @ me on twitter or send me an email (from my GitHub)!

Nice to meet you.

DEV Community: Josh Hawkins

Using Hubot to Mention Everyone in GroupMe

hawkins / groupme-at-all

A Hubot chat bot to mention every user in a GroupMe channel all at once.

Unofficial GroupMe @all

Features

Using

How it works

How it works (technical)

How to install groupme-at-all

Create a GroupMe Bot

Setup Hubot

Setup @all

Configure Environment Variables

Test Your Bot

Success!

(Optional) Further Config

Begone, sleep!

Blacklist

Leveraging Travis-CI for Continuous Deployment to Publish Compiled Binaries to GitHub

What is cross-compilation?

Where Is Your Code?

hawkins / watchdog

⚠️ Watch filesystem for changes and then run a command

Watchdog

Usage

Goals

Initial Travis Config

Travis Releases Provider

Testing Travis Changes

Connecting the Dots

But Wait, There's More!

Renaming Your Binary

Putting it All Together

hawkins / watchdog

⚠️ Watch filesystem for changes and then run a command

Watchdog

Usage

Goals

The Story of How I wrote A CLI for Your Bookmarks using Larder.io

hawkins / lard

📗 A third-party command line interface for larder.io

Lard

Installation

Contributing

Remove use of HTTParty #4

Concerns:

Missing dependencies? #7

Comment for #7

Comment for #7

hawkins / prettier-webpack-plugin

Process your Webpack dependencies with Prettier

Prettier Webpack Plugin

How it works

Looking for a loader?

Installation

Why?

How can we ensure the User-Agent gets updated with every new version? #11

This issue was generated by todo based on a TODO comment in 295e5d9cf7b86de00f6376f9b2627bd11b44f9ac when #10 was merged. cc @hawkins.

Add tests #15

Recap: What I learned

What's next for Lard?

Conclusion

hawkins / lard

📗 A third-party command line interface for larder.io

Lard

Installation

Contributing

Why is an IDE for Interpreted Languages "Overkill?"

(Don't Fear) The Regex: A Practical Introduction to Regular Expressions

Enter: The Regular Expression

Enter: The Regex

Learning to Write Regexs

Example 1

Quantifiers (Repeating Characters)

0 or Many

1 or Many

Wildcards

Character Groups

Example 2

This issue was generated by todo based on a `TODO` comment in 295e5d9cf7b86de00f6376f9b2627bd11b44f9ac when #10 was merged. cc @hawkins.