DEV Community: Anthony Bouvier

Changing GitHub Enterprise users from LDAP to local

Anthony Bouvier — Thu, 22 Apr 2021 02:20:56 +0000

What?

The company I work for recently split off our entire software engineering team, part of other teams, and backoffice people into a stand alone independent software vendor. For the time being we're sharing a single GitHub Enterprise instance on prem and as part of the transition we've moved the new company staff off to their own Active Directory.

Unlike other tooling I have in place like Jenkins or Harbor, GitHub Enterprise only supports using a single AD/LDAP server (version 3.0.4 as of the publishing of this post). So the 6 or so digital marketing developers left behind at the parent company have to be converted from LDAP auth to local authentication.

Thankfully this is possible with GitHub Enterprise via this setting:

But?

The issue is that accounts that were previously LDAP are flagged as that and this setting only allows for new accounts.

I asked GitHub for instructions on how to convert those accounts and they sent me some ghe-console commands to run. And here is what they sent me:

If enabled, temporarily disable LDAP sync to prevent it from re-writing the LDAP associations.

Connect to the server via SSH. Destroy the LDAP mapping for the users you'd like to switch to built-in authentication and set a default password for those users: (replace USERNAME with the user's actual GitHub username, and replace NEW_PASSWORD with a temporary password)

  ghe-console -y
  User.find_by_login("USERNAME").ldap_mapping.destroy
  pass = "NEW_PASSWORD"
  user = User.find_by_login("USERNAME")
  user.update_attributes(password: pass, password_confirmation: pass)
  exit

Ensure these users are not in the LDAP directory before they attempt to login again.

Re-enable LDAP sync if necessary.

Instruct users to update passwords after logging in.

No doubt this is from their support team's knowledgebase. The problem is it throws errors on trying to use the code as-is.

Of course, I only had a few tracebacks to go off of like this one for update_attributes:

Traceback (most recent call last):
        3: from /github/script/console:92:in `<main>'
        2: from (irb):4
        1: from /github/vendor/gems/2.7.1/ruby/2.7.0/gems/activemodel-6.1.0.rc1.8389f99/lib/active_model/attribute_methods.rb:469:in `method_missing'
NoMethodError (undefined method `update_attributes' for #<User:0x000000000000>)
Did you mean?  update_attribute

So then I tried to change it to update_attribute:

Traceback (most recent call last):
        4: from /github/script/console:92:in `<main>'
        3: from (irb):4
        2: from (irb):5:in `rescue in irb_binding'
        1: from /github/vendor/gems/2.7.1/ruby/2.7.0/gems/activerecord-6.1.0.rc1.8389f99/lib/active_record/persistence.rb:612:in `update_attribute'
ArgumentError (wrong number of arguments (given 1, expected 2))

Damn. Well, it sorta looks like I'm sending two arguments instead of one, so what's going on?

Since I now know we're dealing with Ruby in the ghe-console I start looking up some docs and find out that update_attribute only does one attribute at a time and whatever format I'm sending it doesn't like.

I also look up update_attributes and find out it was deprecated. You only need update in this case. So I try update with the same params and it worked:

user.update(password: pass, password_confirmation: pass)

Or what I originally tried works too using a different syntax for sending the attributes that I saw in the update documentation:

user.update(:password => pass, :password_confirmation => pass)

Gut?

Mostly just chose that subtitle to rhyme with the others, but actually using your gut to do such debugging is something I recommend. The longer you're in this business, the more experience you have running up against just weird scenarios like the above. My vendor gave me syntax that was actually out of date, but I used the info given to me for a language I have zero experience with to debug what to do next.

And I wrote them back to let them know to update their knowledgebase.

So follow your gut! Know how to look up docs even for things you don't know about or langauges you're unfamiliar with.

Deduce. Experiment. Test. Repeat. It becomes second nature.

Apple AirTag announced

Anthony Bouvier — Tue, 20 Apr 2021 17:54:24 +0000

The AirTag has been announced by Apple.

"Apple today introduced AirTag, a small and elegantly designed accessory that helps keep track of and find the items that matter most with Apple’s Find My app. Whether attached to a handbag, keys, backpack, or other items, AirTag taps into the vast, global Find My network1 and can help locate a lost item, all while keeping location data private and anonymous with end-to-end encryption. AirTag can be purchased in one and four packs for just $29 and $99, respectively, and will be available beginning Friday, April 30. "

And I think the real key element here is the use of Find My App / Find My Network.

Apparently this leverages the vast, mostly unknown, network that Apple has where every device from them with Bluetooth on contributes to the Find My network. Meaning, even if you don't own a AirTag, but you happen to have Bluetooth on and are near an AirTag, your phone will send and receive data to share the location with the network.

No opt-in. No opt-out besides leaving Bluetooth off.

Though I've definitely been one to lose my keys or backpack and wish I had something like an AirTag or its predecessor called Tile I am wholly uncomfortable that they are using my device to communicate with other people's devices creating a large mesh network with no knowledge of my participation.

This is being discussed on Hacker News right now and other sites too. The benefit seems cool and huge, but much like Amazon's announcement several months ago about its Sidewalk feature, I feel like there will be a backlash.

I'd love to hear what others think because my initial reaction is revulsion. And I'm seeing mixed thoughts on articles concerning it. To me it is without my consent and if it weren't for a single device in my home I use Bluetooth for from my iPhone I would definitely have it turned off all the time.

Why isn't there active, positive consent to participate in this?

Why isn't this theft of resources, although minimal, to participate in this?

Do we really want to participate more in the already egregious tracking done by just participating in having a smart phone?

Temperature and Humidity Sensors (part one)

Anthony Bouvier — Sun, 18 Apr 2021 21:25:02 +0000

This article originally was published on the my site where I discuss my homelab, DevOps, and other tech related topics.

Setting Up

I have a home from 1914 which has a lot of charm and a lot of issues. One of which is the need to constantly pay attention to the humidity levels and temperature in all floors, finished and unfinished, in the house. This project of my homelab will cover the hardware and custom software I write for it.

Hardware

4x Raspberry Pi Zero WH (just like the W, but with pre-soldered headers on the GPIO)

4x Raspberry Pi official zero w cases

4x Raspberry Pi official power supplies

4x Gigastone 16gb MicroSDHC

Operating System

Though I wish I could use Ubuntu LTS on the Pi Zero W, it doesn't seem to be supported or I at least haven't been able to find out how to boot it. Like my pihole project for the lab, I'm using the most recent official Raspberry Pi OS Lite (because I don't want a desktop environment on this headless computer). And of course I use the Raspberry Pi Imager -- it's just really convenient.

After I image the SD card I make a couple changes to the config.txt file on the card itself from the host machine I did the imaging from. This is so I can set the gpu mem split and disable bluetooth.

Note: when the Raspberry Pi imager is done it unmounts the SD card, so you'll need to plug it back in to your host machine to edit the config.txt file. On my Mac, it mounts as boot.

I add these two lines config.txt. There are a couple of lines that already set dtoverlay that are commented out, so I just put both of these in that area:

gpu_mem=16
dtoverlay=disable-bt

Wi-Fi

I also want to go ahead and set up the wifi on the pi zero so that when I turn it on it automatically connects.

Following this tutorial I create the wpa_supplicant.conf file on the root level of the boot drive while the SD is still connected.

SSH

SSH is disabled by default so since I want to log in soon after booting it and stay headless, I merely need to add a blank file named ssh to the root level of the ssd that I still have mounted.

Once I've done that I eject the SD card and install it into the pi zero.

First Boot

After I first boot, I want to make sure to make some changes to each device. There's probably a way to get fancy and make this super repeatable, but I'm not installing 100 sensors; I'm installing 4.

Wrap Up Setup

I find the device on my network by looking at my Unifi OS that shows the whole network as a map and I just found the device and ip there, but I could also scan the network from the command line and look for Raspberry Pi MAC addresses:

arp -a | grep -i "b8:27:eb\|dc:a6:32"

Or if you're on a Mac or a Windows machine that has Bonjour set up you might be able to talk to it at its default hostname on .local via raspberrypi.local

ssh pi@ip.address.you.found
# or 
ssh pi@raspberrypi.local

WARNING: If you use ssh pi@raspberrypi.local more than once because you're setting up multiple sensors, keep in mind ssh is going to complain that raspberrypi.local doesn't match the fingerprint from the last time you did it. Cause each device you set up is unique, but ssh doesn't know that. So I suggest using the IP because it is more likely to be different. And below we're giving each their own hostname anyway.

Change the default password (always a smart thing to do):

passwd

Update apt and upgrade the OS packages:

sudo apt update && sudo apt upgrade -y

Update the hostname. I'm going with a naming scheme of 'temphum1', 'temphum2', etc.

sudo vi /etc/hostname

And now let's shut it down for now because we're going to be installing the sensor and doing some soldering next before we turn this device back on.

The part two of this post will be abour the physical aspects of the project like testing out on a breadboard and soldering the real thing together.

Building my static site with Pelican (part two)

Anthony Bouvier — Sat, 17 Apr 2021 18:59:07 +0000

This article originally was published on the site I'm writing about.

Last I left off in part one I generated the content with invoke build. I want to go over the remaining steps on how to preview the content, how I'm publishing it to GitHub pages with the submodule created previously, and also how I plan to use invoke to automate some of this.

Markdown Tweaks

First before previewing the content, I had to make some adjustments to how the Markdown is processed, especially for code blocks. By default pygments, which pelican uses for code highlighting, didn't mesh well with my theme. I made sure to make my pre/code blocks have a dark background for one. I also wanted it to stop guessing the language (it was giving weird results trying to guess at the command line code blocks since they're not really a 'language' per se), and for it to recognize GitHub style fenced blocks (using backticks to surround code blocks instead of relying on indentation).

I added this to my pelicanconf.py file:

MARKDOWN = {
    'extension_configs': {
        'markdown.extensions.codehilite': {'css_class': 'highlight', 'guess_lang': False},
        'markdown.extensions.fenced_code': {},
    },
    'output_format': 'html5',
}

Preview

The invoke build command looks at my pelicanconf.py file for settings. Some of these settings are theme specific, but they also cover things like what is the site url, how many pages to show before paginating, and so forth. Below I will also be using publishconf.py for prod settings like my Google Analytics tag and FQDN for the site url.

Now that the output directory has my generated content, I want to preview it.

invoke serve

This will create a simple local webserver to show you my content on http://localhost:8000 so I open Firefox and make sure everything seems to be working as expected. I click around, look for formatting weirdness, agonize internally over whether I chose the right theme or not, and then ultimately decide all is well.

That's pretty cool, buuuuuuttttt:

Invoke

Let's dive a bit more into the invoke command and the default tasks.py file that is created.

There are lot of built in invoke tasks in this file already like build, serve, and publish. Below are my tweaks.

First I remove the clean, and rebuild commands by commenting them out (maybe I'll redo them in the future). This is becaues they're destructive and will remove the output folder entirely, but remember that I'm using it as a submodule so it is its own git repo. Deleting the whole folder would be too destructive. I also comment out the preview task as I only need to do a prod build when I'm publishing.

Next I need to change how the publish task works because I'm not using rsync to publish this, I'm pushing the content of output to my thebouv.github.io repo so that it serves automatically with GitHub Pages.

Because I'm running two back to back commands in order to commit the output and the raw content of the main repo, I had to split it into two tasks and have one run before the other. I tried combining this into a single command but it wasn't working. Only one git command would occur and I think that is because the context is lost after git takes control and starts to spit out messages. I'll have to revisit that some day just for curiosity's sake.

@task
def publish_output(c):
    pelican_run('-s {settings_publish}'.format(**CONFIG))

    # Commit the written content to GitHub from this repo
    os.chdir(CONFIG['deploy_path'])
    print(f'In Directory: {os.getcwd()}')
    c.run('git add .')
    c.run('git commit -m "auto-commit output from Invoke"')
    c.run('git push origin master')

@task(publish_output)
def publish(c):
    # Commit the static html pages to thebouv.github.io repo
    os.chdir('..') # my output folder
    print(f'In Directory: {os.getcwd()}')
    c.run('git add .')
    c.run('git commit -m "auto-commit content from Invoke"')
    c.run('git push origin master')

I only need to run invoke publish and the output directory is refreshed with my new generated site with prod settings from publishconf.py. Then the thebouv.github.io repo gets an auto-commit and push. Followed by thebouv-pelican receiving an auto-commit as well (since the submodule has updated, this step commits updating the submodule to point to the new commit SHA of thebouv.github.io).

But wait, there's more

I noticed in the default task.py file that there is a livereload option. I had to try it out by installing the livereload module and then invoking the task.

pip install livereload
pip freeze > requirements.txt
invoke livereload

Now the browser nicely refreshes for me automatically as I edit and save. That's pretty damn handy.

DevOps teams are an anti-pattern, but ...

Anthony Bouvier — Fri, 16 Apr 2021 15:03:37 +0000

People put too much pressure on themselves to do DevOps the "right" way or that they're not doing it like they read about at Netflix or Amazon or any other examples. At small to medium sized companies introducing these new concepts, someone(s) need to own getting it started and to be the evangelists.

Yes, long-term having only a single team as your "DevOps" team is an anti-pattern, but you gotta start somewhere.

What follows is my response to post online about how someone was feeling like they're doing DevOps wrong at their company. And asking why is a DevOps team frowned upon.

I look at it like this as I currently have a DevOps title but didn’t really want one.

DevOps should be your entire company’s culture. Shift responsibility left. Let the devs run what they build.

But you have to start somewhere. You need people to be the catalyst. They need to learn Jenkins first to help others. Much like you describe. A central team is a DevOps anti-pattern long term but I feel like they’re necessary at first. To be the evangelists. To be the first to install the tooling, get it working in your SDLC, to create the pipelines and grease the cogs to get the culture moving.

They likely own the tooling too. Or a tooling team splits off at some point. Someone needs to manage the tooling and host it and upgrade it and juggle licensing.

But be open to others learning. Cross train devs and ops people. Make sure not to bottleneck people needing resources. Find ways they can fire up their own vms, containers, app services, storage accounts, etc. Eventually that is.

Gotta start somewhere. Not everyone is or can be a FAANG company. Not everyone needs every aspect of DevOps people try to shove your way. Not everyone needs to throw k8s at every problem. And so on.

Improve tooling.

Improve dev empowerment.

Improve SDLC.

Improve testing.

Improve blameless culture.

Improve observability.

Improve continuously.

Do those and your company/team will become DevOps too. Your team will evolve too. You’ll continue to expand what you offer and what you can institutionalize.

Or maybe they resist every damn change you propose and never want to improve. At that point you flip the proverbial table and find a place where you CAN do those things. :)

This article originally was published on thebouv.com

Building my static site with Pelican (part one)

Anthony Bouvier — Thu, 15 Apr 2021 14:24:49 +0000

I chose to use pelican for a few reasons:

This site is hosted for free on GitHub and that means it needs to be statically generated.
Pelican is written in python and I'm a python fanboy.
Pelican is still in active development based on the merges, issues, and discussion on its repo in GitHub.
Pelican has all the features I need plus I can supplement with other things like invoke (which I just learned about and am excited to try).

In the final structure, I will have one repo for the pelican install (thebouv-pelican.git) which will have settings, pelican files, raw assets like a post's markdown files and images. And one for the static site files (thebouv.github.io.git) because GitHub only serves from root or /docs. I feel this way is cleaner and allows the I will be automating publishing with invoke.

I am basing this process off of this tutorial but with some of my own ideas and organizational changes. Definitely fewer dependencies as I have no need for Jupyter Notebooks on this site, nor for now do I think I need typogrify or beautifulsoup as per that tutorial. I'll also be looking at the pelican quickstart and other pelican docs to make decisions as I go.

Setting Up

First I cleared all previous content out of my previous static site and leave only the README.md and CNAME files. The README will get updated during the project and CNAME is a text file required by GitHub to have thebouv.com as a custom domain. I could have set up a totally new repo or trashed this one and restarted, but with git I don't need to and I can always roll back one day or refer to the old site if I need.

Some git stuff

I create my empty repo for thebouv-pelican and set it up in GitHub as a remote:

mkdir thebouv-pelican
cd thebouv-pelican
git init
gh repo create

Notice the gh command. If you don't use the GitHub command line tool gh yet I do recommend it. It is pretty handy.

Pelican!

Now to set up my python environment and start getting Pelican set up. I'm doing this all with python 3.9.1 because it is stable and the most recent 3.9 release that pyenv has.

python -m venv .venv
source .venv/bin/activate
pip install "pelican[markdown]" invoke
pip freeze > requirements.txt

Now for the remainder of this post I'm going to no longer talk about being in a venv, activating, deactivating, etc. If I'm in a python project, I've got the venv activated. I assume the reader is managing this aspect.

I'm going to run the pelican_quickstart command line app and answer all the questions it poses. I'm filling out my site's details and pretty much accepting the defaults to things with defaults. Only thing is I'll be deleting the Makefile as I plan to use invoke instead to run the tasks in tasks.py.

I'm also manually creating a pages folder under the content folder generated by pelican_quickstart because I will want some stand-alone non-blog pages in the future like a resume.

pelican_quickstart
rm -f Makefile
mkdir content/pages

This leaves me with this structure below:

thebouv-pelican/
├── content
│   └── pages
├── output
├── pelicanconf.py       # Main settings file
├── publishconf.py       # Settings to use when ready to publish
└── requirements.txt     # saved earlier from pip freeze

Now that those exist and I want to start adding content, I want to do one thing real quick. I want to make my output directory actually be a git submodule of thebouv.github.io.git since the generated output will be in that repo.

git submodule add -f git@github.com:thebouv/thebouv.github.io.git output

Later I will automate committing and pushing both this repo and the generated content up to GitHub with invoke.

Theme

For now I plan to use a theme that is inside this repo called brutalistpelican but host just a fork of the theme for my own version. I'm doing this because I don't need the rest of his pelican set up (I have my own which this post is about) and only the theme itself. After I clone their repo I move the theme folder out and set it all up as its own repo. Then I can add the submodule like I wanted to do:

git submodule add -f git@github.com:thebouv/brutalist-theme.git theme/brutalist-theme

I've definitely made some changes to the theme that I'm not going to go into detail on. One of the main things I've done is get rid of any external javascript except Google Analytics. I also tweak page width to 60em max (non-brutalist but I wanted a slightly wider content area). I'll be making more changes over time and if they're significant, I'll write about them but likely most will be slight tweaks.

Any changes I make to the theme will need to be pushed up as well to its own repo but I can do it from the submodule itself which is handy and something I will be automating with invoke in part two of this post.

cd theme/brutal-theme
git add --all
git commit -m "some message about the changes I made"
git push

Let's fire this baby up!

This article, the one I'm writing and you're reading, is the first post in my blog. So the instructions below are so I can test this out locally and see it for the first time in all its simple glory:

invoke build

This statically generates all the content and uses the theme to style it. Then it puts all those static files in output which you'll recall is a submodule. Then I can go into that folder add all the output to git and push it up. And since it is set up to be hosted by GitHub once this output is pushed up, my site will appear here at https://thebouv.com

cd output
git add --all
git commit -m "auto-generated output from pelican"
git push

Next time

Okay, realize this is getting long and I have more to do but I've detailed everything that has gotten the site to this point. Well, besides the fact that I switched themes like 5 times but I didn't need to document that part. ;)

You can read part two if you want. :)