Poll: What are you using Huginn for?

I'm giving a talk about Huginn next week, and I'd like to include a discussion of some of the awesome things that you're doing with Huginn.

So, what are you using Huginn for?

/cc @albertsun @dsander @knu @afro88 @akilism @ms32035 @scavone @racktear @rloomba @wizonesolutions @snicker @j-wilkins @Alex-Ikanow @rajcybage @ZirconCode @KenYN @tijmenb @dsummersl @kyzh and everyone else!

I'll start.

I use Huginn to:

• alert me about spikes in the occurrence of topics that I care about on Twitter (like "nasa mars discovery", "huginn", and "railsconf cfp")
• alert me about spikes in high frequency terms like "san francisco emergency" on Twitter
• tell me about every occurrence of very low frequency terms on Twitter like "andrew cantino" (ego search!)
• tell me about new XKCD comics, what-if articles, and iTunes trailers
• remind me to take an umbrella when it's going to rain
• alert me when the next train is arriving in the morning (hasn't been super useful, though)
• play with Mechanical Turk
• watch for new posts on a few sites

I use Huginn to monitor various online music retailers (Beatport, iTunes) and Soundcloud for new releases by artists I'm into. I'm a DJ so it's important for me to stay on top of new releases. This should be a pleasure but is usually a chore as most sites aren't optimised to provide you with new music from your favourited artists in a useful way. So you end up wading through a lot of non-relevant crap.

I've got a pretty specific set up to allow me to add artists by entering their name via a manual event agent, which passes through website agents (search pages) and url formatters to get the correct url's to scrape for the artists tracks for each site. All these urls are scraped once per day for a digest that's emailed in the morning. It's kinda like having a PA that is totally on point with their music and recommends you stuff every morning :)

I use Huginn mainly to monitor social media progress. For example: DeviantArt watchers, Facebook likes, Youtube subscribers, Twitter, Instagram, etc... Numbers which are all over the place and often horribly formatted. It compacts them into a nice clear report and even gives me encouragement at certain milestones.

A part from that, I also have it check the weather, Xkcd and a few other comics, some RSS feeds, mostly news. I have it watch for public transit problems and earthquake warnings (Japan: jma.go.jp). I'm trying to get it to accept those episode release lists from Wikipedia and notify me when a new one comes out. I'm also trying to get it to work with Beeminder.

Most importantly of all though, I make sure my Huginn is very nice to me ^_^

At the moment I use Huginn just as a replacement for zapier. It is set up to send me a message via Hipchat for every activity on one of my Basecamp projects.

I use Huginn mainly to monitor webpages that don't provide RSS feeds or to provide updates via email for certain important sites. Also for some infrastructure monitoring.

More here: https://source.opennews.org/en-US/articles/open-source-bot-factory/

I started using it to monitor the release of a laptop specs on lenovo's website. Then i try to integrate it in my reporting and monitoring workflow (logstash and IRC) with moderate success.

I'm using it to watch a few nyc government sites so I can be alerted to new data releases.

I also have an agent I made that allows me to watch what certain members of congress are doing in regards to voting, sponsoring and cosponsoring bills using the Sunlight Foundation API.

I'm using Huginn for a couple things:

• warnings for windy days ahead
• warnings for temperature dropping below freezing in winter so I can turn off my sprinklers
• monitoring bandwidth usage on my home Internet connection
• monitoring upcoming music album releases and music events in my city
• monitoring fluctuations in currency exchanges like BTC and whatever country I'm visiting next to get best exchange rates
• monitoring twitter for certain terms like everyone else it seems
• monitoring status of pending corporate expense reports
• notification of when my employees complete field reports

All of my notifications are funneled through the Growl output agent which gets sent to my Growl bounce server. This server then redistributes any incoming GNTP messages to defined GNTP/Prowl/Pushover/NotifyMyAndroid targets.. currently just my iPhone, my laptop, my tablet, and my office desktop.

Planned (for when I have time to put together the flow or write the agents):

• automatically paying the balance of credit cards used solely for accumulating rewards points
• HabitRPG agent
• Foursquare agent
• Toodledo agent

I use it to check for software updates

I plan to use it for:

• Collecting events from CI farm
• watching for VCS trends and collecting interesting commits
• collecting a digest of warnings from nagios

For the first one I will likely implement a Jenkins agent

I'm currently using it to

• check ticket availability and prices on StubHub for shows/events
• alert me about the weather

I'm planning to use it to

• alert me when new postings are posted to craigslist matching my criteria
• send alerts when certain cryptocurrency prices go above/below threshold
• give me a daily digest of the the top 10 Beatport charts for certain genres

I'm using Huginn to:

• get alerted as the kindergarten bus approaches, watching the bus tracking service online
• subscribe to sites that are only rarely updated or have no feeds
• get notified of new tarballs that are put on an FTP site without any official announcements
• perform some general service monitoring

and so on. It serves me really well!

@mfclarke I use Huginn to monitor various online music retailers (Beatport, iTunes) and Soundcloud for new releases by artists I'm into.

That sounds awesome and is something (though perhaps not as advanced) that i've been wanting to setup for a while. Would be interesting to see what you've configured/how to do it.

@ZirconCode I use Huginn mainly to monitor social media progress.

That's cool. How do you have it grabs the stats from each one? RSS feeds or..?

@snicker monitoring upcoming music album releases

What do you monitor for this? I basically want something I can put all my favourite artists into and get notifications. I currently get an email from http://www.albumreminder.com/, but it tends to get lost in a sea of inbox.

The Growl bounce server sounds kind of interesting. Is it a homegrown thing? Available online at all?

Would also be keen on the HabitRPG/Foursquare agents when they become a thing.

@rloomba send alerts when certain cryptocurrency prices go above/below threshold

Definitely like that idea

Now that all those comments are done, I technically don't actually use it at all yet (mostly because i've only got it on my laptop, which isn't always on) Once I get around to getting it running somewhere more permanently online (suggestions? wheres the best/cheapest place to put it 'in the cloud'?)..

Start by moving my IFTTT recipes over and streamlining them:

• Monitor some websites/RSS feeds and send me an email
• Send me an email when specified users post on tumblr with a certain hashtag
• Save tumblr post/images to dropbox when specified users post with a certain hashtag
• Save facebook photo's I upload to dropbox
• Save facbook photo's i'm tagged in to dropbox
• Save links I share on facebook to evernote
• Save instagram photo's I upload to dropbox
• Save instagram photo's I like to dropbox
• Save foursquare photo's to dropbox
• Notify me when Netatmo tells me the CO2 is too high in the computer room

Then there are a few 'normal' sorts of things:

• Monitor twitter stream for interests+stuff about me
• Grab the weather from cloud services + local data from my netatmo

Then there's a whole plethora of things I think about that I could do with it on a daily basis (working in the areas of personal clouds and event networks, so it's always in the back of my mind! :p):

• Do actions based on tagging an email in gmail (eg. Process/take action then move to a new tag)
• Tell me when new music is out by artists I like
• Be a 'single sink' for all of my productivity needs (eg. Send events to Huginn like a todo list, but then be able to automagically and/or when manually released process them with other services such as virtual assistants, etc)
• Monitor when media releases are available on 'peer distributed media' and acquire them
• Monitor when movies/shows on my watchlist on trakt.tv are available on streaming/'other' services, and maybe add them to be acquired
• Streamline keeping an eye on things by giving me one place to look (be that a dashboard or a summarized email)
• Keep similar but disparate services in sync (eg. online dating profiles or similar, profile picture between sites, social media updates, etc)
• Package tracking/monitoring
• Automating repurchasing of items when they run low (eg. add items to list when i consume them, generates shopping list, purchases items once a fortnight and has them delivered to me)
• Consume all my various smart things/health tracking data and do something cool/useful with it (dashboard/cards would play well here)
• Do something cool with the data from my Freematics OBD-II Adapter V2
• Scrape together location data from all of the services I use to provide a 'complete picture' (without having to have a battery draining app/etc running 24/7). One thing I want to do from there is to provide a simple 'low resolution' (eg. City/State/Country) location to put on a website or similar with my current location.
• HackerNews top 10 posts of the day
• Bayesian/other learning algorithms to figure out what things I actually like from streams of content so it can be filtered down.

No doubt a ton more things that I can't even think of right now (see the issues/agent ideas I keep raising for ideas! :p). I basically see Huginn as the place where I will be able to outsource and automate most things I do currently from my head so I can focus better and not have to think as much about getting things done.

@alias1

What do you monitor for this? I basically want something I can put all my favourite artists into and get notifications. I currently get an email from http://www.albumreminder.com/, but it tends to get lost in a sea of inbox.

Slightly cheating; I use Headphones, which I told to import my Last.fm history. As a result, every artist I've ever listened to on Spotify or otherwise (even accidentally) was imported and is monitored for album releases. A bit of a bane and a boon... but Headphones does not have any sort of built-in notification for when it adds an upcoming album to my "Wanted" list. I have Huginn watch this page for changes and send me a growl notification when something new gets added. I've thought about scraping the code out of Headphones and creating an agent in Huginn directly, but I don't know when I'd have time for that.

The Growl bounce server sounds kind of interesting. Is it a homegrown thing? Available online at all?

You can find it here:

https://github.com/snicker/gntp-regrowl

I had plans to add addtional refinements and may still do so eventually, but it presently just forwards any incoming messages to any targets defined in the configuration. The bonus is that there is support for forwarding to Prowl. Another user implemented support for Pushover and NotifyMyAndroid but I have not pulled those changes in to the main repository at this time.

@alias1 I started on openshift but had issues with memory and processes getting killed. Though this might be an issue of the past with @dsander's latest PR. Ended up moving to Digital Ocean for $10 a month for a 1GB VM and couldn't be happier.

As for the Beatport etc scraping, I'm using 2 agents that I coded up specifically for the job. 1 to add an artist (modified manual agent that has a form) that then runs through a series of scrapers and event formatters to get the correct urls to scrape for tracks. And 1 to hold all the url's that's scheduled to send these urls as events to other scrapers to get the tracks. Works really nicely for me!

Sorry I'm late to this discussion! I'm planning to use Huginn to monitor Docker container clusters. Whenever a container stops, the custom agent I'm writing emails me, and it continually provides info on the number of containers running, the amount of memory being used, etc., that I can display on my product page.(I've built a PaaS for computational math.)

@snicker Cool, thanks for that :) I've come across headphones in the past I think, but never really played with it. That albumreminder site lets you scrape your last.fm, upload an itunes library, manually add artists, etc. Has an RSS feed, but no API that I saw last night for adding new artists unfortunately. Uses amazon as a backend last time I checked (if I remember correctly)

@mfclarke Nice. How hard/straightforward was it to setup on digital ocean? $10/month is definitely within budget. Looking at it now and can't decide between CentOS and Debian. Did you follow one of the setup guides on the wiki (So many.. not sure which is 'best' for a prod setup. I've only run a dev version locally. Any hints?), or just sort of wing it?

Cool. It'd be really interesting to see stuff like this (and what other people setup) when Huginn has a concept of 'scenarios' and being able to share them.

@rudimk Cool! Are you planning on contributing the agent when it's done?

@alias1 For some things, like youtube, a simple xpath with the web agent for a .subscribed class will get me the info neatly. For other things, like deviantart, I have to pass it to another agent and then regex the numbers painfully. It's all very organic and fiddly at the moment, but I do plan on making it easy to set up soon. I want it to be able to handle a slew of account names and give me an organized report.

-> "Notify me when Netatmo tells me the CO2 is too high in the computer room"

That is extremely cool, Huginn turns into the proverbial coal mine canarry. Or maybe not cool, depending on where you're sitting =p

@mfclarke & @alias1 1+ for DigitalOcean. I use them too and there's nothing at all to complain about. I generally wing it with the installation, some of the guides are outdated and some "just" make it work disregarding best practices. I find it's important to keep up to date on how your things are deployed, if you can pay the time toll, of course.

@alias1 I followed the capistrano deployment guide, but ended up having to change things a little to get it to deploy. It was an interesting exercise, I've never deployed Rails apps before. Set aside an evening for it, it won't be a quick and easy experience if you haven't done it before, but it's worth it :)

I did this: https://www.digitalocean.com/community/articles/how-to-deploy-rails-apps-using-unicorn-and-nginx-on-centos-6-5

Then this: https://github.com/cantino/huginn/wiki/Deployment-with-Capistrano%2C-Unicorn%2C-nginx%2C-Foreman%2C-and-Upstart

@ZirconCode Nice :) Haha yeah, I don't necessarily act on it all the time, but cool nonetheless. Would be good to get a smart bulb that I can turn red when it's too high or something like that (as is I don't necessarily look at the notifications when they come through)

@mfclarke Cool, will have to set aside some time to have a proper look and play around. Thanks :) I'm guessing by that you ended up going with CentOS? Any reason in particular, or just because?

@alias1 Yep. I chose CentOS cause that's what we use at work for web hosting, and the Digital Ocean guide was fairly straight forward

Coolies :) Yeah, I run CentOS on my current VPS, but also played with debian a lot with my old job as my pentest distro. Will probably stick to CentOS for this for funsies though. Cheers.

Since getting AWS support in there, I've been using a heavily-modified PostAgent to post things to a custom ElasticSearch instance (which feeds a pretty HUD).

At the moment it's dead easy to use IFTTT to put all my online activity into a Tumblr, and then just get Huginn to feed from Tumblr's RSS, but I plan to move everything into Huginn, especially considering that the Tumblr vocabulary is limited (only text, quote, photo, ~~audio~~, video, link and chat). E.g. I'd like to be able to review a book and have my HUD custom render it with stars, etc. I also have plans to get my voice memos up there (I have an Android app idea called VoiceCapsule, a play on "time capsule").

@opyate Have you seen the custom webhook hack for IFTTT?

What are you using for the HUD? And is it a public thing or private? Sounds like an intersting usecase. How does elasticsearch play into it all?

@alias1 @opyate The HUD or programmable personal dashboard is very interesting. Unfortunately, as far as I know, there are no open solutions yet.

@alias1 Yep, absolutely! Nailing the agent as we speak ;)

@rudimk Nice!! Now THAT is something I am most definitely interested in!!

@alias1 @racktear I've put my hack (WIP!) up on the 'Hub.

I'm aware of the custom webhook hack for IFTTT, but getting a Huginn instance up on AWS was dead easy, and besides -- I want to move away from IFTTT completely.

I don't know if the README gets the intent over yet, but ElasticSearch basically becomes a cache for all my online activity, ultimately populated by Huginn via the PostAgent.

@opyate I was more thinking as an interim bridge between IFTTT and Huginn.

IFTTT posts to a common Tumblr, which is then RSS-fed into ElasticSearch via Huginn

IFTTT wordpress webhook hack -> Huginn -> ElasticSearch

then eventually just Huginn -> ElasticSearch

Currently using Huginn to send me pushover notifications whenever a new reddit post is created and matches a particular regex. example: http://www.reddit.com/r/gifs/new.json

1. Website agent that grabs all titles and selftext (the description).
2. Trigger Agent that matches the title with my regex.
3. Pushover Agent that will send a group notification whenever receiving events from the trigger.

Loving Huginn!