MacOSX command line tricks

MacOSX, , , , ,

Here's a list of MacOSX commands I had to search for all over the internet because I needed to use them lately and I'm sure I'm going to forget since I'm not a big OSX user. So here they are for when I'll need them again ... 🙂

In linux when you want to know which ports are opened and what applications listen on those ports you use netstat -lnp
In MacOSX you get the listening ports with this:

  1.  

Want to see what system call am application is making use strace, for MacOSX thats:

  1.  

Here's how to install an application that comes packaged in a .dmg:

  1.  

uninstall a package

  1.  

pidof required by some mysql scripts

  1.  

remount with noatime

  1.  

More to come ...

Better FIX for Inspiron N7110 touchpad

Linux, , ,

Seth Forshee created a kernel patch and now the ALPS touchapd on this laptop and probably others is recognized as a touchpad instead of falling back to a psmouse.

So now you can use the Touchpad tab in the "Mouse and touchpad settings" (gnome) to control the "click to tap", scrolling and other features and you don't have to use the patched syndaemon from my previous post.

To install this fix on ubuntu just download this deb package, install and reboot. ( tested on Ubuntu 11.10 x86_64 ).

If you want to know all the details go through the comments on this bug report #545307

Fix inspiron N7110 ALPS Touchpad in Ubuntu

Linux, , , , , , , ,

I recently purchased a new dell inspiron N7110. The laptop is great and Ubuntu 11.04 works quite well but there is one important problem.

The problem with most touchpads on laptops is that you'll often touch them accidentally while typing, this gets recorded as a tap/click and the typing cursor might move to another location and thus you might and up tying to a whole different place.

With synaptics touchpads or ALPS touchpads ( this is what N7110 has - ALPS Glidepoint) you can use syndaemon, a program that would run in background, monitor the keyboard and disable the touchpad while you type. But this program only works for touchpads which are being recognized as synaptics or ALPS touchpads. The touchpad on N7110 was recognized as a simple mouse, Xorg loaded the evdev driver instead of synaptics.

So I thought that maybe I can modify syndaemon to make it work with mice too. And I did. I noticed a lot of other people have the same problem so this could be useful even if you have a different touchpad that's also recognized as a mouse.
Download the patch for syndaemon here:
[download id="32"]

To apply the patch:

  1.  

The patch adds a new option to syndaemon to tell it to disable the mouse instead of a touchpad, without this the program will just exit when it can't find the touchpad.

  1.  

Don't forget to start it every time you start X.

Of course this is more like a quick hack then a real fix. A real fix would make Xorg or the kernel ( not sure exactly where the problem really is ) recognize this touchpad as a touchpad not as a mouse.

How to restore mysql replication

databases, , ,

Something went wrong and your mysql replication broke, I'm talking here about problems with the sql thread, not connection problems.

The sql thread shows you an error, what do you do to fix it and resume replication?

Here are 3 ways to fix it, each has advantages and disadvantages, pick the one that fits best to your problem.

1. Skip over the problem

You can try to just skip over the statement that broke the replication by changing the position in log file.

There are two ways to do this:

a) you can skip gradually

slave stop;
SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
slave start;
show slave status \G

That would skip the next 1 statement but you can set the counter higher to skip more the one.
Do it until the slave status shows the SQL thead is running.

b) skip to the current position

Use this is the first method keeps showing other statements that break replication and you don't have time to gradually skip statements.

First go on the master and type: show master status to find which is the current bin log file and the current position within the file.

Then go on the slave, stop it with "slave stop" and change the file name and position. Something like:

slave stop; 
CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.001958', MASTER_LOG_POS=52562937;
slave start;

But do that with your own file name and position taken from the master.

Check the replication status with "show slave status".
If the results are good ( both Slave_IO_Running and Slave_SQL_Running are Yes ) then you can go to the next step otherwise skip to next methods.

At this point you have a working replication but probably the data on the slave is not the same as on master since you skipped a few sql statements.

To fix it you can use maakit ( mk-table-checksum and mk-table-sync )

2. Full Dump and Restore

Connect to master, dump everything in a sql file, copy to replication slave and load it in mysql.

Use --master-data so the replication position is set in the dump file and the slave will know where to start.

Use --disable-keys so the slave will not try to build indexes after each insert and only built them at the end of the import.

Use --add-locks to surround each table dump with lock table/unlock table - this makes the inserts faster in the slave.

Problem:
--master-data will put a read lock on tables so operations on your master will lock waiting for the dump to finish. On large databases this can take a long time and it's unacceptable.

Possible fix:
If you have innodb tables add --single-transaction so a single global lock will be used only for a short time at the beginning of the transaction.

The problem is not so big if you can have filesystem snapshots on the master like the ones created by lvm.

3. Inconsistent Full Dump

This is just another fix for the problem at #1. Dump the data just like before but without using --master-data. This means no locks so the master can still work normally.
But because you don't use --master-data you will have to set the position in the slave yourself.
on the master type:

show master status \G

Take the file name and position and use them in the CHANGE MASTER statement on the slave ( after you load the dump file ) . Something like:

CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.001958', MASTER_LOG_POS=52562937;

Of course all of this will create an inconsistent slave but you can fix this easily with maakit.

If you know other methods I'd love to hear about them. Let me know in the comments.

Wammu backup to CSV for Gmail

Linux, , , ,

This is a modification to the wammu2csv.pl script that you can still find in google cache if you look for it.

The problem with the original script was that wammu ( version 0.35 anyway ) seems to generate backup files encoded in UTF-16 and the regular expressions in that script will not work unless the content is first converted from UTF-16.

Another benefit of the modified script is that now you don't have to convert the backup file from DOS to UNIX anymore.

To use this script:

  1. Connect to your phone using wammu
  2. Retrieve contacts from phone and save them in a file using the backup function
  3. download [download id="31"] and run:
    ./wammu2csv.pl <your-backup-file> > <your-csv-file>

    This will generate a CSV file that you can import in Gmail

Hidden Affiliate Links 1.0.1

wordpress, ,

I just released a new version of the hidden affiliate links plugin.

The previous version had a problem with the keyword replacement code.

This is fixed now so go ahead and download the update form the plugin page

I also updated the plugin page to list the requirements for this plugin to work.

This version was tested on WordPress 3.2.1

Google +1 button position

wordpress, , , ,

I'm using the Google+1 button plugin for wordpress to add a Google+1 button on my blog.

The plugin is great but it registers a filter to add this button and there are other plugins installed that also register their own filters on the body. What I would like is to have google+1 , facebook like and the official tweet button all on one line.

This would be easy if I would not have the contextual related posts plugin that puts the links to the posts right between the google+1 button and the facebook like botton.

Most plugins that add something to a post or page have a way of adding it manually, through a template keyword but this is not useful as I don't want to remember to add all the buttons at the end of each post.

The next solution would be to use the function that outputs the button code in the template. But this needed a small modification to the google+1 plugin.

Here's the patched version of the google+1 plugin version 1.0.4
[download id="28"]
And here's the unified diff file in case you want to apply it over your plugin and to see what really changed:
[download id="29"]

Now you can just use this code inside the posts loop ( after or before the the_content ):

  1.  

This was a quick change to get this done. A better solution would be to modify the plugin to allow you to set the filter order ( priority ) so you can control the position from wordpress admin like you can do it in the wp facebook like and wp tweet button plugin.

Oh and don't forget to +1 , like and/or tweet this post 🙂

WordPress multisite in subfolder

wordpress, , , ,

If you install wordpress in a subfolder like /home/example/public_html/wordpress instead of the main DocumentRoot ( usually /home/example.com/public_html ) you will not be able to enable multisite functionality but with a few small changes to wordpress code you can make it work.

My goal was to have the main site ( example.com ) with multisite, host blogs on subdomains like dom1.example.com and then use domain mapping so I could map a domain like dom1.com to dom1.example.com.

Here are the steps I took:

  1. Extract wordpress-3.1.3.zip in /home/example.com/public_html/wordpress, go to went to http://example.com/wordpress and install, then login as admin to http://example.com/wordpress/wp-admin
  2. In wp-admin -> Settings both WordPress address (URL) and Site address (URL) were set to http://example.com/wordpress, change Site address (URL) to http://example.com
  3. copy index.php and .htaccess from /home/example.com/public_html/wordpress to /home/example.com/public_html/
  4. open /home/example.com/public_html/index.php in an editor and changed the line:
    require('./wp-blog-header.php'); to require('./wordpress/wp-blog-header.php');
    At this point you have a working wordpress installed in a subfolder accessible at http://example.com
  5. Open wp-config.php and insert the following lines at the top ( can be right after the first line )
    define('WP_ALLOW_MULTISITE', true);
  6. Open the file wordpress/wp-admin/network.php in a text editor, look for the line: "function network_step1" and a few lines bellow there should be a line:
    1.  

    , replace it with

    1.  
  7. Now go to http://example.com/wordpress/wp-admin/network.php and create your network.
  8. After you create the network, WordPress will ask you to add a few lines to wp-config.php but you'll have to modify them a bit. Here's what I added:
    1.  

    By default SUBDOMAIN_INSTALL was set to false but you want subdomains, and $base was set to "/wordpress" but this needs to be "/"

    Don't forget to also add the required lines to .htaccess ( the one in the wordpress/ directory )

  9. After the network was created, wordpress messed up a bit with the permalink settings for the main domain. Before, the permalink structure was set to: /%year%/%monthnum%/%day%/%postname%/ and wordpress changed it to /blog/%year%/%monthnum%/%day%/%postname%/.
    But after I added the above lines to wp-config.php I was able to change the permalink structure back to what it was from Settings->Permalinks
  10. Adding domains

    Go to Network admin ( http://example.com/wordpress/wp-admin/network/ ) -> Sites -> Add new . I added
    an example domain dom1.example.com. This is what I'll use as an example from now on.

    After I added the domain I went to Sites , clicked the Edit link bellow the domain and changed Path from / to /wordpress ( with the option "Update siteurl and home as well" checked ) then go to the Settings tab and change the Home to http://dom1.example.com/
    At this point you have a working wordpress installed in a subfolder with multisite configured with subdomains.

  11. Install the domain mapping plugin with my changes
  12. Come back and let me know how it worked in the comments

WpMU Domain mapping in subfolder install

wordpress, , , ,

Here's a solution to run a wordpress multisite installation with domain mapping even if you didn't install wordpress in the DocumentRoot of your site.

If you want to map domains to subdomains in your wordpress install for example womsend.ro to ro.womsend.com you would use the wordpress-mu-domain-mapping plugin but this only works if wordpress is installed dirrectly in the web site's document root not in a subdirectory.

I like to have wordpress installed in a subdirectory because it keeps things organized especially if I have other things installed on your site.

So here is a patch to the wordpress domain mapping plugin that allows it to work with the subdirectory install. Download the original plugin then apply the patch (this patch is for the 0.5.4 version of the plugin):
[download id="26"]

Or just download the plugin already patched for convenience:
[download id="27"]

After adding a new site you have to go to Network Admin -> Sites, select the site and edit some of it's configuration values ( wordpress default values will not work ):

1) In site Info Tab, the Path is set to "/" . You have to add the subdirectory to it. For example if your site's document root is /home/site/public_html and wordpress is installed in /home/site/public_html/wordpress then the Path should be set to /wordpress
2) After saving the changes at #1, in the site Settings tab , Siteurl should already be set to http://{SubDomain}{Path} where {Path} is the path set at #1, but if it's not then set it.
3) The changes at #1 also modified ( in wp 3.1.3 anyway ) the "Home" value in the Settings tab , adding the Path to it but this has to be clean of the path so you have to change it back to http://{SubDomain}/

That's it, let me know how it works in the comments.

Does ISIPP/Suretymail really work?

Email, , ,

ISIPP ( the Institute for Social Internet Public Policy ) or SuretyMail is an accreditation program for email senders ( ESP's or not ). They run a DNS whitelist ( this was named IADB in the past ) and include the ips of their customers in it.

This whitelist can be used by any spam filter to lower the spam score of senders that are found in it. We know for sure that spamassassin comes with rules for checking the whitelist, but isipp doesn't disclose which other filters or ISPs use it.

On their homepage they claim their accreditation service works with all ISPs but is this really true?

The fact is that the big webmail providers ( yahoo, hotmail, gmail and aol ) don't use their whitelist. I know for sure that yahoo and hotmail use a different certification provider and that's the only one they use.

When Suretymail says their service works with all ISPs what they really mean is that they will try to help you get your email delivered to any ISP by trying to find the problems with your messages and tell you how to fix it. I know there's a lot of trying there :).

So let's see the pros and cons of ISIPP/Suretymail

Pro

  1. their whitelist is used by one of the most popular spam filters out there.

Cons

  1. Do you really need to be in spamassassin's whitelist? Most email service providers will have a "Spam check" feature that would check your message for known spam triggers including spamassassin's rules. So you can easily avoid most of spamassassin's rules or enough to get you under the 5 points of spam score anyway.

    Even if you don't use an email service provider that has such a feature you can always send a test message to an email account that has spamassassin on it and you'll see the rules your message triggers.

  2. Their whitelist is NOT used by any of the major ISPs ( yahoo, hotmail, gmail,aol ). Most email senders send between 35% and 90% of their messages to these ISPs so the accreditation doesn't work for a very big chunk of email addresses in your lists.
  3. Their "trying" to help you is really not so good.
    I was a customer for more then 1 year. Here's the stuff I had to put out with:

    • They didn't answer my messages, claiming it went to their spam folder. Isn't it ironic? 🙂 I had to send some of the support messages twice and ask them to confirm they received it after each message
    • They avoided answering some of the questions in my messages. Like when I was inquiring if yahoo, or some spam filters use their whitelist, or when I was inquiring if they can escalate my problem with yahoo
    • Whenever there's a problem they require you to fill a long form with a lot of useless questions even if you have just described the problem in detail in the previous message
    • Whenever there's a deliverability problem the first thing they suggest is modifying your email message content. And of course that almost never works.
    • These support discussions take weeks or even months sometimes. They say they are working on it but then don't communicate anything for weeks. I one case after not answering anything for a few weeks I sent them a message asking about any progress on the issue. They answered like they thought the problem fixed itself. On some cases by the time they decide to answer the problem does fix itself or you find out how to fix it without their help.
    • They claim your bounce and/or complaint rates are too high even if you're at 0.01% or less. This is their last resort, when nothing works blame the customer. Some ESP's will have no problem with their customers having bounce and/or complaint rates as high as 0.1%.
    • They pick on the bounce and complaint rates you have at other ISPs even if the ones for the ISP you have troubles with are fine.
    • They claim your complaint and bounce rates are very high even if you provide them with clear evidence that it's not as they claim. When you ask how they calculated the rates they don't answer. This was just a pretext to drop the accreditation when they realized their were exposed for not being able to help with yahoo deliverability problems.

And now ISIPP is offereing 50% off their application fee to Goodmail customers since Goodmail is shutting down. The problem here is these senders were using goodmail because they guarantied to have their customer's messages delivered to inbox at some big ISPs ( even yahoo till a while ago ), something that ISIPP will never be able to do.

Anyway this was my experience with ISIPP/Suretymail. If you had a similar or different experience with them feel free to share in the comments.