The best way of migrate MySQL to Elastic: L4 Data Migration Tool

Layer 4 (transport layer) based RDBMS to Elastic data migration tool
mysql-elastic-l4.png

How to use Elasticsearch and ELK (Elasticsearch + Logstash + Kibana) more strongly!

This tool is a transport layer (L4) based data migration tool for MySQL to Elastic.

Successfully transfer data from the relational database (RDBMS) to the Elastic Stack by replacing the L7 (application layer) -based data migration tool with the transport layer (L4).

Data can be migrated 200 to 10,000 times faster than conventional methods.

Whether you are planning to develop your own migration tool, or have developed a migration tool, but are still experiencing unsatisfactory performance.

* I need these people

1. If you used a message queue (Redis, Kafka, RabbitMQ, etc.) to transfer data from MySQL (or MariaDB) to the Elastic Stack

=> No programming effort required for message queues when using L4, avoiding the expectation and other adverse effects (slowing down) due to message queues

In order to achieve the purpose of 2.1, I confirmed the SDK document of MySQL and Elastic Stack directly and implements it with Python, Node.js and so on.

=> No need to know Elastic SDK when using L4

To accomplish the purpose of 3.1, convert MySQL data to JSON file, save it, and use Logastash Tail function or Elastic Filebeat.

=> No need to know complex functions of ELK when using L4

If you have used JDBC to accomplish the purpose of Section 4.1, but you are burdened with too high a memory footprint, long-term connection retention, and frequent migration failures (ex: logstash-input-jdbc)

=> Can use less than 1GB of memory when using L4

5. If you have been looking for a separate migration tool to achieve the goal of 1 (ex: go-mysql-elasticsearch)

=> There is no need to look for a separate migration tool when using L4

6. I have made the output of 2, 3, 4, 5, 6 to achieve the purpose of 1, but the data migration rate is very slow. (About 1,000 ~ 10,000 / day when using L7)

=> 200 ~ 10,000 times amplification when using L4

The L4 Data Migration Tool (MySQL to Elastic) reflects the migration issues that occurred during the MySQL to Elastic process from four companies.

* Characteristic

1. Ultra-fast RDBMS to Elastic (type: key-value / JSON) data migration based on L4

2. LAST ID tracking function (Prevent data duplication migration)

3. Support RPC (Remote Procedure Call) based on REST API

4. Immediate comparison of performance with and without L4-based migration. Compare with the stack of Elastic official documents!

Use Catswords Research’s L4 Data Migration Tool (MySQL to Elastic) for a successful data migration!

* Oracle DB Option

Oracle DB is also available. Please select from more options. Oracle2ES over L4 can be used like Oracle Golden Gate.

 

price: $450

contact: gnh1201@gmail.com

SMTP on TLS, 500 Unrecognized command

What happend?

  1. SMTP -> ERROR: AUTH not accepted from server: 500 Unrecognized command
  2. 500 TLS Negotiation Failed

 

Related products

PHPMailer 5.1, STMP, QSMTPD

 

Debug messages

[Options] PHPMailer 5.1: $mail->SMTPDebug = 4, PHP 7.0: display_errors->on, error_reporting->E_ALL

SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “220 mail.example.org ESMTP
SMTP -> get_lines(): $data is “220 mail.example.org ESMTP
SMTP -> FROM SERVER:220 mail.example.org ESMTP
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $str is “250-PIPELINING
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
SMTP -> get_lines(): $str is “250-8BITMIME
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
SMTP -> get_lines(): $str is “250 STARTTLS
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
250 STARTTLS
SMTP -> FROM SERVER: 250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
250 STARTTLS
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “500 Unrecognized command
SMTP -> get_lines(): $data is “500 Unrecognized command
SMTP -> ERROR: AUTH not accepted from server: 500 Unrecognized command
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “221 mail.example.org closing connection. Have a wonderful day.
SMTP -> get_lines(): $data is “221 mail.example.org closing connection. Have a wonderful day.
SMTP -> FROM SERVER:221 mail.example.org closing connection. Have a wonderful day.
SMTP Connect() failed.

or

500 TLS Negotiation Failed (when tls://mail.example.org)

Warning: stream_socket_enable_crypto(): SSL operation failed with code 1. OpenSSL Error messages:
error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed in /var/www/public/modules/Emails/class.smtp.php (in PHP 7.0, PHPMailer)

 

How to solve?

You have to test manually e-mail sending with authorization on TLS.

 

Step 1: Get authentication information

perl -MMIME::Base64 -e ‘print encode_base64(“\admin\@example.org\000fake.password”)’

B2RtaW5AZXhhbXBsZS5vcmcAZmFrZS5wYXNzd29yZA==

 

Step 2. Check your smtp server’s TLS support

$ telnet mail.example.org 587
Trying 127.0.0.1…
Connected to mail.example.org.
Escape character is ‘^]’.
220 mail.example.org ESMTP
ehlo testing
250-mail.example.org Hi Unknown [192.168.0.102]
250-PIPELINING
250-8BITMIME
250 STARTTLS

 

Step 3. Test your email sending by OpenSSL

$ openssl s_client -starttls smtp -crlf -connect mail.example.org:587
CONNECTED(00000003)
depth=1 C = XY, ST = unknown, L = unknown, O = QSMTPD, OU = CA, CN = 220e85afd863, emailAddress = postmaster@220e85afd863
verify error:num=19:self signed certificate in certificate chain
verify return:0

Certificate chain
0 s:/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=Server/CN=220e85afd863/emailAddress=postmaster@220e85afd863
i:/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=CA/CN=220e85afd863/emailAddress=postmaster@220e85afd863
1 s:/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=CA/CN=220e85afd863/emailAddress=postmaster@220e85afd863
i:/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=CA/CN=220e85afd863/emailAddress=postmaster@220e85afd863

Server certificate
—–BEGIN CERTIFICATE—–
MIIDcjCCAlqgAwIBAgIBAjANBgkqhkiG9w0BAQsFADCBjjELMAkGA1UEBhMCWFkx
EDAOBgNVBAgMB3Vua25vd24xEDAOBgNVBAcMB3Vua25vd24xDzANBgNVBAoMBlFT
TVRQRDELMAkGA1UECwwCQ0ExFTATBgNVBAMMDDIyMGU4NWFmZDg2MzEmMCQGCSqG
SIb3DQEJARYXcG9zdG1hc3RlckAyMjBlODVhZmQ4NjMwHhcNMTcwOTE5MDUyMDI0
WhcNMTkwOTE5MDUyMDI0WjCBkjELMAkGA1UEBhMCWFkxEDAOBgNVBAgMB3Vua25v
d24xEDAOBgNVBAcMB3Vua25vd24xDzANBgNVBAoMBlFTTVRQRDEPMA0GA1UECwwG
U2VydmVyMRUwEwYDVQQDDAwyMjBlODVhZmQ4NjMxJjAkBgkqhkiG9w0BCQEWF3Bv
c3RtYXN0ZXJAMjIwZTg1YWZkODYzMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
gQDHKMpOZTEjJdJenoD1bgNJ5+mH+xuC0bQTlDIUSYDP79CBa9q6y1hSrT3e8POQ
+hDQccA4WCFO6sUlfkKCdyCMOMitmnfuMrOV2tmECntX3lUGZasASg+3HVTz2Bdb
bNeJObX9jLZNBOOZF/14vff5Y7eYJKq15bNv3ISWqLEkawIDAQABo1kwVzAiBgNV
HREEGzAZgRdwb3N0bWFzdGVyQDIyMGU4NWFmZDg2MzAeBglghkgBhvhCAQ0EERYP
dGxzIGNlcnRpZmljYXRlMBEGCWCGSAGG+EIBAQQEAwIGQDANBgkqhkiG9w0BAQsF
AAOCAQEAK/ctHc6ciSmOxGRrBQ0pu+aMUu8Msl4Noa5U0pid+x8ErktM7ULdncIr
K4zmTeNKSKjciEc7VPtwaATLcWLQQp0y+Xzc+0KxapKsXVsCYaNlfXBzQfTx2vkX
cUtpQ2UqYKFIvOBMv7R+18+rmWj50BV9FxHZli1Y8+usms5tdsH/o18WytP2pyip
lVqla+wmb8synLh3lUuOlvprQSWWp1+Zh0wXVszHmfQmSNQuVlmRleU04IQo5Ye0
kjbiedcwgCT7mt3mi5lihsiNxfx/cvZ/8k512WEg8xA+FEORHoQdszWKUqNAtjZY
D4HyZo0r2S58LuMvQX2hfbPNZi7/kg==
—–END CERTIFICATE—–
subject=/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=Server/CN=220e85afd863/emailAddress=postmaster@220e85afd863
issuer=/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=CA/CN=220e85afd863/emailAddress=postmaster@220e85afd863

No client certificate CA names sent
Server Temp Key: ECDH, prime256v1, 256 bits

SSL handshake has read 2484 bytes and written 408 bytes

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 28C35D87DA8DC1661418259F5CD17456116E67C7CC41C2212FF23508F61220EC
Session-ID-ctx:
Master-Key: 2330D125C16D0131BEA156E62EC3EFF64AF46F2472F839B9F8D6E4BC98F1E56CD1EBC20E95F7E3C13201F126F7D6ED21
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 – 20 e1 a3 6d 87 d5 da 79-d2 30 2e 1c dc 93 bc 93 ..m…y.0……
0010 – 18 8d 8a b7 01 73 b0 71-3a d4 67 60 ef 2c 0b a6 …..s.q:.g`.,..
0020 – 21 3f a3 58 85 43 93 0a-04 52 cd b0 06 d6 af 3b !?.X.C…R…..;
0030 – 9f ff 3b 0f b8 ad 31 8b-50 3f ae 33 21 c0 92 a6 ..;…1.P?.3!…
0040 – b5 2f 52 79 eb b7 61 12-fd be 41 35 4d 14 51 86 ./Ry..a…A5M.Q.
0050 – 96 c6 9a 28 ac 5b 37 ce-db 08 65 0f 57 19 c3 67 …(.[7…e.W..g
0060 – 91 00 f1 33 fc ae 22 c1-89 23 42 a1 07 2e af 7d …3..”..#B….}
0070 – f7 c6 5f 0f d5 b9 47 7f-aa 85 eb 19 79 b5 16 7b .._…G…..y..{
0080 – 99 94 6a 03 15 a7 7f e9-8b a8 6e 8b 8f 74 28 f6 ..j…….n..t(.
0090 – ca 16 01 70 44 d1 72 d7-5b 0d a2 d6 d1 78 26 c2 …pD.r.[….x&.

Start Time: 1506060093
Timeout : 300 (sec)
Verify return code: 19 (self signed certificate in certificate chain)

250 STARTTLS
ehlo testing
250-mail.example.org Hi Unknown [192.168.0.102]
250-PIPELINING
250-8BITMIME
250 AUTH LOGIN PLAIN
AUTH PLAIN B2RtaW5AZXhhbXBsZS5vcmcAZmFrZS5wYXNzd29yZA==
235 PLAIN authentication successful for admin@example.org – auth success for admin@example.org
mail from: <admin@example.org>
250 <admin@example.org>, sender OK – how exciting to get mail from you!
rcpt to: <gnh1201@gmail.com>
250 <gnh1201@gmail.com>, recipient ok
data
354 go ahead
From: Admin <admin@example.org>
To: Customer <gnh1201@gmail.com>
Subject: Thank you for your interesting.

Hi Customer. Thank you for your interesting.
.
250 Queued! 1506060197 qp 697 <1619abc426b8214dd55ff1829d235633@missing.id>
quit
221 mail.example.org closing connection. Have a wonderful day.
closed

 

Step 4. Apply to PHP mail sender scripts (with PHPMailer 5.1)

 

part 1. Connection syntax.

before:

// connect to the smtp server
$this->smtp_conn = @fsockopen($host, // the host of the server
$port, // the port to use
$errno, // error number if any
$errstr, // error message if any
$tval); // give up after ? secs

after:

$socket_context = stream_context_create(array(‘ssl’ => array(‘verify_peer’ => false,’verify_peer_name’ => false,’allow_self_signed’ => true)));

$this->smtp_conn = stream_socket_client($host . ‘:’ . $port,$errno,$errstr,$tval,STREAM_CLIENT_CONNECT,$socket_context);

 

part 2. Crypto type

before:

stream_socket_enable_crypto($this->smtp_conn, true, STREAM_CRYPTO_METHOD_TLS_CLIENT)

after:

//Allow the best TLS version(s) we can
$crypto_method = STREAM_CRYPTO_METHOD_TLS_CLIENT;
//PHP 5.6.7 dropped inclusion of TLS 1.1 and 1.2 in STREAM_CRYPTO_METHOD_TLS_CLIENT
//so add them back in manually if we can
if (defined(‘STREAM_CRYPTO_METHOD_TLSv1_2_CLIENT’)) {
$crypto_method |= STREAM_CRYPTO_METHOD_TLSv1_2_CLIENT;
$crypto_method |= STREAM_CRYPTO_METHOD_TLSv1_1_CLIENT;
}

stream_socket_enable_crypto($this->smtp_conn, true, $crypto_method)

 

part 3. Send test email

[Options] host->tls://mail.example.org:587, smtp_auth->true

SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “220 mail.example.org ESMTP
SMTP -> get_lines(): $data is “220 mail.example.org ESMTP
SMTP -> FROM SERVER:220 mail.example.org ESMTP
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $str is “250-PIPELINING
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
SMTP -> get_lines(): $str is “250-8BITMIME
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
SMTP -> get_lines(): $str is “250 STARTTLS
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
250 STARTTLS
SMTP -> FROM SERVER: 250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
250 STARTTLS
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “220 Go ahead with TLS
SMTP -> get_lines(): $data is “220 Go ahead with TLS
SMTP -> FROM SERVER:220 Go ahead with TLS
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
SMTP -> get_lines(): $str is “250-PIPELINING
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
SMTP -> get_lines(): $str is “250-8BITMIME
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
SMTP -> get_lines(): $data was “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
SMTP -> get_lines(): $str is “250 AUTH PLAIN LOGIN
SMTP -> get_lines(): $data is “250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
250 AUTH PLAIN LOGIN
SMTP -> FROM SERVER: 250-mail.example.org Hi Unknown [192.168.0.101]
250-PIPELINING
250-8BITMIME
250 AUTH PLAIN LOGIN
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “334 VXNlcm5hbWU6
SMTP -> get_lines(): $data is “334 VXNlcm5hbWU6
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “334 UGFzc3dvcmQ6
SMTP -> get_lines(): $data is “334 UGFzc3dvcmQ6
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “235 LOGIN authentication successful for admin@example.org – auth success for admin@example.org
SMTP -> get_lines(): $data is “235 LOGIN authentication successful for admin@example.org – auth success for admin@example.org
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “250 <admin@example.org>, sender OK – how exciting to get mail from you!
SMTP -> get_lines(): $data is “250 <admin@example.org>, sender OK – how exciting to get mail from you!
SMTP -> FROM SERVER:250 <admin@example.org>, sender OK – how exciting to get mail from you!
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “250 <customer@example.org>, recipient ok
SMTP -> get_lines(): $data is “250 <customer@example.org>, recipient ok
SMTP -> FROM SERVER:250 <customer@example.org>, recipient ok
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “354 go ahead
SMTP -> get_lines(): $data is “354 go ahead
SMTP -> FROM SERVER:354 go ahead
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “250 Queued! 1506308867 qp 14594 <28ef7c771b6ac10e8bd5b252c1e56386@demo.example.org>
SMTP -> get_lines(): $data is “250 Queued! 1506308867 qp 14594 <28ef7c771b6ac10e8bd5b252c1e56386@demo.example.org>
SMTP -> FROM SERVER:250 Queued! 1506308867 qp 14594 <28ef7c771b6ac10e8bd5b252c1e56386@demo.example.org>
SMTP -> get_lines(): $data was “”
SMTP -> get_lines(): $str is “221 mail.example.org closing connection. Have a wonderful day.
SMTP -> get_lines(): $data is “221 mail.example.org closing connection. Have a wonderful day.
SMTP -> FROM SERVER:221 mail.example.org closing connection. Have a wonderful day.

Well done! Congratulations!

 

References

 

illegal sftp packet len, Received unexpected end-of-file from SFTP server

Problems:

  • illegal stfp packet len (FTP libraries)
  • Error: Received unexpected end-of-file from SFTP server (Filezilla)
  • This account is currently not available. (SSH client)

Solution:

  • Check your /etc/passwd by root.

user1:x:1000:1000::/home/user1:/sbin/nologin
user2:x:1001:1001::/home/user2:/bin/bash

You have to change /sbin/nologin to /bin/bash. and save it!

 

See also:

 

PHP 7.0: undefined symbol: php_libxml_node_free_list

What is this problem?

  1. Class ‘DomDocument’ not found
  2. undefined symbol: php_libxml_node_free_list

 

Command history:

username@hostname:~$ sudo apt-get install php7.0-xml

username@hostname:~$ sudo service apache2 restart

username@hostname:~$ php -m
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib/php/20151012/dom.so’ – /usr/lib/php/20151012/dom.so: undefined symbol: php_libxml_node_free_list in Unknown on line 0

PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib/php/20151012/xmlreader.so’ – /usr/lib/php/20151012/xmlreader.so: undefined symbol: dom_node_class_entry in Unknown on line 0

PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib/php/20151012/xsl.so’ – /usr/lib/php/20151012/xsl.so: undefined symbol: dom_node_class_entry in Unknown on line 0

PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib/php/20151012/dom.so’ – /usr/lib/php/20151012/dom.so: undefined symbol: php_libxml_node_free_list in Unknown on line 0

 

How to solve?

(In Ubuntu 14.04 or above)

Step 1:

sudo apt-get install libapache2-mod-php7.0 libphp7.0-embed libssl-dev openssl php7.0-cgi php7.0-cli php7.0-common php7.0-dev php7.0-fpm php7.0-phpdbg

 

Step 2:

sudo service apache2 restart

 

Quick command (copy & paste):

sudo apt-get install libapache2-mod-php7.0 libphp7.0-embed libssl-dev openssl php7.0-cgi php7.0-cli php7.0-common php7.0-dev php7.0-fpm php7.0-phpdbg php7.0-xml && sudo service apache2 restart

 

References

https://bugs.php.net/bug.php?id=74865

 

Trials and errors

  • (with rpm package) extract rpm, and register module
    • download php7.0-xml rpm package : rpmfind.net
    • extract rpm file ( rpm2cpio php-5.1.4-1.esp1.x86_64.rpm | cpio -idmv )
    • copy related files ( dom.so, dom.ini [extension=dom.so] )
    • apache2 restart

 

PAM Authentication failure of CHSH command with shadow password configuration (/etc/password)

# chsh [-s login_shell] [user] (example: chsh -s /bin/bash username)

chsh: PAM authentication failed

 

This error can be occur by invalid shadow password configuration. (/etc/passwd)

Generally, chsh command is run with root privilege.

but, It have riskiness that change login shell of root user if you wasn’t input username with invalid login shell path.

(Fatal example: # chsh -s /home/username [empty])

 

You have to check /etc/password configuration file.

# vi /etc/password

root:x:0:0:root:/root:/home/username

 

this part (/home/username) modify to /bin/bash (or your login shell)

 

Next, you try check chsh command.

How to remove VirtualBox completely?

Related post: VirtualBox: What is difference OSE vs FUEL? and following troubles.

If you want remove VirtualBox in order to solve something issues, you have to remove virtualbox completely.

but, sometimes(or always), package managers (such as APT, YUM, etc) can not remove VirtualBox completely. so, I wrote uninstall guide in this post.

This post based on Debian/Ubuntu distribution.

Tutorial:

  1. Configuration files and dependency packages
    1. VirtualBox OSE
      1. Check VirtualBox service’s process id: /bin/ps aux|grep VBoxSVC
      2. Kill process of VirtualBox service: sudo kill [process id]
      3. sudo apt-get remove –purge virtualbox
      4. sudo apt-get remove –purge dkms libgsoap5 libqt4-network libqt4-opengl libvncserver0 virtualbox-dkms xbase-clients
      5. sudo apt-get install virtualbox
    2. VirtualBox FUEL (version 5.1)
      1. Check VirtualBox service’s process id: /bin/ps aux|grep VBoxSVC
      2. Kill process of VirtualBox service: sudo kill [process id]
      3. sudo apt-get remove –purge virtualbox-5.1
      4. sudo apt-get remove –purge libqt5core5a libqt5dbus5 libqt5gui5 libqt5opengl5 libqt5printsupport5 libqt5widgets5 libqt5x11extras5 libsdl-ttf2.0-0 libxcb-render-util0 qttranslations5-l10n
      5. sudo apt-get install virtualbox-5.1 dkms (dkms not install automatically when install virtualbox-5.1)
  2. Tip: Version
    1. OSE: virtualbox (in OS distribution repository)
    2. FUEL: virtualbox-x.x (in Official provider’s repository)

References:

VirtualBox: What is difference OSE vs FUEL? and following troubles.

What is difference VirtualBox OSE vs VirtualBox FUEL?

  1. Virtualbox OSE
    1. Package Distributors: OS Distribution Repository (ex: apt, yum)
    2. Can get relative lower version
    3. Complied from opened source code
    4. Compiled by opensource community
  2. Virtualbox FUEL (by Oracle Company)
    1. Package Distributors: Oracle Company (Official provider)
    2. Can get latest version always
    3. Complied from opened source code with official provider’s know-hows.
    4. Complied by Oracle Company (Official provider)

DKMS compatible:

  1. VirtualBox included DKMS (type of kernel module), but It haven’t compatible between OSE and FUEL.
  2. OSE is provided by OS distribution repositories with opensource community. but it can make misunderstand easily about a little something. It can not support install latest Virtualbox Extension Pack(vbox-extpack), so maybe you have to reinstall to FUEL version after change package repository.

How to get FUEL version?

  1. You can get FUEL version: website: virtualbox.org

Thanks.

hwloc/LuaJIT header not found when build configure snort

Base on Debian Jessie or Ubuntu 16.04:

  • Trouble case 1: hwloc headers not found
  • Trouble case 2: luajit headers not found
  • Trouble case 3: openssl headers not found

How to troubleshooting:

  1. package name of hwloc library:  libhwloc-dev
    • sudo apt-get install libhwloc-dev
  2. install LuaJIT library by source:
    1. http://luajit.org/install.html
    2. make
    3. sudo make install
    4. sudo ln -s /usr/local/include/luajit-2.0/* /usr/local/include/
  3. install pkg-config
    • sudo apt-get install pkg-config
  4. package name of OpenSSL: openssl-dev
    • sudo apt-get install openssl-dev

You can install snort! Enjoy.

 

References: