Skip to content

Installation Guide

Jump to bottom
Gerald Venzl edited this page Jan 27, 2020 · 35 revisions

Welcome to the csv2db Installation Guide

csv2db is written in Python and can therefore run on any platform that supports Python 3.6+.

Table of Contents

Supported databases

Prerequisites

Installing Python

Installing csv2db

Installing the database driver(s)

Installing the Oracle Database driver

Installing the MySQL driver

Installing the PostgreSQL driver

Installing the SQL Server driver

Installing the Db2 driver

Supported databases

Prerequisites

Installing Python

First, make sure that you have Python 3.6 or higher installed. You can check whether it is by typing python --version in a terminal window. Depending on your setup, Python 3 might be installed as either python3 or python3.N where N is the number of the MINOR release, for example python3.6 or python3.7.

$ python3.7 --version
Python 3.7.0

If Python is not installed, meaning the terminal doesn't show an output similar to this, please go ahead and download and install it from python.org/downloads

You can also check out the following for advanced installation methods:

Installing csv2db

You can install csv2db in two ways. If you happen to have git on your machine, you can just clone this Git repository:

$ git clone https://github.com/csv2db/csv2db
Cloning into 'csv2db'...
remote: Enumerating objects: 183, done.
remote: Counting objects: 100% (183/183), done.
remote: Compressing objects: 100% (85/85), done.
remote: Total 183 (delta 89), reused 169 (delta 75), pack-reused 0
Receiving objects: 100% (183/183), 35.24 KiB | 7.05 MiB/s, done.
Resolving deltas: 100% (89/89), done.
$ cd csv2db/
$ ./csv2db -h
usage: csv2db [-h] {generate,gen,load,lo} ...

The CSV to database command line loader.
Version: 1.4.2
(c) Gerald Venzl

positional arguments:
  {generate,gen,load,lo}
    generate (gen)      Prints a CREATE TABLE SQL statement to create the
                        table and columns based on the header row of the CSV
                        file(s).
    load (lo)           Loads the data from the CSV file(s) into the database.

optional arguments:
  -h, --help            show this help message and exit
$

If you don't have git you can just download the latest release of csv2db from GitHub and unzip it.

$ unzip csv2db-1.4.2.zip
Archive:  csv2db-1.4.2.zip
69576b8f83ed7846582b8a635dba23dba92d4fee
   creating: csv2db-1.4.2/
  inflating: csv2db-1.4.2/.gitignore
  inflating: csv2db-1.4.2/CHANGELOG.md
  inflating: csv2db-1.4.2/LICENSE
  inflating: csv2db-1.4.2/README.md
  inflating: csv2db-1.4.2/csv2db
  inflating: csv2db-1.4.2/icon.txt
  inflating: csv2db-1.4.2/icon_transparent_1000x1000.png
  inflating: csv2db-1.4.2/icon_transparent_500x150.png
  inflating: csv2db-1.4.2/icon_white_1000x1000.png
  inflating: csv2db-1.4.2/icon_white_500x150.png
   creating: csv2db-1.4.2/src/
   creating: csv2db-1.4.2/src/python/
  inflating: csv2db-1.4.2/src/python/config.py
  inflating: csv2db-1.4.2/src/python/csv2db.py
  inflating: csv2db-1.4.2/src/python/functions.py
   creating: csv2db-1.4.2/test/
   creating: csv2db-1.4.2/test/python/
  inflating: csv2db-1.4.2/test/python/unit_tests.py
   creating: csv2db-1.4.2/test/resources/
  inflating: csv2db-1.4.2/test/resources/201811-citibike-tripdata.csv
 extracting: csv2db-1.4.2/test/resources/201811-citibike-tripdata.csv.gz
  inflating: csv2db-1.4.2/test/resources/201811-citibike-tripdata.csv.zip
  inflating: csv2db-1.4.2/test/resources/201812-citibike-tripdata.psv
  inflating: csv2db-1.4.2/test/resources/201812-citibike-tripdata.tsv
   creating: csv2db-1.4.2/test/resources/bad/
  inflating: csv2db-1.4.2/test/resources/bad/201811-citibike-tripdata-bad-data.csv
 extracting: csv2db-1.4.2/test/resources/bad/201811-citibike-tripdata-empty.csv
 extracting: csv2db-1.4.2/test/resources/bad/201811-citibike-tripdata-invalid.csv.zip
$ cd csv2db-1.4.2
$ ./csv2db -h
usage: csv2db [-h] {generate,gen,load,lo} ...

The CSV to database command line loader.
Version: 1.4.2
(c) Gerald Venzl

positional arguments:
  {generate,gen,load,lo}
    generate (gen)      Prints a CREATE TABLE SQL statement to create the
                        table and columns based on the header row of the CSV
                        file(s).
    load (lo)           Loads the data from the CSV file(s) into the database.

optional arguments:
  -h, --help            show this help message and exit
$

Installing the database driver(s)

You only need to install the driver(s) for the databases that you want to load data into. This can be done by using the Python built-in pip module. For the individual driver installations, see below.

Installing the Oracle Database driver

The Oracle Database driver requires the Oracle Instant Client libraries to be present in ~/lib.

$ mkdir ~/lib
$ cd ~/lib
$ mv ~/Downloads/instantclient-basic-*.zip ~/lib/
$ unzip instantclient-basic-*.zip
$ mv instantclient_*/* .
$ rm -r instantclient-basic-*.zip instantclient_*/

Once in place, you can install the Python driver itself:

$ python3 -m pip install cx-Oracle --user
Collecting cx-Oracle
Installing collected packages: cx-Oracle
Successfully installed cx-Oracle-7.2.2
$

See the cx-Oracle Installation Guide for further information.

Installing the MySQL driver

$ python3 -m pip install mysql-connector-python --user
Collecting mysql-connector-python
  Downloading https://files.pythonhosted.org/packages/ec/ce/0edafa96159f1035add21f1834a50ac89b1cbdc913b698c5e3faca0de82c/mysql_connector_python-8.0.14-cp37-cp37m-macosx_10_13_x86_64.whl (3.5MB)
    100% |████████████████████████████████| 3.5MB 3.2MB/s
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.7/site-packages (from mysql-connector-python) (3.6.1)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.7/site-packages (from protobuf>=3.0.0->mysql-connector-python) (1.12.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from protobuf>=3.0.0->mysql-connector-python) (40.2.0)
Installing collected packages: mysql-connector-python
Successfully installed mysql-connector-python-8.0.14
$

Installing the PostgreSQL driver

$ python3 -m pip install psycopg2-binary --user
Collecting psycopg2-binary
  Using cached https://files.pythonhosted.org/packages/fe/df/933e81c7fa95a915a9d67bd5736963a99513568f82cfc937c76d0d6f3414/psycopg2_binary-2.7.6.1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Installing collected packages: psycopg2-binary
Successfully installed psycopg2-binary-2.7.6.1
$

Installing the SQL Server driver

Mac OS X only: The pymssql driver uses the FreeTDS protocol. On Mac OS X only, this protocol needs to be installed first before installing the driver. If you happen to run on Linux, you can skip this step:

$ brew install freetds
==> Downloading https://homebrew.bintray.com/bottles/freetds-1.1.15_1.high_sierra.bottle.tar.gz
Already downloaded: /Users/gvenzl/Library/Caches/Homebrew/downloads/127cf28861035e48162898fce86cca291975b2d15822565e5171d12b40ac247e--freetds-1.1.15_1.high_sierra.bottle.tar.gz
==> Pouring freetds-1.1.15_1.high_sierra.bottle.tar.gz
🍺  /usr/local/Cellar/freetds/1.1.15_1: 2,141 files, 13.1MB
$

Install the SQL Server driver:

$ python3 -m pip install pymssql --user
Collecting pymssql
Installing collected packages: pymssql
Successfully installed pymssql-2.1.4
$

See the pymssql Installation documentation for further information.

Installing the Db2 driver

$ python3 -m pip install ibm-db --user
Collecting ibm-db
  Using cached https://files.pythonhosted.org/packages/3f/61/389c6decacfed61c2724005b8c2be61f2c699f731b53af13517e5a0e6e98/ibm-db-2.0.9.tar.gz
Building wheels for collected packages: ibm-db
  Running setup.py bdist_wheel for ibm-db ... done
  Stored in directory: /Users/gvenzl/Library/Caches/pip/wheels/ef/89/0f/54c86862cc233535d07492a6f58ad5905e9bf922fb27e72a68
Successfully built ibm-db
Installing collected packages: ibm-db
Successfully installed ibm-db-2.0.9
$

Clone this wiki locally