Why do some things have to be so hard?

I have been asking myself that question for the better part of two weeks as I wrestle with SSIS 2008 R2 and getting data out of Oracle and into SQL Server.

It’s like the shell game, only with drivers.

Like, go native man!

Like anyone else working in SSIS and dealing with Oracle I started with the default drivers that ship with SSIS. Technically, they work. There are a couple of glaring caveats. They only work in 32 bit mode. Huge non starter on our 64 bit system. They are slow. I know that’s like saying the sky is blue, especially if you don’t have any context. Well I do have some context. Migrating packages from DTS to SSIS I’ve got historical run times and also did some test runs before actually converting the packages. The native 32 bit drivers were slower or just equal to the equivalent on the SQL Server 2000 box running the Oracle 8 drivers. They don’t return the proper metadata column data types. Everything comes back as a wide string a.k.a. st_wstr or varchar for you table creating types. Decimal(18,2)? Thats a varchar(50) for you. varchar(10)? You guessed it, varchar(50) should do it! This beyond anything else was probably the biggest problem.

No, only use what Oracle provides!

I decided to install the Oracle drivers. Let the pain begin!
First, you have to create an account on the Oracle Developer Network site. Really? I just need some drivers. I guess it could have been worse, like a sharp stick to the eye. You need to download the 32 bit and the 64 bit driver packs. Each one weighs in at 700MB compressed. They do include a ton of tools that I have no clue how to use, thats a bonus. Here is your sharp stick to the eye as you get to use of one of the worst installers in the history of installers. After about a dozen tries I finally found out the magic combination to get only the drivers cutting out about 1.3GB from being piled onto my server. Oh, and you get to do it twice. Next, as you look at where it put the drivers at you realize that each install is named client, client_1 and so on. No clue at all which is the 32 bit or 64 bit install bits. Get that figured out, you can go fix your borked path. The installer will gladly stick its path right at the beginning giving your hours of fun trying to figure out exactly what is broken. That is if your path isn’t already too long and it just skips this bit for you. And finally, you get to manually add a system variable pointing to your tnsnames.ora file usually stored in /app/<nt login>/11.2/client/network/tnsnames.ora

After all your hard work you are rewarded with Ole Db, ADO and ADO.Net drivers HUZZAH!
First thing I found out is the Ole Db drivers work as well as the native drivers as far as metadata is concerned. Performance was better. The ADO and ADO.Net do take it up a notch. You do get some additional metadata goodness from these drivers. I did get decimal and float types back but pretty much every string came back as st_wstr again. It would size correctly them that was nice. It did mean I had to add conversion from wide string to string. On a table with 86 columns and 64 million records this also was a bad combination. You do get to run in both 32 bit and 64 bit.

Use what random people on twitter recommend!

Well, not quite that bad. I posted a 14o character version of this post to the mighty #sqlhelp hashtag and lo’ my friend Merrill Aldrich (twitter|blog) and he simply said “Can you use the Attunity connector? #sqlhelp” Huh? When I did a search for just Attunity it brought me to their website http://www.attunity.com/ I didn’t see exaclty what Merrill was talking about. Doing a search for Attunity Connector brought me to the gold I’d been looking for http://www.attunity.com/products/attunity-connect/ssis-connectors-for-oracle-and-teradata. Apparently, Attunity makes connectors for Oracle and Teradata and releases them for F R E E. There are two versions currently 1.2 for SSIS 2008 R2 and 2.0 for SSIS 2012. The installer is rough but not Oracle rough. One of the nice things is installing the 64 bit drivers also installs the 32 bit drivers. Unless you have a problem with the installer and the 64 bit installer only installs the 32 bit drivers. After a few searches and a few more failed install attempts I found that you need to install the Visual C++ 2008 SP1 redistributable package. I only needed to do this on my server since I had Visual Studio 2010 already installed on my laptop. Once that was done the installer worked just fine. Except it really didn’t “install” everything. You still have to manually add them to your toolbox sidebar for data flows before you will see the source and destination connectors for Oracle. It’s totally worth it.

First off with a bang, all the metadata returned was 100% spot on. No more fussing with conversion steps or guessing what the data type should be. They are faster. Not by a small margin ether. On 50~ packages they were around 25% or more faster than the Oracle provided drivers. You still need to have a TNS names file you can’t use the machine name, port and service name directly. (that I know of)

And there was much joy to be had.

I do think it is sad that neither Microsoft or Oracle has a good solution to this issue. I’m glad Microsoft is supporting Attunity I wished they would ship them by default. As for Oracle, I know why Oracle developers and DBA’s get paid so much. If getting drivers installed was this hard I can’t imagine getting the whole database setup and going, ugh.