Category Archives: Syndicated

Go, Go, Golang!

WHAT!

 

Yep, you read that right. I’m building tools in Go.

I can hear you now…. But Wes you are a SQL Server guy, a WINDOWS guy! What about C#? What about Powershell? What about ANY THING ELSE???

Making Decisions

Sometimes you make choices, not based on what you want to use but what you must use when building your tool set.

Here was my decision making path I hope it isn’t too complex.

It must run on everything from Windows Server 2003 through Windows 2012 R2 where I don’t have total control over the OS or what is installed on it.

That was it. I had no clue it would be so difficult to accomplish. Let me run down several candidates.

C#/.Net/mono

This is my default go to stack. I have been writing in C# for quite a while and I’m fairly comfortable with the language and love the amount of third party libraries and tooling around it. Oh and ReSharper. Microsoft is also pushing hard to open up C# and all aspects of .Net going so far as buying Xamarin, the developers behind mono. All of that is great! There is still one HUGE downside with the .Net framework. You have to have the .Net framework.

But Wes, the framework has been shipping for YEARS and is already installed on probably 90% of all the servers you are going to be using your tools on! You are absolutely correct, which version though? That is a horrible problem to solve. Pretty much the only way I found to solve it was to only use the 2.0 framework. That meant a HUGE part of the improvements to the .Net framework was now unavailable. Oh yeah, ever try to file a bug report to a third party library developer about a bug that only effects the 2.0 framework? You end up building “new” code and now maintaining legacy code from third parties.

Not all is lost! You can compile a “native app” but that can only target Windows 10, unless you use mono. With mono It is possible to build a stand alone self contained executable for Windows.

I’m saved!

Well…. The up side is I get the new shiny bits but mono doesn’t implement the full .Net framework. Also it is hit or miss on third party libraries and if they will also work with mono. Oh and the build process is right out of the 90’s. If all of that wasn’t enough there are possible licensing issues with embedding the mono run time in your application vs. linking to it as a dynamic library.

Powershell

See Above.

No seriously, it suffers from the same kind of problems. I am back to building scripts that target Powershell V1 and hoping security issues don’t blow up in my face.

Ruby/Python

I don’t want the Ruby or Python folks to flatten my tires, I’m not saying they are the same weakly typed, single-threaded interpreted language. Python doesn’t have Rails. ZING!

All kidding aside, I personally like Python over Ruby but that again is a personal preference. Honestly though I am more old school and really love strictly typed languages.

Both have TONS of third party libraries, that run on Linux or OSX. Windows is a bit thinner though. They both have ways of rapping up their respective run time into an object that looks like a standalone executable. There are several companies running both Ruby and Python on Windows and ship software installers to end users. I know it can be done. I also know it is kinda painful too.

Other Contenders

I didn’t stop there!

I looked again at Nodejs which has tooling to build command line applications. Having built a couple of applications using Nodejs and having no love for the “sideways mountain” that is the async call back pattern of Javascript I gave it a pass.
Lua, another favorite language of mine mostly due to is simplicity and ability to write and then read that same code again later.
Open Pascal/Delphi was given a cursory look but it just doesn’t have the critical mass it once had.
C++ is awesome if you want to crash both your program and the computer. I dare you to figure out what part of the 1000 page specification you aren’t going to use to maintain some level of productivity.
C has a huge draw for me but again it is a double-barrel shotgun but one barrel points backwards.

Finally, Go

I eventually found my way to Googles’ Go language. I had poked around with it several years ago but it was a pain to compile for Windows. Fast forward to 2016 and now Go compiles easily on Windows. Most third party packages are written in Go which means it generally compiles on Windows with zero changes. These executables are statically linked that means executables that will probably be larger than some but come with the bonus of not having to carry around all the run time and libraries or hope they are installed on the machine.
Go, as a language, is simple. It isn’t as terse as say K but it is much closer to C and Python than C# or C++ is.

Go is absolutely opinionated. There is the way I like to write code then there is the Go way. They don’t always agree. You want generics? NOT IN MY LANGUAGE YOU DON’T! Declare a variable and then don’t use it? No compile for you! Want to pick your own way of structuring your source code files? That ain’t going to fly.

These are all choices specifically made to satisfy writing LOTS of code worked on by LOTS of people, a Google’s worth so to speak.

So, why does this matter to me and the hand full of people that may actually have to work on my code?

More than you think. There are suggestions, fences, guide rails and walls in the Go ecosystem. They are there to help prevent bugs and improve productivity at scale. I have worked with a lot of programming languages over the years. My favorite ones allowed me to build stuff quickly and hand it to someone else knowing they could also get up to speed quickly. To that end I am finding that I agree with some of the guiding principles of Go.

Simplicity, safety and readability are paramount.
Minimal design: There is one way to write a piece of code.
It’s about expressing algorithms, not the type system.
Tooling is as important as the language.
Build times shouldn’t take over night.
Compiles naively across all platforms.
Things of interest should be easy; even if that means not everything is possible.

That means I have to give up some control. It also means Go may not be the best choice in all situations. To me Go is starting to feel like T-SQL, it isn’t horribly complicated yet tremendously powerful. I’ll be writing about the good and the bad around Go, or golang if you like, soon. I’m only in my first two weeks with it now.

Growing It At Home: Building Your Own Tools

Short On Money?

Sometimes that is the motivation. Maybe the tool you want doesn’t really exist yet? For me it was those things and the joy of building something useful. I’ve built dozens if not hundreds of tools in the form of scripts, batch files and executable code. Did you know some of the tools you are paying your employers hard earned money for came up from the same roots? Someone needed a tool to do X, and lo’ there were others who were willing to pay for that work.

Great, Someone Has Written The Tool…

They also want something called money for me to use it. That may be the case but some for profit companies are very open about what they do. One of my personal favorites is Confio’s Ignite. They have NEVER been dodgy about the data they gather or how they gather it. I’ve seen them in presentations say things like “you can gather this exact data from these DMV’s” and show people the query to pull that data. They will gladly show you their schema and how to write your own queries against it. It seems like they are giving away the secret sauce! Yet, they have a very awesome product that I and others are happy to pay for. Sometimes it isn’t just gathering the data it is everything else rapped around the data that is the real value.

I’m Still Broke, Now What?

If you read my last few blog posts you also now know there are a TON of opensource alternatives for the majority of the tools you would pay for. Some are of better quality, some aren’t. Some don’t do everything you want but maybe two of them together do meet your needs. Again, there isn’t a direct cost in dollars but there may be an indirect cost in your time. For me it isn’t just about the time spent but the enjoyment of building something new or building on someone else’s work to meet my specific needs. My overriding mantra for most opensource tools is “Good Enough”. In general, it may not have the polish of X commercial tool but it is good enough.

What I’m Working On Now?

Isn’t that the question of the day. I’ve currently got not one but two irons in the fire. One is an extension of an existing opensource project another is a project that was all my own and now I’ve started to incorporate some work done by others into it.

MSSQLCompressedBackup

This was a project I recently picked up based on a need I had at my work. As some of you know, I’ve got some experience with backup compression and encryption. We are a SQL Server shop. About 70% of my servers are still on 2005 at the moment. I also wanted to move our encryption to the backup level and not at our tape drive level. We really didn’t want to spend the extra money for one of the commercial products out there if we didn’t need to. Fortunately there was an opensource project that came close to meeting our needs. With a little work I was able to add some additional features like fast LZ compression and AES encryption. Is it as fast as all the commercial products out there? No, probably not. It is fast enough for us though. Does it have all the features of the commercial products out there? Again, that would be a big fat no. Right now though, we don’t need the extra bells and whistles that come with those products, if we do and I can’t find an opensource alternative and I can’t build the feature we need then we will look at the cost again.

Central Operations and Reporting Engine

I’ve been kicking this can around for the better part of 15 years. It really came it to focus at my time with The SCOOTER Store. At one point I came close to releasing a commercial version but the timing was off. That hasn’t stopped me from working on it and releasing bits of it in the form of my SQLDIY stuff. I’ve decided to quit just puttering around with it and go full bore again just to see what would happen. Previously everything was written in C# and T-SQL. It had it’s own scheduling engine, reporting engine and GUI. Most of it is woefully out of date now and I’ll be using powershell to do the heaving lifting where possible. I’m also working on a dashboard, web configuration tool and some other reporting bits to make it even more useful. When the project goes up on codeplex I’ll write a full post on it.

Digging Around For Free Tools – Google Code

Digging Around In Google Code

Google Code was very thin when it comes to SQL Server. I did find a few of interesting projects that may appeal to others as well as me. I will say there was a TON of use SQL Server with x programming language.

Project RoundhousE

“Professional Database Versioning and Change Management”

RoundhousE is an automated database deployment (change management) system that allows you to use your current idioms and gain much more.

It seeks to solve both maintenance concerns and ease of deployment. We follow some of the same idioms as other database management systems (SQL scripts), but we are different in that we think about future maintenance concerns. We want to always apply certain scripts (anything stateless like functions, views, stored procedures, and permissions), so we don’t have to throw everything into our change scripts. This seeks to solves future source control concerns. How sweet is it when you can version the database according to your current source control version?

This was probably the most well rounded project I’ve seen on any of the open source repository sites. It was also one of the most complete versioning and change management solutions as well. I also have seen it on one of the other repository sites but this one seemed more up to date.

NCrontab

This article shows how to use NCrontab to generate occurrences of a crontab-style schedule as a table in SQL Server (2005 or later), which can then be used in queries and especially joins to do interesting things.

This is a solid example of using the CLR for something other than regex. Documentation isn’t horrible and it is unique bit of kit.

dbrefactor

C# library for versioning and refactoring database structure using Microsoft SQL Server

I picked this one because it was a good idea but has been abandoned. This is one of those projects that could be revived by the right developer. Refactoring SQL Server schema and code can be a complete pain in the ass and any tool that makes that better is worth looking at, even if it has aged a bit.

 

Next up:

SourceForge

Digging Around For Free Tools – BitBucket

Digging Around In BitBucket

BitBucket was very thin when it comes to SQL Server. I did find a couple of interesting projects that may appeal to others as well as me.

BonSQL

Turn any version of Microsoft SQL Server into a platform capable of acting just like MongoDB – only you get transactions and the ability to join with traditional relational data as well!

This is a nice example of using newer methods with our good ol’ SQL Server engine.

DeploySQL

Scripts to auto-deploy SQL from SQL Server and to dump database objects to disk (so they can be searched or committed to source control).

There are lots of other projects like this that are designed to aid in versioning and source control. I picked this one because I didn’t want to have just one project from BitBucket 🙂

So, this is why I don’t search BitBucket very often. There just isn’t enough there for me to bother with.

Next up: Google Code

Digging Around For Free Tools – SQL Server

Tool Archaeology, Digging Through The Web

About every three to six months I will burn some time and just go digging for tools. Mostly, I’m looking for stuff related to SQL Server, PowerShell or C# development. There are lots of places to search through that could yield your next life saving tool. All the major sites have a search feature but if you just put in SQL Server you will get a ton of hits that don’t have anything to do with SQL Server. I usually do something like “SQL Server” NOT MySQL NOT Postgresql NOT DB2 NOT Oracle to keep the other databases from cluttering up the results. The other thing that can be difficult is how to filter out projects that don’t have any code or are very old. Most will allow you to sort by downloads or ratings and this can be a good indicator. Lastly, some of the sites don’t really enforce descriptions or comments on the project. Sourceforge is real hit or miss while Codeplex is really good about project descriptions. This can mean the difference in just reading about it and having to dig into the project to get any relevant details. Most of the time, if there isn’t any kind of project description I will just pass them by.

BitBucket

Open source host has some stuff but doesn’t seem to be a big one for Microsoft or SQL Server. I actually had to google Bing how to search BitBucket projects.

Google Code

One of the largest hosts for open source software, again not big on Microsoft or SQL Server. What I have found is lots of stuff to help you connect SQL Server to other things like Python and Node.js

SourceForge

The grand daddy of them all. There is a HUGE amount of projects. Unfortunately, most of them have gone dormant. There are gems to be had here for sure if you are willing do dig for them.

GitHub

The new hotness. The project mix is very good and they tend to be documented. Also, since GitHub isn’t that old you don’t have projects from the 90’s to filter out.

CodePlex

It’s a Microsoft world. You won’t find things like how to make SQL Server to work with X open source language or other tool but all of the projects have a description which is really nice.

My preferences

I spend most of my time on GitHub, SourceForge and Codeplex. I host my projects on GitHub and Codeplex. As far as GitHub is concerned the reason I chose them is I LOVE git. GitHub is easy to use and robust. I also choose Codeplex mainly because if you are Microsoft focused this will be one of the first places to search. Also, now that Codeplex supports git it is much easier to use from a project point of view.

Next we will take a look at some of the projects on each of them to give you an idea on what you can find if you dig hard enough!