Category Archives: SQLDIY

Finding SQL Server Installs with Powershell, Again

But the SQL Listener isn’t started!

Yep, I heard you loud and clear. There is more than one way to find SQL Server on your network. This time we will dig deep into the TDS protocol docs again along with a bit of guess work.

Old Dog New Protocol Plus PowerShell

Last time we used PowerShell and the .net framework to assemble a UDP packet and send it to port 1434. This works out just fine if you have the SQL Browser running. If you are running a single standalone instance on the default port the browser service isn’t needed and may not even be enabled. This time we will be using TCP instead of UDP. This gets us around the browser service issue but also limits the information we can get back. As we dig through the documentation we can send what is known as a PRELOGIN packet. The server will respond with a few bits of information including the version of SQL Server that you are trying to log into. My problem is a simple one. I’m really not sure I’m decoding the return packet correctly. The only thing the docs say about it is it is a valid PRELOGIN response. It isn’t a complete LOGINACK packet, I’m not sure what it is. As soon as I know I’ll update this post. UPDATE: It is a PRELOGIN packet with a 0x04 header type and should conform to the PRELOGIN packet specification. What it does do though is prove if SQL Server is listening on a specific TCP port. As far as I can tell the version I return should be accurate for SQL Server 2005 and above. UPDATE: I have tested this on SQL Server 7.0 through SQL Server 2012 and it works for SQL Server 2000 and above. The version number is correct if the version number minor is above 255. I’m still working to decode the minor version number correctly. This module allows you to pass in a server name and any TCP port to check for SQL Server listening so even if someone has moved from the default of 1433 you can still scan other ports at your leisure  I would warn you that scanning every port on a machine will get you a visit from your network security folks. If you plan on doing a security sweep you should get everyone involved up front. Trust me on this one, they don’t think it is funny when you scan every IP and port on the network looking for SQL Server installs!

UPDATE: Thanks to Kendal Van Dyke(blog|twitter) a real PoSH guru AND SQL Server guy I’ve renamed the function following the PoSH standards of verb-noun.

<#
.SYNOPSIS
    Sends PRELOGIN to TCP port to get SQL Server Version.
.DESCRIPTION
    Does what it says.
#>
function Get-SQLServerVersionTCP{
    [cmdletbinding(
        DefaultParameterSetName = '',
        ConfirmImpact = "low"
    )]
    Param(
        [Parameter(
            Mandatory = $True,
            Position = 0,
            ParameterSetName = '',
            ValueFromPipeline = $True)]
        [string]$Computer,
        [Parameter(
            Mandatory = $False,
            Position = 1,
            ParameterSetName = '',
            ValueFromPipeline = $True)]
        [int]$Port,
        [Parameter(
            Mandatory = $False,
            Position = 2,
            ParameterSetName = '',
            ValueFromPipeline = $True)]
        [int]$ConnectionTimeout
    )

    begin {
        $ErrorActionPreference = "SilentlyContinue"
    }
    process {
        #get the ipaddress of the server
        $IPAddress = [System.Net.Dns]::GetHostEntry($Computer).AddressList[0].IPAddressToString
        #build and connect to TCPSocket
	    $TCPSocket = new-object System.Net.Sockets.TcpClient
        $TCPSocket.Connect($IPAddress, $Port);

        #attach socket to network stream
        $TCPStream = $TCPSocket.GetStream();

        # Build the prelogin packet see notes below
        $PreloginPacket =   0x12,0x01,0x00,0x34,0x00,0x00,
                            0x00,0x00,0x00,0x00,0x15,0x00,
                            0x06,0x01,0x00,0x1b,0x00,0x01,
                            0x02,0x00,0x1c,0x00,0x0c,0x03,
                            0x00,0x28,0x00,0x04,0xff,0x08,
                            0x00,0x01,0x55,0x00,0x00,0x00,
                            0x4d,0x53,0x53,0x51,0x4c,0x53,
                            0x65,0x72,0x76,0x65,0x72,0x00,
                            0x01,0x00,0x00,0x00;

        #send the prelogin packet
        $TCPStream.Write($PreloginPacket, 0, $PreloginPacket.Length);

        #wait a bit for the stream to build up
        Start-Sleep -m 1000

        # Buffer to store the response $ReturnData
        $ReturnData = new-object System.Byte[] 255

        # Read the TcpServer response
        $BytesReturned = $TCPStream.Read($ReturnData, 0, $ReturnData.Length)

        [string]$VersionNumber = $ReturnData[29].ToString()+"."+$ReturnData[30].ToString()+"."+(($ReturnData[31]*256)+$ReturnData[32]).ToString()
        return $VersionNumber
        $TCPStream.Close();
        }
}
Export-ModuleMember -Function Get-SQLServerVersionTCP

And for those who are REALLY curious my notes on assembling the PRELOGIN packet. This should work with SQL Server 2005 and above.

Packet Header
0x12 packet type 12 is prelogin
0x01 status 01 is end of message meaning this is the only packet
0x00,0x34 packet length
0x00,0x00 SPID the client spid for debugging from sql server to client
0x00 packetId unsigned char packetid is incramented by 1 modulo 256 this is currently ignored??
0x00 window currently not used just set it to 00
End Packet Header
Option Stream
0x00, This is the version number packet
0x00,0x15, 21st position
0x00,0x06, 6 bytes long
0x01, This is the encryption option
0x00,0x1b, 27th position
0x00,0x01, 1 byte long
0x02, INSTOPT the instance name
0x00,0x1c, 28th position
0x00,0x0c, 12 bytes long. This will vary currently hard set to MSSQLServer
0x01, ThreadId, currently hard set to 1
0x00,0x28, 40th position
0x00,0x04, 4 bytes long
0xff, Packet Terminator
End Option Stream
Begin Tokenless Stream
0x08,0x00,0x01,0x55, Major build number 0x00,0x00, Minor build number
0x00, No encryption
0x4d,0x53,0x53,0x51,0x4c,0x53,0x65,0x72,0x76,0x65,0x72,0x00, Instance name if known should be set if not default to MSSQLServer
0x01,0x00,0x00,0x00; ThreadId
End Tokenless Stream

The End?

Not quite. I’m working on another method that doesn’t rely on TCP/IP at all. I’ll let you guess what it might be.

ALL HAIL POWERSHELL!

Parsing Glenn Berry’s Diagnostic Information Queries With PowerShell

Do It In Bulk

I usually don’t run queries on just one server. Most of the time I need to get some piece of information from dozens if not hundreds of servers at a time. I also really like Glenn’s diagnostic queries and use them regularly with some modifications and additions of my own. I’m also really digging PowerShell too.

You Got Your Queries In My PowerShell

PowerShell is a very powerful way to gather information across lots of servers. I’ve got a set of templating files that drive my PowerShell scripts and thought I’d apply some of that to Glenn’s queries. The result is a function that takes a the file name of Glenn’s queries and outputs just the server level or the database level queries as an array that you can then loop through to query any number of servers or databases! It currently works with all three sets of files.

<#
.SYNOPSIS
Parse Glenn  Berry's DMV Diagnostic Queries into arrays http://sqlserverperformance.wordpress.com/
.DESCRIPTION
Does what it says so you can then loop through the array and execute the queries against a server ro database!
.PARAMETER FileName
FileName the name of the file you wish to process
.PARAMETER QueryType
QueryType ether server level or database level queries

.EXAMPLE
parseGABDMV '<path to sql file>\SQL Server 2005 Diagnostic Information Queries(December 2012).sql' server
#>
function parseGBDMV{
	[cmdletbinding(
		DefaultParameterSetName = '',
		ConfirmImpact = "low"
	)]
	Param(
		[Parameter(
			Mandatory = $True,
			Position = 0,
			ParameterSetName = '',
			ValueFromPipeline = $True)]
		[string]$FileName,
		[Parameter(
			Mandatory = $True,
			Position = 1,
			ParameterSetName = '',
			ValueFromPipeline = $True)]
		[string]$QueryType
	)
	Begin {
		$ErrorActionPreference = "SilentlyContinue"
		$found = $false
		$endquery = $false
		$startDatabaseQueries = $false
		$ServerQueries = @()
		$DatabaseQueries = @()
		$Query = "";
	}
	Process {
		Get-Content $File | foreach-Object {
			$_ = $_ -replace '\r\n$', "" #strip out empty lines
			$_ = $_ -replace '\t+', ""
			if ($_.Length -ne 0){
				if ($_.trim() -ne ""){
					if ($_.trim() -ne "`t"){
						$String = $_.toString();
					}
					else{
						$String = ""
					}
				}
				else{
					$String = ""
				}
			}
			else{
				$String = ""
			}

			if($found) {
				if($String -notlike "--*"){
					if($String.Trim() -ne ""){
						if($String.Length -ne 0){
							$Query += $String + "`r`n"
						}
					}
				}
			}

			if($String -like "--*"){
				if($String -cmatch '\(Query'){
					$found = $true
					$endquery = $true
					$Query = $Query -replace '\r\n$', ""
					if ($String.trim() -ne ""){
						if ($startDatabaseQueries){
							if (!$Query.Contains("USE YourDatabaseName")){
								$DatabaseQueries += $Query
							}
						}
						else{
							$ServerQueries += $Query
						}
					}
					$Query = ""
				}
			}

			if ($endqery) {
				$found = $false
			}
			if ($String -ne $null){
				if ($String.Contains("-- Database specific queries")){
					$startDatabaseQueries = $true
				}
				if ($String.Contains("-- Switch to user database *******************")){
					$startDatabaseQueries = $true
				}
			}
		}
		#dunno why always have a blank line on the first server query list
		$ServerQueries = $ServerQueries[1 .. ($ServerQueries.count-1)]
		if ($QueryType.ToUpper() -eq "SERVER"){
			return $ServerQueries
		}
		if ($QueryType.ToUpper() -eq "DATABASE"){
			return $DatabaseQueries
		}
	}
}

It may not be the cleanest PowerShell file parser on the planet but it seems to work and speed really isn’t an issue since the files are so small.

Enjoy!

SQLDIY: Index Usage Statistics

There are several different aspects of indexes that can be monitored via DMV’s. One the neatest things is just how much your indexes are used, how they are used or if they are used at all! I’ve put together this stored procedure to gather a ton of useful information on just how your indexes are being used. Using this information you can determine if the index in question is really needed, or if it is truly critical and shouldn’t be messed with. As will all DMV’s if your server was just rebooted you may see quite a number of unused indexes. By sampling data over time we can see when an index is actually called into use. Is it only really used once a quarter or once a year? Maybe we can disable it and then rebuild it before it is needed cutting down on maintenance and lessening the penalty for data modifications over the long term.

IsUsed – Simple, has your index ever been used.
IsExpensive – does it cost a lot to do updates or inserts into this index?

These columns all deal with how your index is accessed and updated. Is it used in a lot of scans, Maybe implying its used mostly for joins? Is it seek heavy, showing that your used are writing good WHERE clauses?

UserSeeks
UserScans
UserLookups
UserUpdates
LastUserSeek
LastUserScan
LastUserLookup
LastUserUpdate

AverageRecordSizeInBytes I look at this one to show me just how wide or narrow a particular index is. I also use it to help do detailed growth analysis, if we add X number of rows what size would the index grow too?
Series to Date SQLDIY: Manage and Monitor SQL Server Yourself

Link to the script Gather Index Usage Statistics

I’m hosting all my scripts from this series on GitHub as I do with all my open source projects.

As always, if you find any bugs please let me know and I will correct them!

SQLDIY: Index Fragmentation Levels

Indexes, indexes, indexes my kingdom for an index! Well, something like that. Indexes are the single most important structure in our relation database world. So, it is understandable that I do everything I can to gather information, monitor their health and maintain them to the best of my ability. One of the keys to index health is how badly fragmented the index is. Many people simply reindex everything on a regular schedule. While this may be fine for smaller indexes say under 50 megabytes but not so great for indexes that may be very large 400 megabytes or more. Knowing how your database is used is also needed if you are going to truly plan for the future and your index health. Is your table built with an identity for the primary key and only receives inserts? Or is it indexed on a GUID and has heavy deletes and updates? In the first case, depending on fill factor you may not need to do full reindex with a high level of frequency. In the second case, you may not be able to keep fragmentation levels under control without sacrificing the availability of your database. The problem is the larger the index the more difficult it becomes to manage from a reindexing strategy. When indexes get in the gigabyte range things like partitioning become as much a necessity for performance as well as maintenance tasks. For years I’ve written and maintained my own index maintenance scripts. Well, that all came to an end when Michelle Ufford put out her index defrag script.

Series to Date SQLDIY: Manage and Monitor SQL Server Yourself

Link to the script Index Fragmentation Levels

I’m hosting all my scripts from this series on GitHub as I do with all my open source projects.

As always, if you find any bugs please let me know and I will correct them!

Your Homework:

Index Fragmentation Findings: Part 1, The Basics Brent Ozar (blog|twitter)

Index Defrag Script v4.1 Michelle Ufford (blog|twitter)

SQLDIY: Statistics Information Including Last Update

One of the most important aspects of keeping your system running well is updating and managing your statistics. Since the optimizer relies on statistics to determine the best execution plan it is imperative that you keep an eye on them and maintain them just like you would indexes. One of the disappointing things in SQL Server 2005/2008 is the deprecation of some of the functionality of the sysindexes system view. In particular the rowmodctr column, since there is no equivalent in any of the new sys schema DMV’s. With that said, the GatherStatisticsInformation stored procedure listed here is missing that bit of information.What I do include is the last time the statistics were updated for a particular index. It is a rare thing to see information from SQL Server take a step backwards. Hopefully they will correct this mistake and add the modified row count back in to a DMV that is accessible.

Series to Date SQLDIY: Manage and Monitor SQL Server Yourself

Link to the script Gather Information On Statistics

I’m hosting all my scripts from this series on GitHub as I do with all my open source projects.

As always, if you find any bugs please let me know and I will correct them!

Your Homework:

Statistics Used by the Query Optimizer in Microsoft SQL Server 2008

SQL Server 2008 Statistics Jose Barreto (blog|twitter)

Lies, Damned Lies and Statistics Elisabeth Redei

How Stale are my Statistics? Kendra Little (blog|twitter)

And everything on SQL Skills