Sev17SQL Server | Sev17

Table-Valued Parameter Example

Chad Miller — Fri, 13 Apr 2012 19:34:53 +0000

I wanted show someone how to use table-valued parameters available in SQL Server 2008 and higher. The main use case of table-valued parameters is for sending a list or array of items as parameter to a SQL Server stored procedure or function. This is more efficient than parsing strings or XML on the SQL Server side. I couldn’t seem to find a complete example of table-valued parameters in the SQL Server documentation. The SQL Server docs only shows the T-SQL portion of the code and not the ADO.NET. I think its difficult to see how you would use this feature without having both the T-SQL and .NET code shown together so, here’s a simple T-SQL and Powershell script demonstrating table-valued parameters:

<#
/* FROM SSMS */
USE AdventureWorks
GO
/* Create a CustomerList table type */
CREATE TYPE Sales.CustomerList AS TABLE 
( CustomerID INT );
GO
 
/* Create a procedure to use new table type */
CREATE PROCEDURE Sales.uspGetCustomer
    @TVP CustomerList READONLY
    AS 
    SET NOCOUNT ON
    SELECT c.*
    FROM Sales.Customer c
    JOIN @TVP t ON
    c.CustomerID = t.CustomerID;
 GO
 
/* Test type and procedure in SSMS */
 
/* Declare a variable that references the type. */
DECLARE @CustomerTVP AS Sales.CustomerList;
 
/* Add data to the table variable. */
INSERT INTO @CustomerTVP (CustomerID)
SELECT * FROM (
	VALUES (1),(2),(3),(4),(5)
) AS v (CustomerID)
 
/* Pass the table variable data to a stored procedure. */
EXEC Sales.uspGetCustomer @CustomerTVP;
GO
#>
 
#FROM Powershell
#Create an ADO.NET DataTable matching the CustomerList Table Type:
$dt = new-object Data.datatable  
$col =  new-object Data.DataColumn  
$col.ColumnName = 'CustomerID'  
$col.DataType = [Int32]
$dt.Columns.Add($Col)
 
#Add a Row to the DataTable
$dr = $dt.NewRow()
$dr.Item('CustomerId') = 1   
$dt.Rows.Add($dr)  
 
#Add a 2nd Row to the DataTable
$dr = $dt.NewRow()
$dr.Item('CustomerId') = 2   
$dt.Rows.Add($dr)  
 
#Add a 3rd Row to the DataTable
$dr = $dt.NewRow()
$dr.Item('CustomerId') = 3   
$dt.Rows.Add($dr)  
 
#Connection and Query Info
$serverName="$env:computername\sql1" 
$databaseName='AdventureWorks' 
$query='Sales.uspGetCustomer' 
 
#Connect
$connString = "Server=$serverName;Database=$databaseName;Integrated Security=SSPI;" 
$conn = new-object System.Data.SqlClient.SqlConnection $connString 
$conn.Open()
 
#Create Sqlcommand type and params
$cmd = new-object System.Data.SqlClient.SqlCommand
$cmd.Connection = $conn
$cmd.CommandType = [System.Data.CommandType]"StoredProcedure"
$cmd.CommandText= $query
$null = $cmd.Parameters.Add("@TVP", [System.Data.SqlDbType]::Structured)
$cmd.Parameters["@TVP"].Value = $dt
 
#Create and fill dataset
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)
$null = $da.fill($ds)
$conn.Close()
 
#Return results
$ds.Tables

Gaining SQL Server SysAdmin Access

Chad Miller — Sat, 22 Oct 2011 19:51:18 +0000

I’ve seen this come a few times at work and I’m sure most you have experienced something similar.

Someone or an application installs SQL Server, doesn’t grant access to the DBA group and asks for DBA support.

In SQL Server 2008 and higher the built-in local administrators group is no longer automatically part of the SQL Server sysadmin role. You should add necessary logins to the sysadmin role as part of your SQL Server installation. Not automatically granting local administrators access to SQL Server is generally a good thing, however when the SQL Server installation is done by say another application then we see issues were support groups do not have access to SQL Server even though they are local administrators on the box. In the last few months I’ve seen this scenario several times and so has Argenis Fernandez (blog|twitter) as he has a helpful blog post entitled “Think Your Windows Administrators Don’t Have Access to SQL Server 2008 by Default? Think Again.” The post describes a technique of using the Sysinternals tool PsExec to gain SQL sysadmin access to SQL Server on which you already have local administrator access. The post also links to a documented way of starting SQL Server in single user mode in order to gain SQL Server sysadmin access (see “Troubleshooting: Connecting to SQL Server When System Administrators Are Locked Out).” And finally the post mentions, but does not demonstrate a method of using the Windows Task Scheduler. If you’re interested in the how’s and why’s this works and how different versions of SQL Servers are well, different in security settings defaults I encourage you to give the post and comments a read.

Armed with information on how to gain SQL Server administration I looked at the various options. The psexec utility is blacklisted in my environment, blocked from download and listed as an untrusted application. The approach of starting SQL Server in single user mode requires taking SQL Server down and since the application is a quasi-production system restarting SQL Server would have to be coordinated or done after hours. So, I chose to to use the Windows Scheduler trick. This should work on Windows 2003/XP and higher and I’ve tested on Windows 2008 and Windows 7. The typical UAC things apply—you’ll need to run as administrator. The script I came up with is listed below. I figured if I’m getting a server were some other group performed the installation or configuration there’s no guarantee PowerShell will be installed, so I’m going old school Windows batch file on this. It feels wrong to post a Windows batch file instead of a Powershell script on my blog, but I think using a batch file is the best approach for the problem. To use, save the script as AddDBA.bat and see the example syntax. The script must be run locally.

@echo off
@if "%1"=="?" goto Syntax
@if "%1"==""  goto Syntax
@if "%2"==""  goto Syntax
rem **********************************
rem Script AddDBA.bat
rem Creation Date: 10/21/2011
rem Last Modified: 10/21/2011
rem Author: Chad Miller
rem ***********************************
rem Description: Adds a Windows Account to SQL Sysadmin Role
rem Use when you have local Windows admin access but lost SQL Sysadmin access
rem ***********************************
 
@echo ************************
@echo *** ServerInstance: %1
@echo ************************
set TMPFILE=%TMP%\AddDBA-%RANDOM%-%TIME:~6,5%.tmp
schtasks  /Create /TN AddDBA /SC Once /ST 12:00 ^
/TR "sqlcmd -S %1 -Q \"CREATE LOGIN [%2] FROM WINDOWS; EXEC sp_addsrvrolemember [%2],[sysadmin];\" -o \"%TMPFILE%\" -e" ^
/RU "NT AUTHORITY\SYSTEM" /F
schtasks /Run /TN AddDBA
schtasks /Query /TN AddDBA /V /FO List
rem Wait 5 seconds
PING 127.0.0.1 -n 6  >NUL
rem Display output file
type %TMPFILE%
schtasks /Delete /TN AddDBA /F
goto :EXIT
 
:Syntax
@echo Syntax: AddDBA ServerInstance WindowsGroupOrLogin
@echo Example: AddDBA Z001\SQL1 Contoso\DBAGroup
goto :EXIT
 
:EXIT

Managing AlwaysOn with Powershell

Chad Miller — Mon, 05 Sep 2011 22:46:32 +0000

Although you can use SQL Server Management Studio or T-SQL to manage AlwaysOn, SQL Server Denali CTP 3 includes 25 cmdlet which together provide complete coverage for creating, confiiguring and administering the AlwaysOn database feature. In this post we’ll look at using Powershell to perform various management tasks for AlwayOn.

Note: This blog post describes features in SQL Server Denali CTP 3 which may change on final product release.

Getting Started

You’ll need a simple Windows 2008 R2 cluster with two standalone installs of SQL Server. I say simple because you don’t have to worry about shared storage, quorum disks and shared MSTDTC installations like you would in a traditional SQL Server installation on a Windows Server Failover Cluster. All you need are two servers running Windows Server 2008 R2 Enterprise Edition. For test purposes I’ve setup a two-node Windows Server Failover Cluster as follows:

1. Configure a Private virtual machine network for intra-cluster communication. Note this step is optional and not really necessary for a bare minimum cluster, but I setup it up anyways to mimic close to what I’ll have in production. This network uses a separate IP subnet than the Internal only network I had already setup in Hyper-V.

2. Setup the private only network which allows communication between virtual machines only.

Right-click virtual machine
Select Virtual Network Manager > Select Private > Add

3. Add the private network to each of the virtual machines.

Shutdown each machine
Select machine in Hyper-V Manager
Select Settings
Select Add Hardware and choose Network Adapter and click Add
Select the private network you created from the Network drop down list and click OK

4. On the virtual machines assign IP addresses under Network and Sharing Center. Here’s a table of my setup:

Machine	Internal Network	Private Network
Node1	192.168.1.71	192.1.1.2
Node2	192.168.1.72	192.1.13
DC1	192.168.1.50	N/A
Clusterxm*	192.168.1.70	N/A
Availability Group* Listener	192.168.1.73	N/A

DC1= Domain Controller

Cluster1 = cluster management IP (assigned during cluster configuration)

Availability Group Listener (assigned during AlwaysOn Availability Group Listener configuration)

*Don’t worry about these for now.

5. Since we’re using a two-node cluster without a quorum disk it is suggested to use a Node and File Share Majority so I’ll setup network share which is read/write accessible by the Cluster Service account. For my testing purposes I created share on my DC1 machine called \\DC1\Share1 located on DC1 C:\Share1 folder.

Setting Up Windows Failover Clustering

1. Add the Failover Cluster Manage feature to both modes by running the following PowerShell commands

import-module ServerManager
Add-WindowsFeature -Name Failover-Clustering

2. Create the cluster by running the following PowerShell commands on one node:

import-module FailoverClusters
new-cluster clusterxm -Node node1,node2 -StaticAddress 192.168.1.70 -NoStorage

3. Set the quorum mode to Node and File Share Majority by running the following command on one node:

 Set-ClusterQuorum -NodeAndFileShareMajority "\\DC1\share1"

Install SQL Server on Both Nodes

Install SQL Server and this important – As a standalone instance. Sorry no Powershell commands here just run through the installation screens. Be sure to set the SQL Server service account to a domain account (I had issues when using Local System).

Database Prerequisites

You need to have a database which is not already part of an AlwaysOn Availability Group in FULL recovery mode and has been backed up. As a test I’ll just use the old school pubs sample database. Run the instpubs.sql file and create a backup using Powershell.

Start SQL Server Management Studio and select “Start PowerShell” from Object Explorer. Run the following command to backup the database to the default backup directory:

PS SQLSERVER:\SQL\NODE1\DEFAULT\Databases\pubs> Backup-SqlDatabase -Database pubs

You’ll need to create a share accessible by both nodes for storing the SQL Server database and transaction log initialization backups. For my example I’ll create a folder called sqlrec under Node1’s C drive C:\sqlrec and share named sqlrec \\node1\sqlrec

AlwaysOn Powershell Documentation

The CTP3 version of Books Online contains some documentation and scripts for configuring AlwaysOn however as to be expected with pre-release software some topics are not covered and there are documentation errors in other sections. As I’ve encountered documentation errors I submitted Connect Items (see Connect Items below for details). Relevant helps topics included in CTP3 are listed below:

Specify the Endpoint URL When Adding or Modifying an Availability Replica

Enable and Disable AlwaysOn

Create and Configure an Availability Group

Rather than use the scripts includes with Books Online which only handle part of the configuration or write my own script I think its more important to demonstrate the commands to create and configure AlwaysOn. By approaching Powershell as simply running command versus vs. writing a script you’ll learn how to use Powershell commands for new administration functions. Once you’re happy with the results of the commands you then can turn the series of commands into a script. Let’s get started…

Using Powershell to Create and Configure AlwaysOn

Note: The following examples work within the SQLServer provider while connected to specific SQL Server machines. In my example I’m using machines named node1 and node2 running a default instance. Pay particular attention to the context in which the commands are run on either Node1 (primary) or Node2 (secondary).

Enable HADRService on both nodes:

PS SQLSERVER:\SQL\NODE1\DEFAULT> Enable-SqlHADRService -ServerInstance NODE1 -force
PS SQLSERVER:\SQL\NODE1\DEFAULT> Enable-SqlHADRService -ServerInstance NODE2 -force

Note: The force switch. Enabling or disabling HADR requires SQL Server service to be restarted. If you omit the force switch you’ll be prompted to confirm SQL Server restart.

Optionally confirm HADRService enabled on both nodes:

PS SQLSERVER:\SQL\NODE1\DEFAULT> get-item . | select IsHadrEnabled
 
IsHadrEnabled
-------------
True
 
PS SQLSERVER:\SQL\NODE1\DEFAULT> pushd
PS SQLSERVER:\SQL\NODE1\DEFAULT> cd SQLServer:\SQL\NODE2\DEFAULT
PS SQLSERVER:\SQL\NODE2\DEFAULT> get-item . | select IsHadrEnabled
 
IsHadrEnabled
-------------
True
 
PS SQLSERVER:\SQL\NODE2\DEFAULT> popd

Note: In order to retrieve the IsHadrEnabled property I need to to cd to node2 and in order to easily change directories back I’m using the pushd and popd commands to store the current location (pushd) and switch back (popd).

Configure HADR Endpoints

Configure HADR Endpoints and set state to started:

PS SQLSERVER:\SQL\NODE1\DEFAULT> cd .\Endpoints
PS SQLSERVER:\SQL\NODE1\DEFAULT\Endpoints> New-SqlHADREndpoint -Name "hadr_endpoint" -Port 5022
 
Name
----
hadr_endpoint
 
PS SQLSERVER:\SQL\NODE1\DEFAULT\Endpoints> dir | Set-SqlHADREndpoint -State "Started"
 
Name
----
hadr_endpoint
 
PS SQLSERVER:\SQL\NODE1\DEFAULT\Endpoints> cd SQLServer:\SQL\NODE2\DEFAULT\Endpoints
PS SQLSERVER:\SQL\NODE2\DEFAULT\Endpoints> New-SqlHADREndpoint -Name "hadr_endpoint" -Port 5022
 
Name
----
hadr_endpoint
 
PS SQLSERVER:\SQL\NODE2\DEFAULT\Endpoints> dir | Set-SqlHADREndpoint -State "Started"
 
Name
----
hadr_endpoint

Backup Database and Transaction Log

1
2
3

PS SQLSERVER:\SQL\NODE2\DEFAULT\Endpoints> cd SQLServer:\SQL\NODE1\DEFAULT
PS SQLSERVER:\SQL\NODE1\DEFAULT> Backup-SqlDatabase pubs \\NODE1\sqlrec\pubs.bak
PS SQLSERVER:\SQL\NODE1\DEFAULT> Backup-SqlDatabase pubs \\NODE1\sqlrec\pubs.trn -BackupAction Log

Create Replicas

Note: This doesn’t actually create the replicate, rather the –AsTemplate parameter allows you to create a definition of the replica which is stored in the $replica1 and $replica2 variables. These variables will be used when creating the availability group next.

1
2

PS SQLSERVER:\SQL\NODE1\DEFAULT> $replica1 = New-SqlAvailabilityReplica -Name NODE1 -EndpointURL "TCP://NODE1:5022" -AsTemplate -AvailabilityMode SynchronousCommit -FailoverMode Automatic  -ConnectionModeInSecondaryRole AllowAllConnections
PS SQLSERVER:\SQL\NODE1\DEFAULT> $replica2 = New-SqlAvailabilityReplica -Name NODE2 -EndpointURL "TCP://NODE2:5022" -AsTemplate -AvailabilityMode SynchronousCommit -FailoverMode Automatic -ConnectionModeInSecondaryRole AllowAllConnections

Create Availability Group

PS SQLSERVER:\SQL\NODE1\DEFAULT> New-SqlAvailabilityGroup AVGPubs -AvailabilityReplica ($replica1,$replica2) -Database pubs
 
Name                 PrimaryReplicaServerName
----                 ------------------------
AVGPubs              NODE1

Join Availability Group on Secondary Node

PS SQLSERVER:\SQL\NODE1\DEFAULT> pushd
PS SQLSERVER:\SQL\NODE1\DEFAULT> cd SQLServer:\SQL\NODE2\DEFAULT
PS SQLSERVER:\SQL\NODE2\DEFAULT> Join-SqlAvailabilityGroup -Name AVGPubs
PS SQLSERVER:\SQL\NODE2\DEFAULT> popd

Optionally Verify Availability Groups

PS SQLSERVER:\SQL\NODE1\DEFAULT> cd .\AvailabilityGroups
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups> dir | select -ExpandProperty AvailabilityReplicas | select name, ConnectionModeInPrimaryRole, ConnectionModeInSecondaryRole
 
Name                                                ConnectionModeInPrimaryRole           ConnectionModeInSecondaryRole
----                                                ---------------------------           -----------------------------
NODE1                                                       AllowAllConnections                     AllowAllConnections
NODE2                                                       AllowAllConnections                     AllowAllConnections

Restore Database and Transaction Log On Secondary Node

1
2
3

PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups> cd SQLServer:\SQL\NODE2\DEFAULT
PS SQLSERVER:\SQL\NODE2\DEFAULT> Restore-SqlDatabase pubs \\NODE1\sqlrec\pubs.bak  -NoRecovery
PS SQLSERVER:\SQL\NODE2\DEFAULT> Restore-SqlDatabase pubs \\NODE1\sqlrec\pubs.trn  -RestoreAction "Log" -NoRecovery

Add Database to Availability Group on Secondary Node

1 2	PS SQLSERVER:\SQL\NODE2\DEFAULT> cd .\AvailabilityGroups PS SQLSERVER:\SQL\NODE2\DEFAULT\AvailabilityGroups> dir \| Add-SqlAvailabilityDatabase -Database pubs

Create Availability Group Listener

Note: This is what you connect to (or a least that’s my impression) from your client machines. The listener provides a network name and IP Address which will failover between nodes.

cd SQLSERVER:\
PS SQLSERVER:\> New-SqlAvailabilityGroupListener -Name Network1 -StaticIp 192.168.1.73/255.255.255.0 -path SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs
Name                 PortNumber      ClusterIPConfiguration
----                 ----------      ----------------------
Network1             1433            ('IP Address: 192.168.1.73')

Optionally Verify Listener Connectivity

PS SQLSERVER:\SQL\NODE1\DEFAULT>cd SQLSERVER:\SQL\NODE1\DEFAULT
PS SQLSERVER:\SQL\NODE1\DEFAULT> Invoke-Sqlcmd -ServerInstance Network1 -Database master -Query "select @@servername"
WARNING: Using provider context. Server = NODE1.
 
Column1
-------
NODE1

Determine AlwaysOn Health

SQL Server includes three cmdlets for verifying the health of the various AlwaysOn components:

PS SQLSERVER:\SQL\NODE1\DEFAULT> get-command -module sqlps -Name test-*
 
CommandType     Name                                                Definition
-----------     ----                                                ----------
Cmdlet          Test-SqlAvailabilityGroup                           Test-SqlAvailabilityGroup [[-Path] <string   []>] [...
Cmdlet          Test-SqlAvailabilityReplica                         Test-SqlAvailabilityReplica [[-Path] <string   []>]...
Cmdlet          Test-SqlDatabaseReplicaState                        Test-SqlDatabaseReplicaState [[-Path] <string   []>...

The easiest way to run the test cmdlets is within the SQL Server provider context as follows:

PS SQLSERVER:\SQL\NODE1\DEFAULT> cd .\AvailabilityGroups
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups> dir | Test-SqlAvailabilityGroup
 
HealthState            Name
-----------            ----
Healthy                AVGPubs
 
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups> dir .\AVGPubs\AvailabilityReplicas | Test-SqlAvailabilityReplica
 
HealthState            AvailabilityGroup    Name
-----------            -----------------    ----
Healthy                AVGPubs              NODE1
Healthy                AVGPubs              NODE2
 
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups> dir .\AVGPubs\DatabaseReplicaStates | Test-SqlDatabaseReplicaState
 
HealthState            AvailabilityGroup    AvailabilityReplica  Name
-----------            -----------------    -------------------  ----
Healthy                AVGPubs              NODE1                pubs
Healthy                AVGPubs              NODE2                pubs

Manually Failing Over an Availability Group

To manually fail over an Availability Group we use the Switch-SqlAvailabilityGroup cmdlet. Interestingly enough I could not figure out a way to fail over the availability resource using the GUI in CTP3, so Powershell is the only way to I could do this for now which is fine by me

Note: Be aware the context in which you run the Switch-SqlAvailabilityGroup cmdlet. The cmdlet should be run from whichever node is functioning as the secondary as we will see in a moment:

1 2	PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups>cd AVGPubs PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs> Switch-SqlAvailabilityGroup

Running the switch-sqlavailabilityGroup command on the prmiary produces the following error:

Switch-SqlAvailabilityGroup : The local availability replica of availability group 'AVGPubs' cannot accept signal 'FAIL
OVER_PENDING' in its current role 'PRIMARY_NORMAL' and state (configuration is in Windows Server Failover Clustering st
ore, local availability replica has joined).  The availability replica signal is invalid given the current replica role
.  Verify that the signal is permitted based on the current role of the local availability replica, and retry the opera
tion.
At line:1 char:28
+ Switch-SqlAvailabilityGroup <<<<
    + CategoryInfo          : InvalidOperation: (:) [Switch-SqlAvailabilityGroup], SqlException
    + FullyQualifiedErrorId : ExecutionFailed,Microsoft.SqlServer.Management.PowerShell.Hadr.FailoverSqlAvailabilityGr
   oupCommand

At first I thought I had configured something incorrectly, but then was able to to failover the availability group through Failover Cluster Manager. It was then I realized this needs to be run from the context of the secondary node.

PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs> pushd
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs> cd SQLSERVER:\SQL\NODE2\DEFAULT\AvailabilityGroups\AVGPubs
PS SQLSERVER:\SQL\NODE2\DEFAULT\AvailabilityGroups\AVGPubs> Switch-SqlAvailabilityGroup
PS SQLSERVER:\SQL\NODE2\DEFAULT\AvailabilityGroups\AVGPubs> popd
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs> Switch-SqlAvailabilityGroup

Pausing and Resuming an Availability Group

You can pause and then resume synchronization of the Always database using the suspend-SqlAvailabilityDatabase and Resume-SqlAvailabilityDatabase cmdlets:

1
2
3

PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs> cd .\AvailabilityDatabases
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs\AvailabilityDatabases> dir | Suspend-SqlAvailabilityDatabase
PS SQLSERVER:\SQL\NODE1\DEFAULT\AvailabilityGroups\AVGPubs\AvailabilityDatabases> dir | Resume-SqlAvailabilityDatabase

Summary

This post has demonstrated the Powershell cmdlets available to help you manage your AlwaysOn configuration. The commands could be used to build a reusable script to provide a consistent configuration of AlwaysOn.

My testing was completed on SQL Server CTP3 as I encountered documentation issues I logged Connect Items. I’ve included a list of Connect Items below. There are other issues with the documentation which I did not log because of missing documentation rather than documentation bugs. I think not having all the documentation complete is to be expected while a product is still in CTP.

I’ll save my commentary on the AlwaysOn cmdlets for a future post, but for now I will say I’m impressed with the coverage and ease of use provided by the AlwaysOn cmdlets and SQL Server provider.

Connect Items

Determine Whether AlwaysOn Availability Groups is Enabled

Enable and Disable AlwaysOn (SQL Server) Documentation

Create and Configure an Availability Group (SQL Server PowerShell) Doc Error

New-SqlAvailabilityReplica cmdlt Allows Incompatible settings

Set-SqlAvailabilityReplica Cmdlet Does Not Take Pipeline input

New-SqlAvailabilityGroupListener Help Example Incorrect

Creating a Network Name for AlwaysOn Availability Groups Obsolete

Counting Licenses for VMs

Chad Miller — Tue, 28 Jun 2011 01:12:57 +0000

This is old news, but I’ve had to explain SQL Server VM licensing to enough people recently that I thought I’d put together a post.

In June 2009, Microsoft released an update to SQL Server licensing which can significantly lower TCO for SQL Server instances running under VMs for not only Hyper-V, but also VMWare.

http://download.microsoft.com/download/6/F/8/6F84A9FE-1E5C-44CC-87BB-C236BFCBA4DF/SQLServer2008_LicensingGuide.pdf is a good guide for helping here.

Microsoft S/W licensing like all software vendors is complex, so SQL Server virtualization licensing is best illustrated by looking at several examples:

Each scenario was calculated using the formulas in the referenced document ( starting on page 25)

In scenario #1 — 1 processor needed.

A. Number of virtual processors supporting the VM – Answer here is 2

B. Number of cores per physical processor (if hyper-threading is off) OR number of threads per physical processor (if

hyper-threading is on) – Answer here is 4

C. Number of physical processors – Answer here is 2

So this plugs into the formula

A / B = Number of licenses ( rounded up to whole number)

2 / 4 = 1 processor license needed.

Scenario # 2 — 1 processor license needed

A. Number of virtual processors supporting the VM – Answer here is 4

B. Number of cores per physical processor (if hyper-threading is off) OR number of threads per physical processor (if

hyper-threading is on) – Answer here is 8

C. Number of physical processors – Answer here is 4

So this plugs into the formula

A / B = Number of licenses ( rounded up to whole number)

4 / 8 = 1 processor license needed.

Scenario # 3 –1 processor license needed

A. Number of virtual processors supporting the VM – Answer here is 8

B. Number of cores per physical processor (if hyper-threading is off) OR number of threads per physical processor (if

hyper-threading is on) – Answer here is 8

C. Number of physical processors – Answer here is 4

So this plugs into the formula

A / B = Number of licenses ( rounded up to whole number)

8 / 8 = 1 processor license needed.

Microsoft has provided generous virtualization licensing which means a VCPU isn’t equal to a processor and in fact is significantly less. Looking at scenario #3 allocating 8 VCPU ‘s is equal one processor license!

How Does SQL Server VM licensing compare to Oracle and IBM?

I think it’s important to point out I am in no way a Microsoft zealot nor I am simply bashing non-Microsoft products. I consider myself a customer of each vendor and virtualization licensing is just one of many factors which should be considered in selecting a vendor along with product capabilities, cost, and skill set of people working on it.

What follows is my summary of how Oracle and IBM address virtualization licensing provided for comparison purposes to Microsoft. These are my opinions only and you should seek the advice of licensing experts whenever choosing a licensing strategy with any vendor.

Oracle

Oracle does not recognize virtualization—or at least most forms of virtualization not their own. Oracle has taken a tough stance on virtualization when compared to Microsoft and IBM as described in their own licensing documents. Oracle differentiates between what they call soft partitioning and hard partitioning.

Here’s an excerpt from Oracle licensing doc http://www.oracle.com/us/corporate/pricing/specialty-topics/index.html in regard to virtualization. Oracle Virtual Machine (OVM) and Solaris Containers are recognized as “hard partitioning” while VMWare is consider “soft partitioning” and is not.

Approved hard partitioning technologies include: Dynamic System Domains (DSD) — enabled by Dynamic Reconfiguration (DR), Solaris 10 Containers (capped Containers only), LPAR (adds DLPAR with AIX 5.2), Micro-Partitions (capped partitions only), vPar, nPar, Integrity Virtual Machine (capped partitions only), Secure Resource Partitions (capped partitions only), Static Hard Partitioning, Fujitsu’s PPAR. Oracle VM can also be used as hard partitioning technology only as described in the following document: http://www.oracle.com/technology/tech/virtualization/pdf/ovm-hardpart.pdf.

This is why you aren’t seeing a lot of virtualization in Oracle. As an example you can use VMWare (there may be some support issues to consider), but because VMWare is a soft partition in Oralce’s view you’d need to license the entire VMWare cluster in order to virtualize a single Oracle server or use one of the hard partitioning technologies described above. One thing I find interesting is that OVM is a Xen based hypervisor which is similar to VMWare yet one’s considered hard partitioning and the other soft partitioning. Needless to say as an Oracle customer I’m not happy with their virtualization licensing stance. As more and more Oracle customers move to virtualization I don’t think this is a sustainable strategy. It is my belief due to customer demand at some point Oracle will need to change its stance on virtualization.

IBM

IBM recognizes virtualization capacity licensing, now IBM licenses on a per core basis instead of a per socket basis like SQL Server. Generally a VCPU is equal to one core, but may not be. See the licensing documentation for specific virtualization technology guidance:

IBM Virtualization Capacity License Counting Rules

Keep in mind IBM uses something called Processor Value Units or PVU’s when calculating licensing. different processor chips may have different PVU multipliers. As an example there’s a higher PVU for Nehalem or higher processors than pre-Nehalem on the Intel platform. The PVU calculations are used in determining virtualization licensing. See http://www-01.ibm.com/software/lotus/passportadvantage/pvu_licensing_for_customers.html for further information.

Summary

Microsoft provides a generous interpretation on virtualization licensing when compared to their competitors. It should be noted one thing I didn’t discuss is the scenario where you license the entire virtualization hosts (physical cluster). My understanding is that this is an acceptable licensing scenario with Microsoft, IBM or Oracle (again check with the vendor). When licensing an entire cluster you’re able to setup an “all you can eat” scenario where you can deploy as many VM’s you want without having to license at the VM level even if you don’t do this on the database-level it may make sense to do so on the operating system level.

Software licensing is complex subject with any vendor so if there are any clarifications, omissions, or anything I misstated, please post a comment.

What’s Going on with SQL Server Service Account Changes?

Chad Miller — Fri, 10 Dec 2010 03:44:27 +0000

For years we’ve been told you should use Enterprise Manager in SQL Server 2000 or SQL Server Configuration Manager in SQL 2005 or higher to change the SQL Service account. Why you may ask well, because according to the documentation Configuration Manager does “magic” stuff to re-ACL things for the new account. I decided to put the notation that SQL Server files, folders or Registry keys get Re-ACL’d when Enterprise Manager (SQL 2000) or SQL Server Configuration Manager (SQL Server 2005 or higher) is used to change the SQL Server service account to a few tests…

Setup

Use a test machine (even your local workstation or laptop will work)
Note: Since I’m using test machines the SQL instances were setup and running under Local System account and then changed to an AD account with no assigned permissions or groups other than the default AD Users group.
Create a test local or AD account, in my case I created an account named “ZZZ”
We’ll need the SID for the account when we dump the ACLs, from a PowerShell host obtain the SID for the account using the following commands:

$domain = "CORP"
$user = "ZZZ"
$ntAccount = new-object System.Security.Principal.NTAccount($domain, $user)
$sid = $ntAccount.Translate([System.Security.Principal.SecurityIdentifier])
$sid.value

For my account this returns:

S-1-5-21-3799912004-3082830092-2763560395-1116

The Tests

For SQL Server version 2000, 2005 and 2008 I’ll use the recommended utilities to change the service account.
I’ll then run a the following PowerShell commands to export the ACL’s for all SQL Server files, folders and registry keys:

#FileSystem
cd "C:\program files\Microsoft SQL Server"
dir -recurse | get-acl | select pschildname, owner, sddl | export-csv -NoTypeInformation -Path $home\sqlfile.csv
#Registry
cd "HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server"
dir -recurse | get-acl | select pschildname, owner, sddl | export-csv -NoTypeInformation -Path $home\sqlreg.csv

Finally I’ll look for any ACL’s assigned to the SID of my new service account (S-1-5-21-3799912004-3082830092-2763560395-1116).

SQL Server 2000 Enterprise Manager

From SQL Server 2000 Enterprise Manager change the service account to the new account. The GUI will prompt for SQL service restart.
After SQL Server has restarted, run the PowerShell commands to export ACL’s as instructed in “The Tests” section.

I’ve posted my output files SQL2K_sqlfilefull2.csv and SQL2K_sqlregfull2.csv in the accompanying download.

Looking at the ACL’s in the CSV file we see, yes in fact Enterprise Manager did re-ACL both folders, files and registry keys. A typical entry includes the file/folder name and the following SDDL syntax with the service account’s SID:

…(A;OICI;FA;;;S-1-5-21-3799912004-3082830092-2763560395-1116)…

Checking off Enterprise, we’ll move to SQL 2005 and 2008…

SQL Server Configuration Manager (2005/2008)

Since the results of both SQL Server 2005 and 2008 were the same, I’ll summarize the steps and results together.

From SQL Server Configuration Manager change the service account to the new account. The utility will prompt for a SQL Server restart.
After SQL Server restarts run the PowerShell commands to export ACL’s as instructed in “The Tests” section.

I’ve posted my output files SQL2K5_sqlfilefull2.csv, SQL2K8_sqlfilefull2.csv, and SQL2K5_sqlregfull2.csv and SQL2K8_sqlregfull2.csv in the accompanying download.

This time, when we look at the ACL’s only two files show a change not in the permissions, but in the owner of the file ERRORLOG and log_*.trc (the current default trace file). The change in file ownership isn’t surprising as the new service account will create the files on restart which has nothing to do with re-ACL-ing. Also there aren’t any changes in ACL’s for the registry. As an additional check I verified the service account wasn’t auto-magically added to any of the built-in Windows groups SQL Server 2005 or higher creates on the local machine. EDIT: Dan Jones posted a blog entry clarifying the behavior which I verified in SQL Server 2005/Windows 2003 the account is added to SQLServer2005MSSQLUser% local group by SQLCM. As noted by Dan this is OS and SQL version specific behavior where if you are running pre-Vista (Windows 2003/XP) or SQL 2005 group membership is modified. So, where’s the “magic” re-ACL-ing?

Summary

According to my tests none of the ACL’s were changed in SQL 2005 or higher while in SQL 2000 they were. So is the BOL documentation wrong or am I missing something? Or is the “Only use SQL Server Configuration Manager” directive a “SQL Myth?”

Download

Scripting Guy Guest Blog Post

Chad Miller — Tue, 02 Nov 2010 03:26:42 +0000

Ed Wilson (Blog|Twitter) aka Scripting Guy is kicking off a SQL Server week (Nov 1st 2010) with my guest blog post. The post, Use PowerShell to Collect Server Data and Write to SQL, demonstrates my number one use case for database related PowerShell activities—loading data into a SQL Server table. I’ve used the techniques described in the post to build many solutions including:

Each problem comes down to one thing—get the data into a SQL Server database! Learn the basics and you can easily create your own PowerShell-based solutions.

Building A Remote Desktop Manager Cluster List

Chad Miller — Wed, 30 Jun 2010 20:37:29 +0000

The Microsoft Clustering and High Availablity bloggers have taken noticed of the Remote Desktop Connection Manager (RDCMan) utility from a cluster management perspective. I’ve previously blogged about Building A Remote Desktop Manager Connection List from the SQL Server Central Management Server (CMS) tables. The RDCMan use case for cluster administration is very useful and it just so happens that I built a similar solution for cluster servers. So, as a follow up here’s a way to auto generate an RDCMan file from a list of clusters, nodes and virtuals.

First create and populate a cluster, cluster_node and cluster_virtual tables defined in the following T-SQL script file:

Note: You’ll need to insert your cluster information into the three tables.
Next use the following T-SQL XQuery to generate the RDCMan XML and save the file as cluster.rdg.

Finally open the cluster.rdg file in Remote Desktop Connection Manager. RDCMan allows to you to have multiple rdg files open at one time which means our previous CMS-based list of SQL Server and our cluster-based list can both be used simultaneously.

Enjoy!

Building A Remote Desktop Manager Connection List

Chad Miller — Tue, 15 Jun 2010 19:02:51 +0000

Remote Desktop Connection Manager or RDCMan is a free download from Microsoft for managing multiple remote desktop connections. The functionality provided by RDCMan is above and beyond what you’ll find in the built-in Remote Desktop MMC and comparable to other 3rd party Remote Desktop utilities. If you’re connecting to multiple remote desktop sessions, RDCMan is a tool worth looking into.

Because my environment has hundreds of servers, one of the things I look for in management tools is a method to import an existing list of servers. Fortunately the XML file called an RDCMan file used for the connection list in RDCMan is a simple structure you can build yourself. A quick web search turns up a post by Jan Egil Ring titled Dynamic Remote Desktop Connection Manager connection list. The approach Jan takes builds an RDCMan file from the computers in Active Directory using PowerShell which isn’t quite what I want, but it gives me an idea on using T-SQL/XQuery…

As a SQL Server DBA I maintain a list of SQL Servers and groups in a Central Management Server or CMS. A CMS is not only useful for maintaining a central list of SQL Servers, driving Policy Based Managment, and executing multi-server queries but can also be used as input for other things including building an RDCMan file. Because the CMS data is already in SQL Server tables it’s easier to use a T-SQL approach to shape the XML. The following T-SQL query creates an RDCMan XML file from CMS servers and groups. The same servers and groups in your CMS will be represented in the resulting RDCMan file:

DECLARE @xml XML
 
;WITH [file] AS
(SELECT
1 AS 'RDCMan'
,2.2 AS 'version'
,'CMS' AS 'name'
,'True' AS 'expanded'
,'Generated from CMS' AS 'comment'
,'FromParent' AS 'logonCredentials'
,'FromParent' AS 'connectionSettings'
,'FromParent' AS 'gatewaySettings'
,'FromParent' AS 'remoteDesktop'
,'FromParent' AS 'localResources'
,'FromParent' AS 'securitySettings'
,'FromParent' AS 'displaySettings'
)
,grp AS
(SELECT
 server_group_id
,name
,'True' AS 'expanded'
,'Generated from CMS' AS 'comment'
,'FromParent' AS 'logonCredentials'
,'FromParent' AS 'connectionSettings'
,'FromParent' AS 'gatewaySettings'
,'FromParent' AS 'remoteDesktop'
,'FromParent' AS 'localResources'
,'FromParent' AS 'securitySettings'
,'FromParent' AS 'displaySettings'
FROM msdb.dbo.sysmanagement_shared_server_groups_internal
WHERE is_system_object = 0)
,srv AS
(SELECT
 DISTINCT server_group_id
,CASE
	WHEN PATINDEX('%\%',name) > 0 THEN
		SUBSTRING(name,1, (PATINDEX('%\%',name) -1 ))
	WHEN PATINDEX('%,%',name) > 0  THEN
		SUBSTRING(name,1, (PATINDEX('%,%',name) -1 ))
	ELSE
		name
END AS name
,CASE
	WHEN PATINDEX('%\%',name) > 0 THEN
		SUBSTRING(name,1, (PATINDEX('%\%',name) -1 ))
	WHEN PATINDEX('%,%',name) > 0  THEN
		SUBSTRING(name,1, (PATINDEX('%,%',name) -1 ))
	ELSE
		name
END AS 'displayName'
,'Generated from CMS' AS 'comment'
,'FromParent' AS 'logonCredentials'
,'FromParent' AS 'connectionSettings'
,'FromParent' AS 'gatewaySettings'
,'FromParent' AS 'remoteDesktop'
,'FromParent' AS 'localResources'
,'FromParent' AS 'securitySettings'
,'FromParent' AS 'displaySettings'
FROM msdb.dbo.sysmanagement_shared_registered_servers_internal)
 
SELECT @XML = (
SELECT
(SELECT
 name
,expanded
,comment
,(SELECT logonCredentials AS "@inherit" FOR XML PATH('logonCredentials'), TYPE)
,(SELECT connectionSettings AS "@inherit" FOR XML PATH('connectionSettings'), TYPE)
,(SELECT gatewaySettings AS "@inherit" FOR XML PATH('gatewaySettings'), TYPE)
,(SELECT remoteDesktop AS "@inherit" FOR XML PATH('remoteDesktop'), TYPE)
,(SELECT localResources AS "@inherit" FOR XML PATH('localResources'), TYPE)
,(SELECT securitySettings AS "@inherit" FOR XML PATH('securitySettings'), TYPE)
,(SELECT displaySettings AS "@inherit" FOR XML PATH('displaySettings'), TYPE)
FROM [file]
FOR XML PATH('properties'),TYPE)
 
,(SELECT 
 
 (SELECT
 g.name
,g.expanded
,g.comment
,(SELECT g.logonCredentials AS "@inherit" FOR XML PATH('logonCredentials'), TYPE)
,(SELECT g.connectionSettings AS "@inherit" FOR XML PATH('connectionSettings'), TYPE)
,(SELECT g.gatewaySettings AS "@inherit" FOR XML PATH('gatewaySettings'), TYPE)
,(SELECT g.remoteDesktop AS "@inherit" FOR XML PATH('remoteDesktop'), TYPE)
,(SELECT g.localResources AS "@inherit" FOR XML PATH('localResources'), TYPE)
,(SELECT g.securitySettings AS "@inherit" FOR XML PATH('securitySettings'), TYPE)
,(SELECT g.displaySettings AS "@inherit" FOR XML PATH('displaySettings'), TYPE)
FROM grp g WHERE [GROUP].server_group_id = g.server_group_id
FOR XML PATH('properties'),TYPE)
 
,(SELECT
 s.name
,s.displayName
,s.comment
,(SELECT s.logonCredentials AS "@inherit" FOR XML PATH('connectionSettings'), TYPE)
,(SELECT s.connectionSettings AS "@inherit" FOR XML PATH('connectionSettings'), TYPE)
,(SELECT s.gatewaySettings AS "@inherit" FOR XML PATH('gatewaySettings'), TYPE)
,(SELECT s.remoteDesktop AS "@inherit" FOR XML PATH('remoteDesktop'), TYPE)
,(SELECT s.localResources AS "@inherit" FOR XML PATH('localResources'), TYPE)
,(SELECT s.securitySettings AS "@inherit" FOR XML PATH('securitySettings'), TYPE)
,(SELECT s.displaySettings AS "@inherit" FOR XML PATH('displaySettings'), TYPE)
FROM srv s WHERE [GROUP].server_group_id = s.server_group_id
FOR XML PATH('server'),TYPE)
FROM grp [GROUP]
ORDER BY [GROUP].name
FOR XML PATH('group'),TYPE)
 
FROM [file]
FOR XML AUTO, ELEMENTS, ROOT('RDCMan')
)

SET @XML.modify('
insert >2.2> 
as first
into (/RDCMan)[1]') 
 
SET @xml.modify('insert attribute schemaVersion{"1"} as last into (RDCMan)[1]')
 
SELECT @XML

To use run the query in the msdb database on your CMS and save the output as an .rdg file. Next, from Remote Desktop Connection Manager select File->Open and select the RDCMan file. Once you’ve imported the list of servers, you’ll want to take a look at the global settings and features of RDCMan. In particular you’ll want to set Logon Credentials which will allow you to autologon to various servers.
The code syntax highlighter I’m using seems to mess up the case of several items, so I’m including the SQL script for download:

The Truth about SQLPS and PowerShell V2

Chad Miller — Fri, 28 May 2010 01:30:38 +0000

With the release of SQL Server 2008 R2 there have been claims that sqlps is really PowerShell V1 under the covers or that sqlps is PowerShell V2 because it returns $psversiontable information. Although, technically, sqlps neither PowerShell V1 nor V2, the answer to this question is a little more complicated and a closer look into the inner workings of sqlps is needed. Before we can look into the implementation details of sqlps let’s review how various Microsoft products implement custom shells.

Custom Shell Implementation and History

Microsoft products that provide PowerShell integration excluding the sqlps implementation which will cover next; provide a custom shell/console in one of two ways: (Note: profiles are not mentioned because they are not used in package products)

Use a console file
Call a PowerShell script on startup

Console Files

A console file is a special XML file with a psc1 extension that stores the configuration information for a PowerShell session including all the custom Snapins loaded. Creating a console file is simple and anyone can do it calling the cmdlet Export-Console. You can then use the console file when starting PowerShell by specifying the –PSConsolefile parameter:

Powershell.exe –PSConsoleFile path_to_consolefile

See Richard Siddaway’s post on console files for more information.

If you look at the shortcut properties of the following Microsoft products you’ll find the use of console files with the following syntax:

*Product*	*Syntax*
Exchange 2007	C:\WINDOWS\system32\WindowsPowerShell\v1.0\PowerShell.exe -PSConsoleFile “C:\Program Files\Microsoft\Exchange Server\bin\exshell.psc1″ -noexit -command “. ‘C:\Program Files\Microsoft\Exchange Server\bin\Exchange.ps1′”
System Center Virtual Machine Manager 2008	C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -PSConsoleFile “C:\Program Files\Microsoft System Center Virtual Machine Manager 2008 R2\bin\cli.psc1″ -NoExit
Operations Manager	C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -PSConsoleFile Microsoft.EnterpriseManagement.OperationsManager.ClientShell.Console.psc1 -NoExit .\Microsoft.EnterpriseManagement.OperationsManager.ClientShell.Startup.ps1

Startup Scripts

PowerShell console files have a couple of limitations first they don’t support modules and second they require starting PowerShell with the PSConsolefile parameter. There are also non-technical issues in the use of console files—many PowerShell users simply don’t like custom consoles and console files don’t lend themselves to reuse in a regular PowerShell console. Looking at Exchange 2010 we see the console file used in Exchange 2007 has been replaced with a startup script. The script is nothing more than a plain old PowerShell script with a ps1 extension. The properties of Exchange 2010 console looks like this:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noexit -command “. ‘C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange.ps1′; Connect-ExchangeServer -auto”

I’m not sure if other products use startup scripts. There doesn’t seem to be much documentation on best practices for custom console creation, but I suspect because of the limitations of console files we will see more products use startup scripts.

SQLPS

This brings us to sqlps, how does SQL Server provide a custom console? Well, sqlps is implemented as something called a minishell. The sqlps use of a minishell is unique among Microsoft products. In fact sqlps is the only product to develop PowerShell integration in this fashion and will probably be last. The use of a minishell was not well received upon the initial release of SQL Server 2008. (Note: the intention of this post is not to critique sqlps usage of minishell, but rather to provide historical summary to the reader, so please no flaming sqlps). Unfortunately the release of SQL Server 2008 R2 includes zero changes to sqlps so, we’ll have to look to the next release of SQL Server to get it fixed. In the next section we’ll create our own minishell in order to see demonstrate sqlps is implemented

Creating A MiniShell

To really understand how a program works we must delve into the code.

Note: The demonstration that follows uses a deprecated tool called make-shell. This tool should not be used for creating PowerShell solutions as part of a product distributed. The purpose of the sample code is illustrate how sqlps minishell is created.

According to Michiel Wories’ (author of sqlps and creator of SMO) blog post SQL Server Powershell is here! sqlps was created via a utility called make-shell. Because make-shell has been deprecated the MSDN documentation has been pulled. A Bing search turns up little other an old post from PowerShell V1 pre-release days which seems to indicate that make-shell was used in beta builds of PowerShell to create cmdlets. The approach demonstrated is even more awkward than the installutil plus add-pssnapin approach to adding cmdlets adopted in V1 and much more complex than the simple import-module approach used in PowerShell V2. We should be thankful make-shell didn’t make into mainstream PowerShell V1 development and how truly simple it is to add cmdlets in PowerShell V2.

Although the make-shell tool has largely disappeared from online docs, the utility and documentation are still available as part of the Windows SDK (both 6.0A and 7.0 versions). Once you install the SDK (any version higher than 6.0A), you can simply copy the standalone make-shell executable to any machine. We’re now ready to build our own SQL Server minishell.

Setup

First you’ll need to install SQL Server 2008 or SQL Server 2008 R2. The Express edition will suffice. Next we need to setup our build directory. To keep things simple and because the paths are a little tricky, we’ll create a directory called C:\makeshell and copy the following files from C:\Program Files\Microsoft SQL Server\100\Tools\Binn, C:\Windows\System32\WindowsPowerShell\v1.0 and C:\Program Files\Microsoft SDKs\Windows\v7.0\Bin. The contents of C:\makeshell should contain:

*
Certificate.format.ps1xml
DotNetTypes.format.ps1xml
FileSystem.format.ps1xml
Help.format.ps1xml
make-shell.exe
Microsoft.SqlServer.Management.PSProvider.dll
Microsoft.SqlServer.Management.PSSnapins.dll
PowerShellCore.format.ps1xml
PowerShellTrace.format.ps1xml
Registry.format.ps1xml
SQLProvider.Format.ps1xml
SQLProvider.Types.ps1xml
types.ps1xml

*The en directory contains the MAML help file for the SQL Server provider and cmdlets (Microsoft.SqlServer.Management.PSProvider.dll-Help.xml and Microsoft.SqlServer.Management.PSSnapins.dll-Help.xml)

The make-shell utility needs to reference the System.Management.Automation.dll. If you’re using PowerShell V1 this assembly is available in the C:\Program Files\Reference Assemblies\Microsoft\WindowsPowerShell\v1.0\ directory. Stating with PowerShell V2 the reference assemblies are no longer included with the PowerShell installation. Instead you’ll need to install Windows SDK 7.0 or higher which provides the reference assemblies in the same location as V1 (C:\Program Files\Reference Assemblies\Microsoft\WindowsPowerShell\\v1.0\ ).

Test Scenario #1:

Test machine OS: Windows XP with SP3
PowerShell version 1
SQL Server 2008 R2 Express Edition
Makeshell build directory with the files and directory documented in the setup section

Steps

Launch PowerShell with -noprofile switch
CD to C:\makshell directory
Run the following command to create a minishell called sqlps2:

.\make-shell.exe -out sqlps2 -ns Community -lib "C:\\Program Files\\Microsoft SQL Server\\100\SDK\\Assemblies\\,C:\\Program Files\\Microsoft SQL Server\\100\Tools\Binn\\,C:\\Program Files\\Reference Assemblies\\Microsoft\\WindowsPowerShell\\v1.0\\" -reference "Microsoft.SqlServer.Management.PSSnapins.dll,Microsoft.SqlServer.Management.PSProvider.dll,Microsoft.SqlServer.Smo.dll,Microsoft.SqlServer.Dmf.dll, Microsoft.SqlServer.SqlWmiManagement.dll,Microsoft.SqlServer.ConnectionInfo.dll,Microsoft.SqlServer.SmoExtended.dll,Microsoft.SqlServer.Management.Sdk.Sfc.dll, Microsoft.SqlServer.SqlEnum.dll,Microsoft.SqlServer.RegSvrEnum.dll,Microsoft.SqlServer.WmiEnum.dll,Microsoft.SqlServer.ServiceBrokerEnum.dll,Microsoft.SqlServer.ConnectionInfoExtended.dll, Microsoft.SqlServer.Management.Collector.dll,Microsoft.SqlServer.Management.CollectorEnum.dll" -formatdata SQLProvider.Format.ps1xml -typedata SQLProvider.Types.ps1xml

And voila, you should have a sqlps2.exe minshell ready for use. An interesting side-effect of make-shell is that it also produces a C# source file. I’ve included both the executable and generated source code in the download. I could not find any licensing agreement that will allow me to distribute the SQL Server provider, snapin or PowerShell help files, so you’ll need to copy the files specified in the setup section. The source code is really interesting as it provides details on why minishell’s behave as they do. Looking at sqlp2.cs source code we see a minishell is really nothing more than a PowerShell host that implements its own RunspaceConfiguration class. As Joel Bennett noted in his blog post Is PowerShell $ShellId too big a burden? using the RunspaceConfiguration class in a custom PowerShell host comes with some expensive side effects:

You have to configure the available .net Assemblies, Cmdlets, Format files, Initialization Scripts, Providers, Scripts, and Type files.
You don’t inherit anything from PowerShell.exe (like for instance, the ExecutionPolicy)
Some cmdlets just don’t work like Add-PSSnapin which means you can not load additional cmdlets

This explains why a minishell behaves the way it does. It also explains why running sqlps on PowerShell V2 results in the same cmdlets as sqlps on PowerShell V1—you have to explicitly list each cmdlet available to the host when creating the shell. The make-shell utility is generating the executable and source code to create the list of cmdlets automatically for the PowerShell host developer. Since sqlps was created using make-shell and PowerShell V1 assemblies ONLY PowerShell V1 cmdlets are available in sqlps.

We’re now ready to startup our new minishell and run a few commands:

PS C:\makeshell> $shellid
Community.sqlps2
PS C:\makeshell> get-command | where {$_.commandtype -eq ‘Cmdlet’} | measure-object

Count : 130
Average :
Sum :
Maximum :
Minimum :
Property :

PS C:\makeshell>

Our minishell, sqlps2.exe, is functionally equivalent to sqlps.exe and the output above shows the number of cmdlets is 130, the same as sqlps. Note: if you’re using an OS with UAC enabled you’ll need to run the minishell as administrator in order to set the execution policy to remotesigned on first run.

Test Scenario #2

Test machine OS: Windows 7 x64
PowerShell version 2
SQL Server 2008 R2 Express Edition
Makeshell build directory with the files and directory documented in the setup section
Windows 7 and .NET 3.5 SP1 SDK (provides PowerShell V2 assembly)

Before creating a new sqlps2 First let’s take the sqlps2.exe including the entire folder created from a PowerShell V1 assemblies in Test Scenario #1 and run the executable on a system with PowerShell V2.

PS C:\makeshell> $shellid
Community.sqlps2
PS C:\makeshell> get-command | where {$_.commandtype -eq ‘Cmdlet’} | measure-object

Count : 130
Average :
Sum :
Maximum :
Minimum :
Property :

PS C:\makeshell>

The number of cmdlets is still 130 and zero V2 cmdlets are available within our sqlps2.exe minishell. This is not surprising considering sqlps2 like sqlps was created by referencing PowerShell V1 assemblies. Next let’s create a new sqlps2 referencing the PowerShell V2 assembly.

Steps

Launch PowerShell with -noprofile switch
CD to C:\makshell directory
Run the following command to create a minishell called sqlps2:

.\make-shell.exe -out sqlps2 -ns Community -lib "C:\\makeshell\\,C:\\Program Files\\Microsoft SQL Server\\100\SDK\\Assemblies\\,C:\\Program Files\\Microsoft SQL Server\\100\Tools\Binn\\,C:\\Program Files\\Reference Assemblies\\Microsoft\\WindowsPowerShell\\v1.0\\" -reference "Microsoft.SqlServer.Management.PSSnapins.dll,Microsoft.SqlServer.Management.PSProvider.dll,Microsoft.SqlServer.Smo.dll,Microsoft.SqlServer.Dmf.dll, Microsoft.SqlServer.SqlWmiManagement.dll,Microsoft.SqlServer.ConnectionInfo.dll,Microsoft.SqlServer.SmoExtended.dll,Microsoft.SqlServer.Management.Sdk.Sfc.dll, Microsoft.SqlServer.SqlEnum.dll,Microsoft.SqlServer.RegSvrEnum.dll,Microsoft.SqlServer.WmiEnum.dll,Microsoft.SqlServer.ServiceBrokerEnum.dll,Microsoft.SqlServer.ConnectionInfoExtended.dll, Microsoft.SqlServer.Management.Collector.dll,Microsoft.SqlServer.Management.CollectorEnum.dll" -formatdata SQLProvider.Format.ps1xml -typedata SQLProvider.Types.ps1xml

We’re now ready to startup our new PowerShell V2 based minishell and run a few commands:

PS C:\makeshell> $shellid
Community.sqlps2
PS C:\makeshell> get-command | where {$_.commandtype -eq ‘Cmdlet’} | measure-object

Count : 220
Average :
Sum :
Maximum :
Minimum :
Property :

PS C:\makeshell>

The V2 based minshell contains most of the 236 base PowerShell V2 cmdlets minus those that are not implemented in RunspaceConfiguration class. Interestingly enough import-module is included which allows us to add cmdlets to sqlps2.

Conclusions

sqlps runs on PowerShell V1 or PowerShell V2
sqlps is neither PowerShell V1 nor PowerShell V2, rather sqlps is a minishell.
Minishells implement the RunspaceConfiguration class which forces the shell developer to explicitly define the cmdlets available to the minishell
Because sqlps was created using make-shell referencing the PowerShell V1 assembly, only PowerShell V1 cmdlets are available in sqlps
The SQL Server 2008 R2 installation media includes PowerShell V1 under \1033_ENU_LP\xx\redist (where xx represents platform x86, x64, IA64)
The SQL Server 2008 and 2008 R2 installer checks whether PowerShell is installed (V1 or V2) and passes the check whether PowerShell is installed or not
The SQL Server 2008 and 2008 R2 installer will install PowerShell V1 if PowerShell is not installed on the machine.
There are no changes in sqlps between SQL Server 2008 and SQL Server 2008 R2

SQL Agent Visual Job Schedule Viewer

Chad Miller — Sat, 15 May 2010 18:54:18 +0000

If you have experience with the SQL Server Agent you quickly realize the difficulty in managing job schedules. On a busy server with many SQL jobs scheduled to run throughout the day at various times finding clashing job schedules and long running jobs is hard to do using the native SQL Server tools.

To address this issue I’ve been looking for a simple way to visualize the SQL Agent job schedule information and I was just about to write my own tool before finding a very nice product called SQLJobVis from SQLSoft. Using SQLJobVis I was able to quickly see jobs that were long running and jobs with overleaping schedules:

SQLJobVis is a great example of how data visualization in the right way makes finding information much easier. At this point, the utility is freely available from SQLSoft and they are just looking feedback and feature requests. So, give SQLJobVis a try and be sure to let them know if you find the tool useful.