The mother of all dynamic SQL Crosstab View Builders.
Ok, so maybe it’s not too poetic, but it’s descriptive. As I found myself writing a dynamic crosstab-query builder in MS SQL for probably the fifth time of my career, I decided I’d post it here to save you all just a little time in your day. Make sure you use the saved time for something I’d approve of (smoking a good cigar, playing a little poker, something like that).
This proc is designed for when you have a dynamic set of row-oriented data you want to pivot and show column-wise. It takes four parameters:
@tableName – Name of the source table (or view) that will be queried by the new crosstab view.
@excludeCols – A comma separated list of columns to exclude from the crosstab. In most cases, will be the ID column of the table.
@valueColName – Name of the column in the @tableName table that contains the value in a given row.
@targetViewName – Name of the view that the stored procedure will create. It will *automatically* drop this column before re-creating it, so be warned.
This proc also assumes that the @tableName table (or view) has a column called [colName]. It uses this value to create the corresponding columns across the top of the output view.
Good luck!
ALTER PROCEDURE BuildCrosstabView(
@tableName VARCHAR(100),
@excludeCols VARCHAR(1000),
@valueColName VARCHAR(1000),
@targetViewName VARCHAR(100)
)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE @cols VARCHAR(4000)
SET @cols = ''
EXEC ('SELECT DISTINCT colName INTO ##colList FROM ' + @tableName)
DECLARE col_cursor CURSOR FOR
SELECT colName
FROM ##colList
OPEN col_cursor
DECLARE @colName VARCHAR(100)
FETCH NEXT FROM col_cursor INTO @colName
WHILE @@FETCH_STATUS = 0
BEGIN
IF @cols <> ''
BEGIN
SET @cols = @cols + ', '
END
SET @cols = @cols + 'MAX(CASE colName WHEN ''' + @colName + ''' THEN ' + @valueColName + ' ELSE NULL END) AS [' + @colName + ']
'
FETCH NEXT FROM col_cursor INTO @colName
END
CLOSE col_cursor
DEALLOCATE col_cursor
DROP TABLE ##colList
DECLARE @sql VARCHAR(3000)
SET @sql = 'SELECT DISTINCT ' + @excludeCols + ', ' + @cols + ' FROM ' + @tableName + ' GROUP BY ' + @excludeCols
PRINT @sql
EXECUTE (@sql)
IF EXISTS (SELECT * FROM sys.views WHERE object_id = OBJECT_ID(N'[dbo].[' + @targetViewName + ']'))
EXECUTE ('DROP VIEW ' + @targetViewName)
EXECUTE ('CREATE VIEW ' + @targetViewName + ' AS ' + @sql)
END
GO
Bulk SQL inserting, anyone?
As part of the BlinkStory app, I needed to upload about 200K (200 thousand, for the geek-impaired) records into a MySql database hosted on a very inexpensive (cheap) hosting provider. I’d tried several options to get the data in, with frustrating results. Either my file upload would be interrupted, or an error would occur around row 75K, wasting an hour or so of my time.
So I decided, like any good programmer, to roll my own solution. I created a tool for bulk uploading SQL over the Internet that I call mysqlxfer. Not wanting to hoard it, I submitted it to SourceForge.net as an open source project. If you need to upload a bunch of data over an unreliable and slow connection, give it a shot. It does a couple of cool things, like automatically detecting your field separator character and logging the failed inserts in SQL format to a .err file. Check it out!
A picture of misery.
View Larger Map
I was reading the local paper on Thursday and came across the list of properties that will be auctioned by the Richland County treasurer for back taxes. I got curious, and found the list of properties on the county web site, but a big list of names and amounts just wasn’t very easy for me to picture. So I wrote a little script, geocoded the addresses, converted the results to KML, and uploaded to it to Google Maps. Hard to believe that behind every little dot there’s a story of someone’s home lost.
Check out my submission to the Truveo development contest.
I just submitted my final version of Serendipity to this development contest sponsored by Truveo and run by TopCoder. Check out my application here:
It’s basically a video search tool that takes the results of one search, finds “interesting” words in the descriptions of the matching videos and searches some more. Check it out!
Windows hosting with GoDaddy is a royal PITA.
At least the shared stuff is. I’ve spent at least four hours today trying to get my SQL 2005 database configured on their server. Since they don’t allow remote access through SQL Server Management Studio, I’ve been forced to use their not-so-powerful web-based administration tool. Compounding that is the fact that I used to have references between two different databases on my local server that I now need to merge into a single monolithic database, I’m not a happy camper.
As part of this process I’ve had to populate a bunch of tables with application-specific values. After wrestling with the crappy CSV import function that GoDaddy gave me I decided to write my own little tool to generate SQL INSERT statements for the data in my tables. Since WordPress won’t let me upload it, here’s the code:
/*
* Created by SharpDevelop.
* User: smeans
* Date: 5/6/2008
* Time: 11:58 AM
*
* To change this template use Tools | Options | Coding | Edit Standard Headers.
*/
using System;
using System.Text;
using System.Data;
using System.Data.SqlClient;
using System.IO;
namespace sql2insert
{
class sql2insert
{
public static void Main(string[] args)
{
foreach (string dsn in args) {
Console.WriteLine("writing files to " + System.Environment.CurrentDirectory);
using (SqlConnection cn = new SqlConnection(args[0])) {
cn.Open();
DataTable dt = cn.GetSchema("Tables");
foreach (DataRow row in dt.Rows) {
for (int i = 0; i < dt.Columns.Count; i++) {
if (((string)row[3]).Equals("BASE TABLE")) {
dumpTable(cn, (string)row[2]);
}
}
}
}
}
}
static void dumpTable(SqlConnection cn, string tableName) {
SqlCommand cmd = new SqlCommand(String.Format("SELECT * FROM [{0}]", tableName), cn);
SqlDataReader sdr = cmd.ExecuteReader();
if (sdr.HasRows) {
Console.WriteLine("writing data for table " + tableName);
using (TextWriter tw = new StreamWriter(String.Format("{0}.sql", tableName))) {
tw.WriteLine(String.Format("SET IDENTITY_INSERT [{0}] ON", tableName));
tw.WriteLine("GO\\r\\n");
StringBuilder sbCols = new StringBuilder();
for (int i = 0; i < sdr.FieldCount; i++) {
switch (sdr.GetFieldType(i).ToString()) {
case "System.Byte[]": {
} break;
default: {
if (sbCols.Length > 0) {
sbCols.Append(", ");
}
sbCols.Append(String.Format("[{0}]", sdr.GetName(i)));
} break;
}
}
tw.WriteLine();
while (sdr.Read()) {
tw.Write(String.Format("INSERT INTO [{0}] ({1}) VALUES (", tableName, sbCols.ToString()));
for (int i = 0; i < sdr.FieldCount; i++) {
if (sdr.IsDBNull(i)) {
if (i > 0) {
tw.Write(',');
}
tw.Write("null");
} else {
switch (sdr.GetFieldType(i).ToString()) {
case "System.Int32": {
if (i > 0) {
tw.Write(',');
}
tw.Write(sdr[i].ToString());
} break;
case "System.Decimal": {
if (i > 0) {
tw.Write(',');
}
tw.Write(sdr[i].ToString());
} break;
case "System.Byte[]": {
} break;
case "System.DateTime": {
if (i > 0) {
tw.Write(',');
}
tw.Write(String.Format("'{0}'", ((DateTime)sdr[i]).ToString("M/d/yyyy h:m:s tt")));
} break;
default: {
if (i > 0) {
tw.Write(',');
}
tw.Write(String.Format("'{0}'", sdr[i].ToString().Replace("\\'", "\\'\\'")));
} break;
}
}
}
tw.WriteLine(")\\r\\nGO");
}
tw.WriteLine(String.Format("SET IDENTITY_INSERT [{0}] OFF", tableName));
tw.WriteLine("GO\\r\\n");
tw.Close();
}
}
sdr.Close();
}
}
}
