My MySQL tool is on softpedia.com.
I just got the notice today. I wrote this bulk row insert tool for MySQL that is designed to run over the Internet. It’s for really large data sets being loaded into … ahem … inexpensive hosted MySQL databases. Check out the mysqlxfer softpedia page.
Windows hosting with GoDaddy is a royal PITA.
At least the shared stuff is. I’ve spent at least four hours today trying to get my SQL 2005 database configured on their server. Since they don’t allow remote access through SQL Server Management Studio, I’ve been forced to use their not-so-powerful web-based administration tool. Compounding that is the fact that I used to have references between two different databases on my local server that I now need to merge into a single monolithic database, I’m not a happy camper.
As part of this process I’ve had to populate a bunch of tables with application-specific values. After wrestling with the crappy CSV import function that GoDaddy gave me I decided to write my own little tool to generate SQL INSERT statements for the data in my tables. Since WordPress won’t let me upload it, here’s the code:
/*
* Created by SharpDevelop.
* User: smeans
* Date: 5/6/2008
* Time: 11:58 AM
*
* To change this template use Tools | Options | Coding | Edit Standard Headers.
*/
using System;
using System.Text;
using System.Data;
using System.Data.SqlClient;
using System.IO;
namespace sql2insert
{
class sql2insert
{
public static void Main(string[] args)
{
foreach (string dsn in args) {
Console.WriteLine("writing files to " + System.Environment.CurrentDirectory);
using (SqlConnection cn = new SqlConnection(args[0])) {
cn.Open();
DataTable dt = cn.GetSchema("Tables");
foreach (DataRow row in dt.Rows) {
for (int i = 0; i < dt.Columns.Count; i++) {
if (((string)row[3]).Equals("BASE TABLE")) {
dumpTable(cn, (string)row[2]);
}
}
}
}
}
}
static void dumpTable(SqlConnection cn, string tableName) {
SqlCommand cmd = new SqlCommand(String.Format("SELECT * FROM [{0}]", tableName), cn);
SqlDataReader sdr = cmd.ExecuteReader();
if (sdr.HasRows) {
Console.WriteLine("writing data for table " + tableName);
using (TextWriter tw = new StreamWriter(String.Format("{0}.sql", tableName))) {
tw.WriteLine(String.Format("SET IDENTITY_INSERT [{0}] ON", tableName));
tw.WriteLine("GO\\r\\n");
StringBuilder sbCols = new StringBuilder();
for (int i = 0; i < sdr.FieldCount; i++) {
switch (sdr.GetFieldType(i).ToString()) {
case "System.Byte[]": {
} break;
default: {
if (sbCols.Length > 0) {
sbCols.Append(", ");
}
sbCols.Append(String.Format("[{0}]", sdr.GetName(i)));
} break;
}
}
tw.WriteLine();
while (sdr.Read()) {
tw.Write(String.Format("INSERT INTO [{0}] ({1}) VALUES (", tableName, sbCols.ToString()));
for (int i = 0; i < sdr.FieldCount; i++) {
if (sdr.IsDBNull(i)) {
if (i > 0) {
tw.Write(',');
}
tw.Write("null");
} else {
switch (sdr.GetFieldType(i).ToString()) {
case "System.Int32": {
if (i > 0) {
tw.Write(',');
}
tw.Write(sdr[i].ToString());
} break;
case "System.Decimal": {
if (i > 0) {
tw.Write(',');
}
tw.Write(sdr[i].ToString());
} break;
case "System.Byte[]": {
} break;
case "System.DateTime": {
if (i > 0) {
tw.Write(',');
}
tw.Write(String.Format("'{0}'", ((DateTime)sdr[i]).ToString("M/d/yyyy h:m:s tt")));
} break;
default: {
if (i > 0) {
tw.Write(',');
}
tw.Write(String.Format("'{0}'", sdr[i].ToString().Replace("\\'", "\\'\\'")));
} break;
}
}
}
tw.WriteLine(")\\r\\nGO");
}
tw.WriteLine(String.Format("SET IDENTITY_INSERT [{0}] OFF", tableName));
tw.WriteLine("GO\\r\\n");
tw.Close();
}
}
sdr.Close();
}
}
}
Fun with SqlCommand.ExecuteXmlReader()
So I’m coding along this morning, minding my own business, when I run up against a really annoying problem with the XML support in the .Net SQL support classes. What I was trying to do was fetch some values from a support table as a simple XML document with an element for each row. The SQL FOR XML AUTO clause was just the ticket, so I wrote the following code:
SqlCommand cmd = new SqlCommand();
cmd.Connection = cn;
cmd.CommandType = CommandType.Text;
cmd.CommandText = "SELECT * FROM "
+ "supportTable AS valueName FOR XML AUTO";
XmlReader res = cmd.ExecuteXmlReader();
XmlDocument doc = new XmlDocument();
doc.Load(res);
but when I tried to run the code I got the following error on the doc.Load() call:
This document already has a DocumentElement node.
It really had me irritated, because I’d already used the exact same code in another part of my application without a hitch. So after some research, it turns out that the problem (which is obvious in retrospect) is that if multiple rows are returned, the resulting XML is not a valid document, but a document fragment. Since there is no top-level element, the Load() method chokes when it encounters the second row. Not good.
I found several workarounds on the web, but none that I really cared for. So I resorted to some MS SQL funny business that I’ve used in the past that did the trick. The problem here is that there is no single top-level element to serve as a root element. One possible approach is to abandon the FOR XML AUTO in favor of FOR XML EXPLICIT, but the explicit support is so incredibly complex and unusable that I’d rather cut my own foot off with a rusty tin can lid. So, to save my foot, I resorted to some fun with joins.
The FOR XML AUTO clause will actually create nested elements in the case of joins between tables, so to create a single top-level element for my support table document I created a new table in my DB called dummy. It has a single column (dummy) and a single row with a single value (dummy). The single row part is important, because by changing the SQL statement above slightly, I end up with the nice, valid XML document I want:
SELECT * FROM dummy AS rootElementName,
supportTable AS valueName FOR XML AUTO
Voila! Now my code works and I can move on to more interesting pursuits, like the development of a Comet service for the masses. Good stuff!
