Let’s say a user asks you to create a flat file output from a database query. Piece of cake. Open SSMS, write the query, run it and copy the results (with headers using SSMS 2008) into a text file. Send the user the file and and you’re done with time left over for a siesta.
But oh wait—they want to be able to open it in an older version of Excel, which will only display ~ 65k rows per sheet. This file has 190k rows. So they ask you to write the output to multiple files, limiting the total rows per file to 60k.
You could simply open the raw flat file, go to line 60,001 and cut-paste lines 60,001–120,000 into another file and repeat as necessary. That’s ok for a one-off task but not a good strategy if you are going to be doing this several times. It can be cumbersome and error prone to manually create several files this way.
No—the better approach is to make SSIS do the heavy lifting. I had to do this recently and thought I’d share my solution. [Read more...]