CODEWITHSUNDEEP
sqlsql-serversql-server-2008t-sqlsql-server-2005
HungryHippos
HungryHippos

Reputation: 1543

Splitting delimited values in a SQL column into multiple rows

I would really like some advice here, to give some background info I am working with inserting Message Tracking logs from Exchange 2007 into SQL. As we have millions upon millions of rows per day I am using a Bulk Insert statement to insert the data into a SQL table.

In fact I actually Bulk Insert into a temp table and then from there I MERGE the data into the live table, this is for test parsing issues as certain fields otherwise have quotes and such around the values.

This works well, with the exception of the fact that the recipient-address column is a delimited field seperated by a ; character, and it can be incredibly long sometimes as there can be many email recipients.

I would like to take this column, and split the values into multiple rows which would then be inserted into another table. Problem is anything I am trying is either taking too long or not working the way I want.

Take this example data:

message-id                                              recipient-address
[email protected]   [email protected]
[email protected]     [email protected]
[email protected]              [email protected];[email protected];[email protected]

I would like this to be formatted as followed in my Recipients table:

message-id                                              recipient-address
[email protected]   [email protected]
[email protected]     [email protected]
[email protected]              [email protected]
[email protected]              [email protected]
[email protected]              [email protected]

Does anyone have any ideas about how I can go about doing this?

I know PowerShell pretty well, so I tried in that, but a foreach loop even on 28K records took forever to process, I need something that will run as quickly/efficiently as possible.

Thanks!

Upvotes: 38

Views: 148048

Answers (4)

anon
anon

Reputation:

If you are on SQL Server 2016+

You can use the new STRING_SPLIT function, which I've blogged about here, and Brent Ozar has blogged about here.

SELECT s.[message-id], f.value
  FROM dbo.SourceData AS s
  CROSS APPLY STRING_SPLIT(s.[recipient-address], ';') as f;

If you are still on a version prior to SQL Server 2016

Create a split function. This is just one of many examples out there:

CREATE FUNCTION dbo.SplitStrings
(
    @List       NVARCHAR(MAX),
    @Delimiter  NVARCHAR(255)
)
RETURNS TABLE
AS
    RETURN (SELECT Number = ROW_NUMBER() OVER (ORDER BY Number),
        Item FROM (SELECT Number, Item = LTRIM(RTRIM(SUBSTRING(@List, Number, 
        CHARINDEX(@Delimiter, @List + @Delimiter, Number) - Number)))
    FROM (SELECT ROW_NUMBER() OVER (ORDER BY s1.[object_id])
        FROM sys.all_objects AS s1 CROSS APPLY sys.all_objects) AS n(Number)
    WHERE Number <= CONVERT(INT, LEN(@List))
        AND SUBSTRING(@Delimiter + @List, Number, 1) = @Delimiter
    ) AS y);
GO

I've discussed a few others here, here, and a better approach than splitting in the first place here.

Now you can extrapolate simply by:

SELECT s.[message-id], f.Item
  FROM dbo.SourceData AS s
  CROSS APPLY dbo.SplitStrings(s.[recipient-address], ';') as f;

Also I suggest not putting dashes in column names. It means you always have to put them in [square brackets].

Upvotes: 83

rohan bharti
rohan bharti

Reputation: 1

for table = "yelp_business", split the column categories values separated by ; into rows and display as category column.

SELECT unnest(string_to_array(categories, ';')) AS category
   FROM yelp_business;

Upvotes: -1

Alex Vazhev
Alex Vazhev

Reputation: 1461

You may use CROSS APPLY (available in SQL Server 2005 and above) and STRING_SPLIT function (available in SQL Server 2016 and above):

DECLARE @delimiter nvarchar(255) = ';';

-- create tables
CREATE TABLE MessageRecipients (MessageId int, Recipients nvarchar(max));
CREATE TABLE MessageRecipient (MessageId int, Recipient nvarchar(max));

-- insert data
INSERT INTO MessageRecipients VALUES (1, '[email protected]; [email protected]; [email protected]');
INSERT INTO MessageRecipients VALUES (2, '[email protected]; [email protected]');

-- insert into MessageRecipient
INSERT INTO MessageRecipient
SELECT MessageId, ltrim(rtrim(value))
FROM MessageRecipients 
CROSS APPLY STRING_SPLIT(Recipients, @delimiter)

-- output results
SELECT * FROM MessageRecipients;
SELECT * FROM MessageRecipient;

-- delete tables
DROP TABLE MessageRecipients;
DROP TABLE MessageRecipient;

Results:

MessageId   Recipients
----------- ----------------------------------------------------
1           [email protected]; [email protected]; [email protected]
2           [email protected]; [email protected]

and

MessageId   Recipient
----------- ----------------
1           [email protected]
1           [email protected]
1           [email protected]
2           [email protected]
2           [email protected]

Upvotes: 5

Oscar Perez
Oscar Perez

Reputation: 111

SQL Server 2016 include a new table function string_split(), similar to the previous solution.

The only requirement is Set compatibility level to 130 (SQL Server 2016)

Upvotes: 5

Related Questions