CODEWITHSUNDEEP
sqlsql-servert-sql
GitZine
GitZine

Reputation: 475

Import multiple text files into SQL Server

I am using SQL Server 2017 Express edition on Microsoft Windows 10. I want to import all text files in a directory into a database at one time. My idea is that that SQL Server loops over the files in that directory and imports them all at the same time. Is there a way that I can achieve this?

Best Regards,

Upvotes: 0

Views: 2273

Answers (2)

Alex Kudryashev
Alex Kudryashev

Reputation: 9460

As noted in another answer, xp_commandshell is problematic. SQL Server 2016+ allows another approach. See example

declare @tbl table(fn varchar(255), depth int,[isfile] int,primary key(fn))
declare @dir varchar(50)='C:\temp\' --'starting dir
insert @tbl
EXEC Master.dbo.xp_DirTree @dir,1,1 --dir, depth (1 - current only), file (0 - dirs only, 1+ - dirs and files)

delete @tbl where [isfile]=0 or fn not like '%.txt' --keep .txt files only
--select * from @tbl
--will insert into this table
create table #fileTbl (id int identity(1,1) primary key,fn varchar(255),txt varchar(max))
declare @fn varchar(255), @query nvarchar(4000)
select top 1 @fn=fn from @tbl
while @@ROWCOUNT>0--number of rows from the last query executed
begin
--dynamic query required to build OPENROWSET
set @query='insert #fileTbl(fn,txt) select '''+@dir+@fn+''',BulkColumn from openrowset(bulk ''c:\temp\'+@fn+''',single_blob) t'
exec sp_executesql @query
delete @tbl where fn=@fn
select top 1 @fn=fn from @tbl --proceed
end
select * from #fileTbl

Upvotes: 1

BoCoKeith
BoCoKeith

Reputation: 946

declare @dir varchar(1000) = 'C:\Temp';
declare @command varchar(1000);
declare @files table (id int identity, filename varchar(1000));

set @command = 'dir /b ' + @dir;
insert into @files execute xp_cmdshell @command;

declare commands cursor for 
    select 'bulk insert <your table name here> from ''' + @dir + '\' + filename + ''' <other options, WITH FORMAT, etc. here>;' from @files;
open commands;
fetch commands into @command;
while (@@fetch_status = 0) begin
    --print @command;
    execute(@command);
    fetch commands into @command;
end;
close commands;
deallocate commands;

Modify @dir and the bulk insert command that is being build and you're done.

You may have to enable 'xp_cmdshell', and this could be a problem for your DBA; and using 'execute' is always a potential issue (SQL injection, etc.).

To enable xp_cmdshell:

-- To allow advanced options to be changed.  
EXEC sp_configure 'show advanced options', 1;  
GO  
-- To update the currently configured value for advanced options.  
RECONFIGURE;  
GO  
-- To enable the feature.  
EXEC sp_configure 'xp_cmdshell', 1;  
GO  
-- To update the currently configured value for this feature.  
RECONFIGURE;  
GO

Upvotes: 2

Related Questions