CODEWITHSUNDEEP
sql-serverssis
Tim
Tim

Reputation: 109

How to retrieve the name of a file and store it in the database using SSIS package?

I'm doing an Excel loop through fifty or more Excel files. The loop goes through each Excel file, grabs all the data and inputs it into the database without error. This is the typical process of setting delay validation to true, and making sure that the expression for the Excel Connection is a string variable called EFile that is set to nothing (in the loop).

What is not working: trying to input the name of the Excel file into the database.

What's been tried (edit; SO changed my 2 to 1 - don't know why):

  1. Add a derived column between the Excel file and database input, and add a column using the EFile expression (so under Expression in the Derived Column it would be @[User::EFile]). and add the empty. However, this inputs nothing a blank (nothing).

  2. One suggestion was to add ANOTHER string variable and set its properties EvaluateAsExpression to True and set the Expression to the EFile variable (@[User::EFile]). The funny thing is that this does the same thing - inputs a blank into the database.

Numerous people on blogs claim they can do this, yet I haven't seen one actually address this (I have a blog and I will definitely be showing people how to do this when I get an answer because, so far, these others have fallen short). How do I grab an Excel file's name and input it in a database during a loop?

Added: Forgot to add, no scripts; the claim is that it can be done without them, so I want to see the solution without them.

Note: I already have the ability to import the data from the Excel files - that's easy (see my GitHub account, as I have two different projects for importing all sorts of txt, csv, xls, xlsx data). I am trying to also get the actual name of the file being imported also into the database. So, if there are fifty Excel files, along with the data in each file, the database will have the fifty file names alongside that data (so if each file has 1000 rows of data, each 1000 rows would also have the name of the file they came from next to them as an additional column). This point seems to cause a lot of confusion, as people assume I'm having trouble importing data in files - NOPE, see my GitHub; again that's easy. It's the FILENAME that needs to also be imported.

Test package: https://github.com/tmmtsmith/SSISLoopWithFileName

Solution: @jaimet pointed out that the Derived Column needed to be the @[User::CurrentFile] (see the test package). When I first ran the package, I still got a blank value in my database. But when we originally set up the connection, we do point it to an actual file (I call this "fooling the package"), then change the expression on the connecting later to the @[User::CurrentFile], which is blank. The Derived Column, using the variable @[User::CurrentFile], showed a string of 0. So, I removed the Derived Column, put the full file path and name in the variable, then added the variable to the Derived Column (which made it think the string was 91 characters long), then went back and set the variable to nothing (English teacher would hate the THENs about right now). When I ran the package, it inputted the full file path. Maybe, like the connection, it needs to initially think that a file exists in order for it to input the full amount of characters?

Appreciate all the help.

Upvotes: 1

Views: 9171

Answers (4)

user756519
user756519

Reputation:

The issue is because of blank value in the variable @[User::FileNameInput] and this caused the SSIS package to assume that the value of this variable will always be of zero length in the Derived Column transformation.

Original

Change the expression on the Derived column transformation from @[User::FileNameInput] to (DT_STR, 2000, 1252)@[User::FileNameInput].

Type casting the derived column to 2000 sets the column length to that maximum value. The value 1252 represents the code page. I assumed that you are using ANSI code page. I took the value 2000 from your table definition because the FilePath column had variable VARCHAR(2000). If the column data type had been NVARCHAR(2000), then the expression would be (DT_WSTR, 2000)@[User::FileNameInput]

New expression

Upvotes: 5

jamiet
jamiet

Reputation: 12254

Tim,

You're using the wrong variable in your Derived Column component. You are storing the filename in @[User::CurrentFile] but the variable that you're using in your Derived Column component is @[User::FileNameInput]

Change your Derived Column component to use @[User::CurrentFile] and you'll be good.

Hope that helps.

JT

Upvotes: 1

jamiet
jamiet

Reputation: 12254

TO all intents and purposes your method #1 should work. That's exactly how I would attempt to do it. I am baffled as to why it is not working. Could you perhaps share your package?

Tony, thanks very much for the link. Much appreciated.

Regards Jamie

Upvotes: 1

Tony
Tony

Reputation: 10327

If you are using a ForEach loop to process the files in a folder then I have have used the technique described in SSIS Junkie's blog to get the filename in to an SSIS variable: SSIS: Enumerating files in a Foreach loop

You can use the variable later in your flow to write it to the database.

Upvotes: 1

Related Questions