Reputation: 71
I am attempting to use CsvHelper to import a CSV file and write it to my SQL database. The data imported does not belong to any predefined class and has to be determined at run time. A lot of forums pointed to using FastMember to doing this but I have been unable to make it work.
As I do not have a predefined class, I am reading the CSV with the following code:
var records = csv.GetRecords<dynamic>().ToList();
I then use fastmember's ObjectReader to structure the data
using (var reader = ObjectReader.Create(data, copyParameters))
{
await sbc.WriteToServerAsync(reader);
}
At this stage, I get the error "Specified argument was out of the range of valid values. Parameter name: name"
I'll include the entire code snippet below, but the copyParameters is a string array containing all of the columns exactly as they appear in the CSV.
Small sample CSV used for testing:
DateTime,Column1,Column2,Column3,Column4 1/8/2014 18:20,1,0,0.3,0 1/8/2014 18:21,1,0,0.3,0 1/8/2014 18:22,1,0,0.2,0 1/8/2014 18:23,1,0,0.2,0 1/8/2014 18:24,1,0,0.2,0 1/8/2014 18:25,1,0,0.2,0
Here is the complete code:
public async Task InsertTimeDataFromImport(IFormFile file, List<ImportCurveGridViewModel> curves, string tableName)
{
try
{
using (var reader = new StreamReader(file.OpenReadStream()))
{
var csv = new CsvReader(reader);
csv.Configuration.HasHeaderRecord = true;
csv.Read();
var records = csv.GetRecords<dynamic>().ToList();
var copyParameters = curves
.Where(c => c.Import)
.Select(c => c.CurveName)
.ToArray();
var batchSize = records.Count();
await _repository.InsertData(tableName, batchSize, records, copyParameters);
}
}
catch (Exception e)
{
System.Diagnostics.Debug.WriteLine(e.Message);
throw;
}
}
public async Task InsertData(string tableName, int batchSize, IEnumerable<dynamic> data, string[] copyParameters)
{
using (SqlBulkCopy sbc = new SqlBulkCopy(_context.Database.GetDbConnection().ConnectionString, SqlBulkCopyOptions.KeepIdentity))
{
sbc.DestinationTableName = "[" + tableName + "]";
sbc.BatchSize = 5000;
foreach (var param in copyParameters)
{
sbc.ColumnMappings.Add(param, param);
}
using (var reader = ObjectReader.Create(data, copyParameters))
{
await sbc.WriteToServerAsync(reader);
}
}
}
Upvotes: 3
Views: 876
Reputation: 43
In my case the error was caused by implemented [ ] operator (see comment in the code below). After changing it to methods there was no exception.
private DataTable TestFastReader()
{
List<TestClass> data = new List<TestClass>();
data.Add(new TestClass(1, "Raz"));
data.Add(new TestClass(2, "Dwa"));
data.Add(new TestClass(3, "Fszy"));
DataTable table = new DataTable();
using (var reader = FastMember.ObjectReader.Create(data))
{
table.Load(reader);
}
return table;
}
class TestClass
{
public int Id { get; set; }
public string Description { get; set; }
public TestClass(int id, string description)
{
this.Id = id;
this.Description = description;
}
// Causes the error in FastMember.ObjectReader
public object this[string propertyName]
{
get { return this.GetType().GetProperty(propertyName).GetValue(this, null); }
set { this.GetType().GetProperty(propertyName).SetValue(this, value, null); }
}
public object GetCustomProperty(string propertyName)
{
return this.GetType().GetProperty(propertyName).GetValue(this, null);
}
public void SetCustomProperty(string propertyName, object val)
{
this.GetType().GetProperty(propertyName).SetValue(this, val, null);
}
}
Upvotes: 0
Reputation: 71
After no progress on this, I decided to bite the bullet and upgrade to .net core 2.1. Once I had access to DataTable again, I was able to circumvent this problem quickly.
Upvotes: 0