Troubleshooting Column Limits in Bulk Copy about csvreader HOT 8 CLOSED

Can you provide any test data? I create random data with 2000 columns and 2000 rows and there is no exception.

from csvreader.

@jakubmaguza - I sure can, let me put together a sample. Thanks for the help!

from csvreader.

@kirikintha Hi. Unfortunatelly the url is broken. Can you send it one more time?

from csvreader.

@jakubmaguza - here is another attachment. Let's see how this goes. The columns have fairly long names in some places, and some eccentricity, but they are all quoted.

Audit-Large Column List.txt

from csvreader.

@kirikintha File works gr8, but I can see only one column in this file with 196 rows.

from csvreader.

I took a quick look at this file and it's braking at last row, where you don't have comma as last character.

from csvreader.

@kirikintha Looking at the sample test @jakubmaguza provided the issue is the parsing mode for the file.

By default the action on a missing field error is ParseError which means if you have a malformed line, the parser will throw an exception.

However you could set this to ReplaceByNull or ReplaceByEmpty, you may still have a issue with SQLBulkCopy as it is very particular about column types but this may be helped with the column default value setting.

I've added further tests around the MissingFieldAction into the code...

    [Test]
    public void MissingFieldNull()
    {
        using (var csv = new CsvReader(new StringReader("a,b,\nc,d"), false))
        {
            csv.SkipEmptyLines = true;
            csv.MissingFieldAction = MissingFieldAction.ReplaceByNull;

            RowCheck(csv, "a", "b", string.Empty);
            RowCheck(csv, "c", "d", null);

            Assert.That(csv.ReadNextRecord, Is.False);
        }
    }

    [Test]
    public void MissingFieldEmpty()
    {
        using (var csv = new CsvReader(new StringReader("a,b,\nc,d"), false))
        {
            csv.SkipEmptyLines = true;
            csv.MissingFieldAction = MissingFieldAction.ReplaceByEmpty;

            RowCheck(csv, "a", "b", string.Empty);
            RowCheck(csv, "c", "d", string.Empty);

            Assert.That(csv.ReadNextRecord, Is.False);
        }
    }

    [Test]
    public void MissingFieldException()
    {
        using (var csv = new CsvReader(new StringReader("a,b,\nc,d"), false))
        {
            csv.SkipEmptyLines = true;
            csv.MissingFieldAction = MissingFieldAction.ParseError;

            RowCheck(csv, "a", "b", string.Empty);
            RowCheck(csv, "c", "d");

            Assert.Throws<MissingFieldCsvException>(() => { var x = csv[2]; });
        }
    }

from csvreader.

Hi everyone, thank you kindly for the update on how to handle the parse errors - I am working with the missing field exception examples and getting farther down the road. I also did an analysis on the file and it is malformed so I am having to go in and try some variation on the DDL and see if I can make it work.

Thank you so much for your help and analysis!

from csvreader.