taazz: yes, if you want to write a csv dataset, be my guest, but sdf is different than csv (sigh).
In my opinion, the sdf dataformat is just plain )$*(%#()*$% ugly. However, interoperability with Delphi is therefore (I think) about the only reason anybody would want this abomination.
I would definitely try to support everything a normal Delphi app would spit out - suggested: with strictdelimiter:=false, as that is the default.
Unfortunately, because of lack of a better alternative, loads of people (including me in the beginning) insisted on using sdfdataset to load their csv files, which often works but breaks horribly on boundary conditions where
- sdf specs and/or
- the Delphi implementation which deviates a bit from the spec and/or
- the FPC implementation, which differs rather more from the spec, see bug 19610
differ from what any sane csv format would be.
Looking at the Delphi output for bug report 19610:
normal_string;quoted_string;"quoted;delimiter";quoted and space;"""quoted_and_starting_quote";"""quoted, starting quote, and space";quoted_with_tab character;quoted_multi
line; UnquotedSpacesInfront;UnquotedSpacesAtTheEnd ;" ""Spaces before quoted string""";Spaces after quoted string; ;
gives:
(The numbers below indicate field number)
Resulting elements with strictdelimiter false:
0normal_string
1quoted_string
2quoted;delimiter
3quoted and space
4"quoted_and_starting_quote
5"quoted, starting quote, and space
6quoted_with_tab character
7quoted_multi
line
8UnquotedSpacesInfront
9UnquotedSpacesAtTheEnd
10Spaces before quoted string
11Spaces after quoted string
12
Well, perhaps supporting the spaces after quoted string thing is too much.
I'm almost done cleaning up the test cases to closely match the Delphi test program in 19610.
I'll separate out the Spaces after quoted string case, and remove some of your added quote tests.
Understand and agree with your further changes, but I think those may actually be better done in a CSV dataset.
I would really like to see an RFC 4180 compliant CSV dataset and I would *strongly* suggest you take a look at combining csvdocument (see the wiki), as it's csvparser beautifully supports all the intricacies of RFC4180, as well as Excel mode etc.
This means we don't need to implement a parser of our own.
The rest of the dataset support could be built on this, e.g. by using memds or bufdataset or possibly ripping out the sdfdataset code (which I have my doubts about but by now you know much more about it than I).
Writing out the csv to file should once again be easy as csvdocument has a class for that as well.
I'll polish up the test cases for sdfdataset and post them...
Awaiting with interest to hear your opinion!
Thanks,
BigChimp