Conversation
Task 9 (Testing Improvements): Add CsvParserProperties.fs with 10 tests: - FsCheck property-based roundtrip test (500 random test cases) verifying that arbitrary string values survive encode → parse roundtrip with comma separator and double-quote quoting - Targeted tests for fields containing separators, quotes, newlines (LF), CRLF, tab separators, custom quote chars, multiple rows, empty fields, and single-column empty-field rows (which previously had no coverage) Task 8 (Performance): Add CsvBenchmarks.fs with 8 benchmarks covering: - Parse-only and full row-iteration for AirQuality (3 KB), banklist (40 KB), Titanic (60 KB), and MSFT (328 KB) CSV files - Also fix Program.fs to actually run HtmlBenchmarks and CsvBenchmarks (HtmlBenchmarks was defined but never wired into the program entry point) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
4 tasks
dsyme
deleted the
repo-assist/improve-csv-tests-and-benchmarks-20260309-89ae3e835c77fb70
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 This is an automated pull request from Repo Assist, an AI assistant for this repository.
Addresses two gaps identified in the test/benchmark infrastructure:
Task 9 — Testing Improvements: CSV Property-Based Tests
Adds
tests/FSharp.Data.Core.Tests/CsvParserProperties.fswith 10 new tests:readCsvFile, and verifies a complete roundtrip. Covers embedded commas, quotes, newlines, CRLF, and empty strings automatically.The key insight driving the design: the CSV parser skips blank lines at the top level, so a single-column row containing only an empty string must be quoted (
"") rather than left as an empty line. The newencodeCsvFieldhelper always quotes empty strings to ensure correct roundtripping. This pattern is modelled directly on the existingJsonParserProperties.fs.Task 8 — Performance: CSV Benchmarks
Adds
tests/FSharp.Data.Benchmarks/CsvBenchmarks.fswith 8 benchmarks:ParseAirQualityCsv/IterateAirQualityCsvParseBanklistCsv/IterateBanklistCsvParseTitanicCsv/IterateTitanicCsvParseMSFTCsv/IterateMSFTCsvThe parse-vs-iterate split is important because
CsvFile.Parseis lazy — the parse benchmark measures header parsing and reader setup, while the iterate benchmark measures full row-by-row iteration.Also fixes
Program.fs:HtmlBenchmarkswas defined but never wired into the benchmark runner entry point. This PR addshtmlandcsvas command-line arguments and includes both in the default "run all" path.Test Status