← Back to branch summary

~ubuntu-branches/ubuntu/quantal/ruby1.9.1/quantal

« back to all changes in this revision

Viewing changes to test/csv/test_csv_parsing.rb

  • Committer: Bazaar Package Importer
  • Author(s): Lucas Nussbaum
  • Date: 2010-07-31 17:08:39 UTC
  • mfrom: (1.1.4 upstream) (8.1.2 sid)
  • Revision ID: james.westby@ubuntu.com-20100731170839-j034dmpdqt1cc4p6
Tags: 1.9.2~svn28788-1
http://bugs.debian.org/580852
http://bugs.debian.org/590002
http://bugs.debian.org/543805
http://bugs.debian.org/542927
* New release based on upstream snapshot from the 1.9.2 branch,
  after 1.9.2 RC2. That branch is (supposed to be) binary-compatible
  with the 1.9.1 branch.
  + Builds fine on i386. Closes: #580852.
* Upgrade to Standards-Version: 3.9.1. No changes needed.
* Updated generated incs.
* Patches that still need work:
  + Unclear status, need more investigation:
   090729_fix_Makefile_deps.dpatch
   090803_exclude_rdoc.dpatch
   203_adjust_base_of_search_path.dpatch
   902_define_YAML_in_yaml_stringio.rb.dpatch
   919_common.mk_tweaks.dpatch
   931_libruby_suffix.dpatch
   940_test_thread_mutex_sync_shorter.dpatch
  + Maybe not needed anymore, keeping but not applying.
   102_skip_test_copy_stream.dpatch (test doesn't block anymore?)
   104_skip_btest_io.dpatch (test doesn't block anymore?)
   201_gem_prelude.dpatch (we don't use that rubygems anyway?)
   202_gem_default_dir.dpatch (we don't use that rubygems anyway?)
   940_test_file_exhaustive_fails_as_root.dpatch
   940_test_priority_fails.dpatch
   100518_load_libc_libm.dpatch
* Add disable-tests.diff: disable some tests that cause failures on FreeBSD.
  Closes: #590002, #543805, #542927.
* However, many new failures on FreeBSD. Since that version is still an
  improvement, add the check that makes test suite failures non-fatal on
  FreeBSD again. That still needs to be investigated.
* Re-add 903_skip_base_ruby_check.dpatch
* Add build-dependency on ruby1.8 and drop all pre-generated files.

Show diffs side-by-side

added added

removed removed

Lines of Context:
test/csv/test_csv_parsing.rb
12
12
 
13
13
require "csv"
14
14
 
15
 
# 
16
 
# Following tests are my interpretation of the 
17
 
# {CSV RCF}[http://www.ietf.org/rfc/rfc4180.txt].  I only deviate from that 
 
15
#
 
16
# Following tests are my interpretation of the
 
17
# {CSV RCF}[http://www.ietf.org/rfc/rfc4180.txt].  I only deviate from that
18
18
# document in one place (intentionally) and that is to make the default row
19
19
# separator <tt>$/</tt>.
20
 
# 
 
20
#
21
21
class TestCSVParsing < Test::Unit::TestCase
22
22
  BIG_DATA = "123456789\n" * 1024
23
 
  
 
23
 
24
24
  def test_mastering_regex_example
25
25
    ex = %Q{Ten Thousand,10000, 2710 ,,"10,000","It's ""10 Grand"", baby",10K}
26
26
    assert_equal( [ "Ten Thousand", "10000", " 2710 ", nil, "10,000",
27
27
                    "It's \"10 Grand\", baby", "10K" ],
28
28
                  CSV.parse_line(ex) )
29
29
  end
30
 
  
 
30
 
31
31
  # Old Ruby 1.8 CSV library tests.
32
32
  def test_std_lib_csv
33
33
    [ ["\t", ["\t"]],
79
79
      assert_equal(csv_test.last, CSV.parse_line(csv_test.first))
80
80
    end
81
81
  end
82
 
  
 
82
 
83
83
  # From:  http://ruby-talk.org/cgi-bin/scat.rb/ruby/ruby-core/6496
84
84
  def test_aras_edge_cases
85
85
    [ [%Q{a,b},               ["a", "b"]],
102
102
        assert_equal(edge_case.last, CSV.parse_line(edge_case.first))
103
103
      end
104
104
  end
105
 
  
 
105
 
106
106
  def test_james_edge_cases
107
107
    # A read at eof? should return nil.
108
108
    assert_equal(nil, CSV.parse_line(""))
109
 
    # 
 
109
    #
110
110
    # With Ruby 1.8 CSV it's impossible to tell an empty line from a line
111
111
    # containing a single +nil+ field.  The old CSV library returns
112
112
    # <tt>[nil]</tt> in these cases, but <tt>Array.new</tt> makes more sense to
113
113
    # me.
114
 
    # 
 
114
    #
115
115
    assert_equal(Array.new, CSV.parse_line("\n1,2,3\n"))
116
116
  end
117
 
  
 
117
 
 
118
  def test_rob_edge_cases
 
119
    [ [%Q{"a\nb"},                         ["a\nb"]],
 
120
      [%Q{"\n\n\n"},                       ["\n\n\n"]],
 
121
      [%Q{a,"b\n\nc"},                     ['a', "b\n\nc"]],
 
122
      [%Q{,"\r\n"},                        [nil,"\r\n"]],
 
123
      [%Q{,"\r\n."},                       [nil,"\r\n."]],
 
124
      [%Q{"a\na","one newline"},           ["a\na", 'one newline']],
 
125
      [%Q{"a\n\na","two newlines"},        ["a\n\na", 'two newlines']],
 
126
      [%Q{"a\r\na","one CRLF"},            ["a\r\na", 'one CRLF']],
 
127
      [%Q{"a\r\n\r\na","two CRLFs"},       ["a\r\n\r\na", 'two CRLFs']],
 
128
      [%Q{with blank,"start\n\nfinish"\n}, ['with blank', "start\n\nfinish"]],
 
129
    ].each do |edge_case|
 
130
      assert_equal(edge_case.last, CSV.parse_line(edge_case.first))
 
131
    end
 
132
  end
 
133
 
 
134
  def test_non_regex_edge_cases
 
135
    # An early version of the non-regex parser fails this test
 
136
    [ [ "foo,\"foo,bar,baz,foo\",\"foo\"",
 
137
        ["foo", "foo,bar,baz,foo", "foo"] ] ].each do |edge_case|
 
138
      assert_equal(edge_case.last, CSV.parse_line(edge_case.first))
 
139
    end
 
140
 
 
141
    assert_raise(CSV::MalformedCSVError) do
 
142
      CSV.parse_line("1,\"23\"4\"5\", 6")
 
143
    end
 
144
  end
 
145
 
118
146
  def test_malformed_csv
119
147
    assert_raise(CSV::MalformedCSVError) do
120
148
      CSV.parse_line("1,2\r,3", row_sep: "\n")
121
149
    end
122
 
    
 
150
 
123
151
    bad_data = <<-END_DATA.gsub(/^ +/, "")
124
152
    line,1,abc
125
153
    line,2,"def\nghi"
126
 
    
 
154
 
127
155
    line,4,some\rjunk
128
156
    line,5,jkl
129
157
    END_DATA
130
158
    lines = bad_data.lines.to_a
131
159
    assert_equal(6, lines.size)
132
160
    assert_match(/\Aline,4/, lines.find { |l| l =~ /some\rjunk/ })
133
 
    
 
161
 
134
162
    csv = CSV.new(bad_data)
135
163
    begin
136
164
      loop do
141
169
      assert_equal( "Unquoted fields do not allow \\r or \\n (line 4).",
142
170
                    $!.message )
143
171
    end
144
 
  
 
172
 
145
173
    assert_raise(CSV::MalformedCSVError) { CSV.parse_line('1,2,"3...') }
146
 
    
 
174
 
147
175
    bad_data = <<-END_DATA.gsub(/^ +/, "")
148
176
    line,1,abc
149
177
    line,2,"def\nghi"
150
 
    
 
178
 
151
179
    line,4,8'10"
152
180
    line,5,jkl
153
181
    END_DATA
154
182
    lines = bad_data.lines.to_a
155
183
    assert_equal(6, lines.size)
156
184
    assert_match(/\Aline,4/, lines.find { |l| l =~ /8'10"/ })
157
 
    
 
185
 
158
186
    csv = CSV.new(bad_data)
159
187
    begin
160
188
      loop do
165
193
      assert_equal("Illegal quoting on line 4.", $!.message)
166
194
    end
167
195
  end
168
 
  
 
196
 
169
197
  def test_the_parse_fails_fast_when_it_can_for_unquoted_fields
170
198
    assert_parse_errors_out('valid,fields,bad start"' + BIG_DATA)
171
199
  end
172
 
  
 
200
 
173
201
  def test_the_parse_fails_fast_when_it_can_for_unescaped_quotes
174
202
    assert_parse_errors_out('valid,fields,"bad start"unescaped' + BIG_DATA)
175
203
  end
176
 
  
 
204
 
177
205
  def test_field_size_limit_controls_lookahead
178
206
    assert_parse_errors_out( 'valid,fields,"' + BIG_DATA + '"',
179
207
                             field_size_limit: 2048 )
180
208
  end
181
 
  
 
209
 
182
210
  private
183
 
  
 
211
 
184
212
  def assert_parse_errors_out(*args)
185
213
    assert_raise(CSV::MalformedCSVError) do
186
214
      Timeout.timeout(0.2) do