Currently, the reading of the metadata file infers the file extension (.jsonl or .csv) from the passed filename. However, downloaded files from the Hub don't have file extension. For example:
- the original file: `hf://datasets/__DUMMY_TRANSFORMERS_USER__/test-dataset-5916a4-16977085077831/metadata.jsonl`
- corresponds to the downloaded path: `/tmp/pytest-of-username/pytest-46/cache/datasets/downloads/9f5374dbb470f711f6b89d66a5eec1f19cc96324b26bcbebe29138bda6cb20e6`, which does not have extension

In the case where the metadata file does not have an extension, the reader assumes it is a JSONL file, thus the reported error when trying to read a CSV file as a JSONL one: `ArrowInvalid: JSON parse error: Invalid value. in row 0`

This behavior was introduced by:
- #4837

This PR extracts the metadata file extension from the original filename (instead of the downloaded one) and passes it as a parameter to the read_metadata function.

Fix #6315.
Fix loading Hub datasets with CSV metadata file
<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.008896 / 0.011353 (-0.002456) | 0.005811 / 0.011008 (-0.005197) | 0.108582 / 0.038508 (0.070074) | 0.096509 / 0.023109 (0.073399) | 0.481725 / 0.275898 (0.205827) | 0.534743 / 0.323480 (0.211263) | 0.005517 / 0.007986 (-0.002468) | 0.006479 / 0.004328 (0.002151) | 0.081313 / 0.004250 (0.077062) | 0.063578 / 0.037052 (0.026525) | 0.493977 / 0.258489 (0.235488) | 0.551897 / 0.293841 (0.258056) | 0.051835 / 0.128546 (-0.076711) | 0.014105 / 0.075646 (-0.061541) | 0.385866 / 0.419271 (-0.033405) | 0.069131 / 0.043533 (0.025598) | 0.484780 / 0.255139 (0.229641) | 0.493221 / 0.283200 (0.210021) | 0.039560 / 0.141683 (-0.102123) | 1.782331 / 1.452155 (0.330176) | 1.899193 / 1.492716 (0.406477) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.329978 / 0.018006 (0.311972) | 0.600839 / 0.000490 (0.600349) | 0.013187 / 0.000200 (0.012987) | 0.000499 / 0.000054 (0.000444) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.031835 / 0.037411 (-0.005576) | 0.103740 / 0.014526 (0.089214) | 0.115875 / 0.176557 (-0.060681) | 0.189880 / 0.737135 (-0.547255) | 0.132614 / 0.296338 (-0.163725) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.596255 / 0.215209 (0.381046) | 5.967993 / 2.077655 (3.890339) | 2.612675 / 1.504120 (1.108555) | 2.251461 / 1.541195 (0.710266) | 2.308585 / 1.468490 (0.840095) | 0.816516 / 4.584777 (-3.768261) | 5.241791 / 3.745712 (1.496079) | 4.680745 / 5.269862 (-0.589117) | 2.997370 / 4.565676 (-1.568307) | 0.098632 / 0.424275 (-0.325643) | 0.010912 / 0.007607 (0.003305) | 0.659092 / 0.226044 (0.433047) | 6.825562 / 2.268929 (4.556634) | 3.323844 / 55.444624 (-52.120780) | 2.796203 / 6.876477 (-4.080274) | 2.946994 / 2.142072 (0.804922) | 1.002814 / 4.805227 (-3.802413) | 0.202613 / 6.500664 (-6.298051) | 0.072011 / 0.075469 (-0.003459) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.613873 / 1.841788 (-0.227914) | 24.500990 / 8.074308 (16.426682) | 21.941599 / 10.191392 (11.750207) | 0.214450 / 0.680424 (-0.465974) | 0.031227 / 0.534201 (-0.502974) | 0.498297 / 0.579283 (-0.080986) | 0.597460 / 0.434364 (0.163096) | 0.558152 / 0.540337 (0.017815) | 0.789693 / 1.386936 (-0.597243) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.011299 / 0.011353 (-0.000053) | 0.005103 / 0.011008 (-0.005905) | 0.083161 / 0.038508 (0.044653) | 0.094201 / 0.023109 (0.071092) | 0.560457 / 0.275898 (0.284559) | 0.590459 / 0.323480 (0.266980) | 0.007059 / 0.007986 (-0.000926) | 0.004418 / 0.004328 (0.000090) | 0.081343 / 0.004250 (0.077093) | 0.067069 / 0.037052 (0.030016) | 0.538137 / 0.258489 (0.279648) | 0.600416 / 0.293841 (0.306575) | 0.049046 / 0.128546 (-0.079500) | 0.014299 / 0.075646 (-0.061347) | 0.093631 / 0.419271 (-0.325641) | 0.062536 / 0.043533 (0.019003) | 0.557238 / 0.255139 (0.302099) | 0.571050 / 0.283200 (0.287850) | 0.035881 / 0.141683 (-0.105802) | 1.918487 / 1.452155 (0.466332) | 2.013979 / 1.492716 (0.521263) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.400995 / 0.018006 (0.382989) | 0.634898 / 0.000490 (0.634408) | 0.041809 / 0.000200 (0.041609) | 0.000279 / 0.000054 (0.000224) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.034160 / 0.037411 (-0.003251) | 0.109996 / 0.014526 (0.095470) | 0.124335 / 0.176557 (-0.052222) | 0.188100 / 0.737135 (-0.549035) | 0.135897 / 0.296338 (-0.160442) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.639751 / 0.215209 (0.424542) | 6.403312 / 2.077655 (4.325657) | 3.146453 / 1.504120 (1.642333) | 2.840358 / 1.541195 (1.299164) | 2.908667 / 1.468490 (1.440177) | 0.818767 / 4.584777 (-3.766010) | 5.416939 / 3.745712 (1.671227) | 4.853498 / 5.269862 (-0.416364) | 3.023526 / 4.565676 (-1.542150) | 0.110850 / 0.424275 (-0.313425) | 0.013103 / 0.007607 (0.005496) | 0.799720 / 0.226044 (0.573676) | 7.837704 / 2.268929 (5.568775) | 4.016526 / 55.444624 (-51.428099) | 3.338965 / 6.876477 (-3.537512) | 3.715721 / 2.142072 (1.573648) | 1.088340 / 4.805227 (-3.716887) | 0.213610 / 6.500664 (-6.287054) | 0.079244 / 0.075469 (0.003775) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.833175 / 1.841788 (-0.008612) | 25.307218 / 8.074308 (17.232910) | 23.716075 / 10.191392 (13.524683) | 0.259114 / 0.680424 (-0.421310) | 0.035171 / 0.534201 (-0.499029) | 0.530128 / 0.579283 (-0.049155) | 0.651484 / 0.434364 (0.217120) | 0.589414 / 0.540337 (0.049077) | 0.862691 / 1.386936 (-0.524245) |

</details>
</details>

![](https://cml.dev/watermark.png#1bdfba93b8a739b9d885b8fb1909d47ff689bbc2 "CML watermark")
 _The documentation is not available anymore as the PR was closed or merged._ Me too, I thought the same... quite surprised... :open_mouth:  <details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006929 / 0.011353 (-0.004423) | 0.004345 / 0.011008 (-0.006663) | 0.085522 / 0.038508 (0.047014) | 0.083380 / 0.023109 (0.060271) | 0.310332 / 0.275898 (0.034434) | 0.350525 / 0.323480 (0.027045) | 0.004367 / 0.007986 (-0.003618) | 0.005503 / 0.004328 (0.001175) | 0.066311 / 0.004250 (0.062061) | 0.059545 / 0.037052 (0.022492) | 0.314090 / 0.258489 (0.055601) | 0.366661 / 0.293841 (0.072821) | 0.031581 / 0.128546 (-0.096965) | 0.008852 / 0.075646 (-0.066794) | 0.289312 / 0.419271 (-0.129960) | 0.052960 / 0.043533 (0.009427) | 0.308134 / 0.255139 (0.052995) | 0.330342 / 0.283200 (0.047142) | 0.026157 / 0.141683 (-0.115526) | 1.488463 / 1.452155 (0.036308) | 1.561441 / 1.492716 (0.068725) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.327735 / 0.018006 (0.309729) | 0.568162 / 0.000490 (0.567672) | 0.012097 / 0.000200 (0.011897) | 0.000438 / 0.000054 (0.000383) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.029503 / 0.037411 (-0.007909) | 0.084327 / 0.014526 (0.069801) | 0.102065 / 0.176557 (-0.074492) | 0.157392 / 0.737135 (-0.579744) | 0.101428 / 0.296338 (-0.194910) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.386767 / 0.215209 (0.171558) | 3.870757 / 2.077655 (1.793102) | 1.870048 / 1.504120 (0.365928) | 1.678221 / 1.541195 (0.137026) | 1.799423 / 1.468490 (0.330933) | 0.477718 / 4.584777 (-4.107059) | 3.618351 / 3.745712 (-0.127361) | 3.577921 / 5.269862 (-1.691941) | 2.146217 / 4.565676 (-2.419459) | 0.056290 / 0.424275 (-0.367985) | 0.007378 / 0.007607 (-0.000229) | 0.460678 / 0.226044 (0.234633) | 4.606243 / 2.268929 (2.337314) | 2.303460 / 55.444624 (-53.141164) | 1.982662 / 6.876477 (-4.893814) | 2.103891 / 2.142072 (-0.038182) | 0.570700 / 4.805227 (-4.234527) | 0.131747 / 6.500664 (-6.368918) | 0.060915 / 0.075469 (-0.014554) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.286364 / 1.841788 (-0.555424) | 20.106330 / 8.074308 (12.032022) | 14.780833 / 10.191392 (4.589441) | 0.164301 / 0.680424 (-0.516123) | 0.018730 / 0.534201 (-0.515471) | 0.398530 / 0.579283 (-0.180754) | 0.418084 / 0.434364 (-0.016280) | 0.468735 / 0.540337 (-0.071602) | 0.690122 / 1.386936 (-0.696814) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007262 / 0.011353 (-0.004091) | 0.004228 / 0.011008 (-0.006780) | 0.065866 / 0.038508 (0.027358) | 0.096151 / 0.023109 (0.073042) | 0.409352 / 0.275898 (0.133454) | 0.441234 / 0.323480 (0.117754) | 0.005946 / 0.007986 (-0.002039) | 0.003630 / 0.004328 (-0.000698) | 0.066271 / 0.004250 (0.062020) | 0.061567 / 0.037052 (0.024515) | 0.409097 / 0.258489 (0.150608) | 0.447675 / 0.293841 (0.153834) | 0.032804 / 0.128546 (-0.095743) | 0.008793 / 0.075646 (-0.066853) | 0.070790 / 0.419271 (-0.348482) | 0.048650 / 0.043533 (0.005117) | 0.411021 / 0.255139 (0.155882) | 0.421398 / 0.283200 (0.138198) | 0.025305 / 0.141683 (-0.116378) | 1.494826 / 1.452155 (0.042671) | 1.580441 / 1.492716 (0.087724) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.321871 / 0.018006 (0.303865) | 0.526471 / 0.000490 (0.525982) | 0.006913 / 0.000200 (0.006713) | 0.000108 / 0.000054 (0.000054) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.034889 / 0.037411 (-0.002522) | 0.096096 / 0.014526 (0.081570) | 0.111920 / 0.176557 (-0.064636) | 0.166103 / 0.737135 (-0.571032) | 0.111162 / 0.296338 (-0.185176) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.428037 / 0.215209 (0.212828) | 4.294150 / 2.077655 (2.216495) | 2.270331 / 1.504120 (0.766211) | 2.108235 / 1.541195 (0.567041) | 2.242560 / 1.468490 (0.774070) | 0.489941 / 4.584777 (-4.094836) | 3.688111 / 3.745712 (-0.057601) | 3.450180 / 5.269862 (-1.819681) | 2.175106 / 4.565676 (-2.390570) | 0.057657 / 0.424275 (-0.366619) | 0.007478 / 0.007607 (-0.000130) | 0.505242 / 0.226044 (0.279198) | 5.047817 / 2.268929 (2.778888) | 2.724125 / 55.444624 (-52.720500) | 2.419765 / 6.876477 (-4.456711) | 2.723231 / 2.142072 (0.581159) | 0.602382 / 4.805227 (-4.202846) | 0.132362 / 6.500664 (-6.368302) | 0.060600 / 0.075469 (-0.014869) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.363356 / 1.841788 (-0.478431) | 21.446474 / 8.074308 (13.372165) | 15.074732 / 10.191392 (4.883340) | 0.191837 / 0.680424 (-0.488587) | 0.020565 / 0.534201 (-0.513636) | 0.396692 / 0.579283 (-0.182591) | 0.432390 / 0.434364 (-0.001974) | 0.491747 / 0.540337 (-0.048591) | 0.699203 / 1.386936 (-0.687733) |

</details>
</details>

![](https://cml.dev/watermark.png#c3a8a87c841426495d3a7ed1863c26660a6a551f "CML watermark")
