⬅︎ Back to Rust > Go > Python ...to parse millions of dates in CSV files
What about pandas python module.pd.read_csv('my file.csv', parse_dates = ["date_column_name"])
As per the pandas documentation, using "infer_datetime_format=True" with pd.read_csv can increase the parsing speed by 5-10x
Tempting to try but parsing CSV is just a part of it. There's also triggering concurrency/parallelism, doing the gzip decompression, and doing the date parsing.
Comment
What about pandas python module.
pd.read_csv('my file.csv', parse_dates = ["date_column_name"])
Replies
As per the pandas documentation, using "infer_datetime_format=True" with pd.read_csv can increase the parsing speed by 5-10x
Tempting to try but parsing CSV is just a part of it. There's also triggering concurrency/parallelism, doing the gzip decompression, and doing the date parsing.