提问者:小点点

IndexError:列表索引超出范围被抛出,因为我已经改变了文件的读取方式


我正在尝试读取和重新格式化一个非常大(2GB)的. out文件,它的结构类似于csv。我以前使用过标准的open(),没有这样的问题,但是把它改成了codecs.open(),因为它在一些字符上有问题。

它现在正在投掷

Traceback(最近一次调用最后一次):第21行,在

import sys
import csv
import datetime
import codecs
maxInt=sys.maxsize
decrement=True

while decrement:
    decrement=False
    try:
        csv.field_size_limit(maxInt)
    except OverflowError:
        maxInt = int(maxInt/10)
        decrement = True

with codecs.open("file.out", 'rU', 'utf-16-be') as source:
    rdr = csv.reader(source)
    with open("out.csv","w", newline='') as result:
        wtr = csv.writer(result)
        wtr.writerow(("Column1", "column2", "column3", "etc..."))
        for r in rdr:
            if(r[5]==""):
                continue
            wtr.writerow((datetime.datetime.strptime(r[5], '%m/%d/%Y').strftime('%Y-%m-%d'), r[3], r[7], r[9]+r[10]+" "+r[12]))

使用utf-8抛出UnicodeDecodeError:'utf-8'编解码器无法解码位置12中的字节0xc9:无效的延续字节

使用latin-1或ISO-8859-1抛出UnicodeEncodeError:'charmap'编解码器无法编码位置57-58中的字符:字符映射到

输入文件如下所示:

"A00017","K","G","1999","4530","01/12/1999","","","","PEOPLE TO ELECT MANGINELLI","","","","258 MAGNIOLIA DRIVE","SELDEN","NY","11784","","","404.57","","","","","","","2","","NAA","07/22/1999 08:43:59"
"A00037","K","G","1999","999999","01/12/1999","","","","CITIZENS TO ELECT TEDISCO TO ASSEMBLY","","","","","","","","","","0","","","","","","","2","","",""
"A00037","K","N","1999","1693","01/15/1999","","","","OUTSTANDING LOAN","","","","2176 GUILDERLAND AVE","SCHENECTADY","NY","12306","","","10474.8","10474.8","","","OTHER","","PREVIOUS LOAN FROM JAMES TEDISCO","","P","JM","07/15/1999 15:08:17"
"A00037","J","N","2000","1694","01/13/2000","","","","OUTSTANDING LOAN","","","","2176 GUILDERLAND","SCHENECTADY","NY","12306","","","10474.8","10474.8","","","OTHER","","LOANS FROM PREVIOUS CAMPAIGNS FROM J","","P","JM","01/14/1900 16:35:09"
"A00037","K","X","2000","999999","","","","","","","","","","","","","","","","","","","","","","","","","07/20/2000 00:00:00"
"A00037","J","X","2001","999999","","","","","","","","","","","","","","","","","","","","","","","","","01/17/2001 00:00:00"
"A00037","K","X","2002","999999","","","","","","","","","","","","","","","","","","","","","","","","","07/19/2002 00:00:00"
"A00037","J","X","2003","999999","","","","","","","","","","","","","","","","","","","","","","","","","01/21/2003 00:00:00"
"A00037","K","X","2003","999999","","","","","","","","","","","","","","","","","","","","","","","","","07/16/2003 00:00:00"
"A00037","J","X","2004","999999","","","","","","","","","","","","","","","","","","","","","","","","","01/22/2004 00:00:00"

我走到这一步要归功于:

行包含NULL字节在CSV阅读器(Python)

_csv错误:字段大于字段限制(131072)


共1个答案

匿名用户

在您正在读取的“file. out”中,找出行中每个单元格元素之间的分隔字符。像'\t'-tab或','-逗号并将其传递给'delimiter'属性。

尝试打印'r'并查看列名或行中值之间的字符

rdr = csv.reader(source,delimiter=<separator>)