Kaiko - Bittrex Python index

Kaiko (no affiliation) is a company that provides historical data from various exchanges. I just bought an archive for the Bittrex exchange. Data looks quite cool. There are a few days missing here and there and sometimes there's a few week's lag between the time a coin is listed and the archive picking it up. That's not a problem in my case.

The archive is right now about half a GB and it's all .gz compressed files. It's a bit hard to process and I just crated a nice Python package that allows you to build a quick index over the files and process them easily. You can find it in github python-kaiko-bittrex. After you install it with pip install kaikobittrex, you can use it like this:


from kaikobittrex import Index

idx = Index.from_file()

def process_pair(year, month, day, pair, f):
    for lineno, line in enumerate(f.readlines()):
        line = line.strip()
        if lineno == 0:
            assert line == "id,exchange,symbol,date,price,amount,sell"
        else:
            record = idx.parse_line(line)
            print record

idx.process_all(process_pair)