Implementation of ticks to 1min and 15 mins candles in zerodha kiteconnect using python queues. This code is modified version of the code given in http://ezeetrading.in/Articles/Candles_formation_from_tick_data_zerodha.html. The difference is that in on_ticks functions the only action performed is that the ticks are place in a event queue. This …

create-kiteconnect-candlestick-q.py

################## Ticks to candles in kiteconnect python ####################
# Author : Arun B
# Reference  : http://ezeetrading.in/Articles/Candles_formation_from_tick_data_zerodha.html
# Purpose : Convert ticks to candles by putting ticks in a queue. This redues time wasted in on_ticks function
################################################################################

from kiteconnect import KiteTicker
import datetime
from copy import copy
import queue
import os
import pandas as pd
import json

class Event(object):
    """
    Event is base class providing an interface for all subsequent 
    (inherited) events, that will trigger further events in the 
    trading infrastructure.   
    """
    pass

class TickEvent(Event):
    """
    Handles the event of receiving a new market ticks
    """

    def __init__(self,ticks):
        """
        Initialises the MarketEvent.
        """
        self.type = 'TICK'
        self.data = ticks

class CandleEvent(Event):
    """
    Handles the event of receiving a 1min candle
    """
    
    def __init__(self, symbol,candle):
        
        self.type = 'CANDLE'
        self.symbol = symbol
        self.data = candle

    def print_self(self):
        print ("CANDLE:",self.data)
        
class CandleEvent15Min(Event):
    """
    Handles the event of 15 min candle.
    
    """

    def __init__(self, symbol, candle):
        """
        Initialises the 15mincandle event.
        """

        self.type = '15MinCANDLE'
        self.symbol = symbol
        self.data = candle

    def print_self(self):
        """
        Outputs the values within the Order.
        """
        print("CANDLE: = ",self.data)

# Enter the user api_key and the access_token generated for that day
credentials={'api_key': " ", 'access_token': " "}
# Initialise
kws = KiteTicker(credentials["api_key"], credentials["access_token"])

#token_dict = {256265:'NIFTY 50',260105:'NIFTY BANK'} #prev definition is wrong. It has to be nested dictionary
token_dict = {256265:{'Symbol':'NIFTY 50'},260105:{'Symbol':'NIFTY BANK'}} #you can put any F&O tokens here
# Creation of 1min and 15 min candles
candles_1={}
candles_15={}

EventQ = queue.Queue()


def on_ticks(ws, ticks):
    # Callback to receive ticks.
    tick = TickEvent(ticks)
    EventQ.put(tick)
  

def on_connect(ws, response):
    # Callback on successful connect.
    # Subscribe to a list of instrument_tokens (RELIANCE and ACC here).
    instruments=list(token_dict.keys())
    print(instruments)
    #exit()
    ws.subscribe(instruments)

    # Set tick in `full` mode.
    ws.set_mode(ws.MODE_FULL, instruments)

kws.on_ticks = on_ticks
kws.on_connect = on_connect
# Infinite loop on the main thread. Nothing after this will run.
# You have to use the pre-defined callbacks to manage subscriptions.
kws.connect(threaded=True)
def main():
    while True:
        try:
            event = EventQ.get(False)
            #print (ticks)
        except queue.Empty:
            continue
        else:
            if event.type=='TICK':
                ticks = event.data
                for tick in ticks:
                    instrument=tick["instrument_token"]
                    #instrument = token_dict[instrument_token]
                    #print(instrument_token,instrument)
                    ltt=tick["timestamp"]
                    #print(tick)
                    ltt_min_1=datetime.datetime(ltt.year, ltt.month, ltt.day, ltt.hour,ltt.minute)
                    ltt_min_2=ltt_min_1 - datetime.timedelta(minutes=1)
                    ltt_min_15 = datetime.datetime(ltt.year, ltt.month, ltt.day, ltt.hour, ltt.minute//15 * 15)
                    ltt_min_215 = ltt_min_15 - datetime.timedelta(minutes=15)
                    #print(ltt_min_1,end='\r')
                    #print(ltt_min_15,ltt_min_215)
                    #exit()
                    # For any other timeframe. Simply change ltt_min_1 variable defination.
                    # e.g.
                    # ltt_min_15=datetime.datetime(ltt.year, ltt.month, ltt.day, ltt.hour,ltt.minute//15*15)

                    ### Forming 1 Min Candles...

                    if instrument in candles_1:
                        if ltt_min_1 in candles_1[instrument]:
                            #print(tick)

                            candles_1[instrument][ltt_min_1]["high"] = max(candles_1[instrument][ltt_min_1]["high"],
                                                                             tick["last_price"])  # 1
                            candles_1[instrument][ltt_min_1]["low"] = min(candles_1[instrument][ltt_min_1]["low"],
                                                                            tick["last_price"])  # 2
                            candles_1[instrument][ltt_min_1]["close"] = tick["last_price"]  # 3
                            if tick["tradable"]: # for F & O 
                                candles_1[instrument][ltt_min_1]["volume"] = tick["volume"]
                                candles_1[instrument][ltt_min_1]["oi"] = tick['oi']
                                candles_1[instrument][ltt_min_1]["atp"] = tick['average_price']

                                if ltt_min_2 in candles_1[instrument]:
                                    candles_1[instrument][ltt_min_1]["vol"] = tick["volume"] -candles_1[instrument][ltt_min_2][
                                                                                    "volume"]  # 3.5
                            else:
                                candles_1[instrument][ltt_min_1]["volume"] = 0
                                candles_1[instrument][ltt_min_1]["oi"] = 0
                                candles_1[instrument][ltt_min_1]["atp"] = 0

                        else:
                            # print(instrument,str(ltt_min_1),candles_1[instrument][ltt_min_1])
                            candles_1[instrument][ltt_min_1] = {}
                            candles_1[instrument][ltt_min_1]["high"] = copy(tick["last_price"])  # 8
                            candles_1[instrument][ltt_min_1]["low"] = copy(tick["last_price"])
                            candles_1[instrument][ltt_min_1]["open"] = copy(tick["last_price"])
                            candles_1[instrument][ltt_min_1]["close"] = copy(tick["last_price"])
                            candles_1[instrument][ltt_min_1]["volume"] = 0
                            candles_1[instrument][ltt_min_1]["vol"] = 0
                            candles_1[instrument][ltt_min_1]["oi"] = 0
                            candles_1[instrument][ltt_min_1]["atp"] = 0
                            if ltt_min_2 in candles_1[instrument]:
                                # print(candles_1)
                                candle = {"token": instrument, "Time": ltt_min_2,
                                          "open": candles_1[instrument][ltt_min_2]["open"],
                                          "high": candles_1[instrument][ltt_min_2]["high"],
                                          "low": candles_1[instrument][ltt_min_2]["low"],
                                          "close": candles_1[instrument][ltt_min_2]["close"],
                                          "volume": candles_1[instrument][ltt_min_2]["vol"],
                                          "oi": candles_1[instrument][ltt_min_2]["oi"],
                                          'atp': candles_1[instrument][ltt_min_2]['atp']
                                          }
                                candleevent = CandleEvent(instrument, candle)
                                EventQ.put(candleevent)
                    else:
                        candles_1[instrument]={}
                        print("created dict for "+str(instrument))

                    ### Forming 15 Min Candles...
                    if instrument in candles_15:
                        if ltt_min_15 in candles_15[instrument]:
                            #print(tick)

                            candles_15[instrument][ltt_min_15]["high"] = max(candles_15[instrument][ltt_min_15]["high"],
                                                                             tick["last_price"])  # 1
                            candles_15[instrument][ltt_min_15]["low"] = min(candles_15[instrument][ltt_min_15]["low"],
                                                                            tick["last_price"])  # 2
                            candles_15[instrument][ltt_min_15]["close"] = tick["last_price"]  # 3
                            if tick["tradable"]:
                                candles_15[instrument][ltt_min_15]["volume"] = tick["volume"]
                                candles_15[instrument][ltt_min_15]["oi"] = tick['oi']
                                candles_15[instrument][ltt_min_15]["atp"] = tick['average_price']

                                if ltt_min_215 in candles_15[instrument]:
                                    candles_15[instrument][ltt_min_15]["vol"] = tick["volume"] -candles_15[instrument][ltt_min_215][
                                                                                    "volume"]  # 3.5
                            else:
                                candles_15[instrument][ltt_min_15]["volume"] = 0
                                candles_15[instrument][ltt_min_15]["oi"] = 0
                                candles_15[instrument][ltt_min_15]["atp"] = 0

                        else:
                            #print(instrument,str(ltt_min_15),candles_15[instrument][ltt_min_15])
                            candles_15[instrument][ltt_min_15] = {}
                            candles_15[instrument][ltt_min_15]["high"] = copy(tick["last_price"])  # 8
                            candles_15[instrument][ltt_min_15]["low"] = copy(tick["last_price"])
                            candles_15[instrument][ltt_min_15]["open"] = copy(tick["last_price"])
                            candles_15[instrument][ltt_min_15]["close"] = copy(tick["last_price"])
                            candles_15[instrument][ltt_min_15]["volume"] = 0
                            candles_15[instrument][ltt_min_15]["vol"] = 0
                            candles_15[instrument][ltt_min_15]["oi"] = 0
                            candles_15[instrument][ltt_min_15]["atp"] = 0
                            if ltt_min_215 in candles_15[instrument]:
                                #print(candles_15)
                                candle = {"token": instrument, "Time": ltt_min_215,
                                          "open": candles_15[instrument][ltt_min_215]["open"],
                                          "high": candles_15[instrument][ltt_min_215]["high"],
                                          "low": candles_15[instrument][ltt_min_215]["low"],
                                          "close": candles_15[instrument][ltt_min_215]["close"],
                                          "volume": candles_15[instrument][ltt_min_215]["vol"],
                                          "oi": candles_15[instrument][ltt_min_215]["oi"],
                                          'atp': candles_15[instrument][ltt_min_215]['atp']
                                          }
                                candleevent = CandleEvent15Min(instrument, candle)
                                EventQ.put(candleevent)
                    else:
                        candles_15[instrument]={}
                        print("created dict for "+str(instrument))

            elif event.type=="CANDLE":
                #print(event.type)
                #print(event.symbol,event.data)
                event.data.update(token_dict[event.symbol])
                df = pd.DataFrame(event.data,index=[0])
                print(df)
            elif event.type=="15MinCANDLE":
                #print(event.symbol, event.data)
                event.data.update(token_dict[event.symbol])
                df = pd.DataFrame(event.data, index=[0])
                # print(df)
                print(df)

                #print('\n')

if __name__ == "__main__":
    main()

yes .. you can.. it wont interfere with the candle formation

great thanks 👍

sorry for silly question btw what is the purpose if this event.data.update(token_dict[event.symbol])
i only took data from event.data

right.. the candle stick data is present in event.data
But the symbol is not present. It has only zerodha's instrument token
If u insert this candlestick in database, u would need to query by symbol
Thats why that line adds the symbol.

Ohh I see thank you for your inputs

how to use candles_1 dict in TA-Lib to get some indicator values?
Do i need to covert this dict to dataframe? I believe I can use candles_1 dict directly? could you please guide
I need to access each candles sequentially to place/exit trade, i think it would be ordered by timestamp already?

can someone post the output of print(df) for multiple symbols?

Hey, thanks for the code. I have a doubt though about the latency here, if the processing of ticks is slower than the speed of the incoming ticks, then at one point, there will be significant delays and the memory usage will also shoot up. For example , hypothetically if the ticks are coming in at every 1 second, and the tick processing code takes 2 seconds, then after like say 1 hour or so, we would still be processing 30 min ago ticks, also in our queue all these previous 30 min ticks would be stored, waiting to be processed. I chose some arbitrary numbers to only explain my doubt. And I don't have kiteconnect API to test this out. So kindly shed some light on this

why doesn't it work for the stocks only for indices
getting following error:

[4430593]
created dict for 4430593
created dict for 4430593
Traceback (most recent call last):
File "/Users/dhruv/Desktop/Stock-TA/ACML/ticker.py", line 382, in
main()
File "/Users/dhruv/Desktop/Stock-TA/ACML/ticker.py", line 251, in main
candles_1[instrument][ltt_min_1]["high"] = max(candles_1[instrument][ltt_min_1]["high"],
TypeError: max() takes 1 positional argument but 2 were given

Hi @oldmonkABA , does this work in real time candle formation ? Thanks in advance.