Implementation of ticks to 1min and 15 mins candles in zerodha kiteconnect using python queues. This code is modified version of the code given in http://ezeetrading.in/Articles/Candles_formation_from_tick_data_zerodha.html. The difference is that in on_ticks functions the only action performed is that the ticks are place in a event queue. This …

create-kiteconnect-candlestick-q.py

################## Ticks to candles in kiteconnect python ####################
# Author : Arun B
# Reference  : http://ezeetrading.in/Articles/Candles_formation_from_tick_data_zerodha.html
# Purpose : Convert ticks to candles by putting ticks in a queue. This redues time wasted in on_ticks function
################################################################################

from kiteconnect import KiteTicker
import datetime
from copy import copy
import queue
import os
import pandas as pd
import json

class Event(object):
    """
    Event is base class providing an interface for all subsequent 
    (inherited) events, that will trigger further events in the 
    trading infrastructure.   
    """
    pass

class TickEvent(Event):
    """
    Handles the event of receiving a new market ticks
    """

    def __init__(self,ticks):
        """
        Initialises the MarketEvent.
        """
        self.type = 'TICK'
        self.data = ticks

class CandleEvent(Event):
    """
    Handles the event of receiving a 1min candle
    """
    
    def __init__(self, symbol,candle):
        
        self.type = 'CANDLE'
        self.symbol = symbol
        self.data = candle

    def print_self(self):
        print ("CANDLE:",self.data)
        
class CandleEvent15Min(Event):
    """
    Handles the event of 15 min candle.
    
    """

    def __init__(self, symbol, candle):
        """
        Initialises the 15mincandle event.
        """

        self.type = '15MinCANDLE'
        self.symbol = symbol
        self.data = candle

    def print_self(self):
        """
        Outputs the values within the Order.
        """
        print("CANDLE: = ",self.data)

# Enter the user api_key and the access_token generated for that day
credentials={'api_key': " ", 'access_token': " "}
# Initialise
kws = KiteTicker(credentials["api_key"], credentials["access_token"])

#token_dict = {256265:'NIFTY 50',260105:'NIFTY BANK'} #prev definition is wrong. It has to be nested dictionary
token_dict = {256265:{'Symbol':'NIFTY 50'},260105:{'Symbol':'NIFTY BANK'}} #you can put any F&O tokens here
# Creation of 1min and 15 min candles
candles_1={}
candles_15={}

EventQ = queue.Queue()


def on_ticks(ws, ticks):
    # Callback to receive ticks.
    tick = TickEvent(ticks)
    EventQ.put(tick)
  

def on_connect(ws, response):
    # Callback on successful connect.
    # Subscribe to a list of instrument_tokens (RELIANCE and ACC here).
    instruments=list(token_dict.keys())
    print(instruments)
    #exit()
    ws.subscribe(instruments)

    # Set tick in `full` mode.
    ws.set_mode(ws.MODE_FULL, instruments)

kws.on_ticks = on_ticks
kws.on_connect = on_connect
# Infinite loop on the main thread. Nothing after this will run.
# You have to use the pre-defined callbacks to manage subscriptions.
kws.connect(threaded=True)
def main():
    while True:
        try:
            event = EventQ.get(False)
            #print (ticks)
        except queue.Empty:
            continue
        else:
            if event.type=='TICK':
                ticks = event.data
                for tick in ticks:
                    instrument=tick["instrument_token"]
                    #instrument = token_dict[instrument_token]
                    #print(instrument_token,instrument)
                    ltt=tick["timestamp"]
                    #print(tick)
                    ltt_min_1=datetime.datetime(ltt.year, ltt.month, ltt.day, ltt.hour,ltt.minute)
                    ltt_min_2=ltt_min_1 - datetime.timedelta(minutes=1)
                    ltt_min_15 = datetime.datetime(ltt.year, ltt.month, ltt.day, ltt.hour, ltt.minute//15 * 15)
                    ltt_min_215 = ltt_min_15 - datetime.timedelta(minutes=15)
                    #print(ltt_min_1,end='\r')
                    #print(ltt_min_15,ltt_min_215)
                    #exit()
                    # For any other timeframe. Simply change ltt_min_1 variable defination.
                    # e.g.
                    # ltt_min_15=datetime.datetime(ltt.year, ltt.month, ltt.day, ltt.hour,ltt.minute//15*15)

                    ### Forming 1 Min Candles...

                    if instrument in candles_1:
                        if ltt_min_1 in candles_1[instrument]:
                            #print(tick)

                            candles_1[instrument][ltt_min_1]["high"] = max(candles_1[instrument][ltt_min_1]["high"],
                                                                             tick["last_price"])  # 1
                            candles_1[instrument][ltt_min_1]["low"] = min(candles_1[instrument][ltt_min_1]["low"],
                                                                            tick["last_price"])  # 2
                            candles_1[instrument][ltt_min_1]["close"] = tick["last_price"]  # 3
                            if tick["tradable"]: # for F & O 
                                candles_1[instrument][ltt_min_1]["volume"] = tick["volume"]
                                candles_1[instrument][ltt_min_1]["oi"] = tick['oi']
                                candles_1[instrument][ltt_min_1]["atp"] = tick['average_price']

                                if ltt_min_2 in candles_1[instrument]:
                                    candles_1[instrument][ltt_min_1]["vol"] = tick["volume"] -candles_1[instrument][ltt_min_2][
                                                                                    "volume"]  # 3.5
                            else:
                                candles_1[instrument][ltt_min_1]["volume"] = 0
                                candles_1[instrument][ltt_min_1]["oi"] = 0
                                candles_1[instrument][ltt_min_1]["atp"] = 0

                        else:
                            # print(instrument,str(ltt_min_1),candles_1[instrument][ltt_min_1])
                            candles_1[instrument][ltt_min_1] = {}
                            candles_1[instrument][ltt_min_1]["high"] = copy(tick["last_price"])  # 8
                            candles_1[instrument][ltt_min_1]["low"] = copy(tick["last_price"])
                            candles_1[instrument][ltt_min_1]["open"] = copy(tick["last_price"])
                            candles_1[instrument][ltt_min_1]["close"] = copy(tick["last_price"])
                            candles_1[instrument][ltt_min_1]["volume"] = 0
                            candles_1[instrument][ltt_min_1]["vol"] = 0
                            candles_1[instrument][ltt_min_1]["oi"] = 0
                            candles_1[instrument][ltt_min_1]["atp"] = 0
                            if ltt_min_2 in candles_1[instrument]:
                                # print(candles_1)
                                candle = {"token": instrument, "Time": ltt_min_2,
                                          "open": candles_1[instrument][ltt_min_2]["open"],
                                          "high": candles_1[instrument][ltt_min_2]["high"],
                                          "low": candles_1[instrument][ltt_min_2]["low"],
                                          "close": candles_1[instrument][ltt_min_2]["close"],
                                          "volume": candles_1[instrument][ltt_min_2]["vol"],
                                          "oi": candles_1[instrument][ltt_min_2]["oi"],
                                          'atp': candles_1[instrument][ltt_min_2]['atp']
                                          }
                                candleevent = CandleEvent(instrument, candle)
                                EventQ.put(candleevent)
                    else:
                        candles_1[instrument]={}
                        print("created dict for "+str(instrument))

                    ### Forming 15 Min Candles...
                    if instrument in candles_15:
                        if ltt_min_15 in candles_15[instrument]:
                            #print(tick)

                            candles_15[instrument][ltt_min_15]["high"] = max(candles_15[instrument][ltt_min_15]["high"],
                                                                             tick["last_price"])  # 1
                            candles_15[instrument][ltt_min_15]["low"] = min(candles_15[instrument][ltt_min_15]["low"],
                                                                            tick["last_price"])  # 2
                            candles_15[instrument][ltt_min_15]["close"] = tick["last_price"]  # 3
                            if tick["tradable"]:
                                candles_15[instrument][ltt_min_15]["volume"] = tick["volume"]
                                candles_15[instrument][ltt_min_15]["oi"] = tick['oi']
                                candles_15[instrument][ltt_min_15]["atp"] = tick['average_price']

                                if ltt_min_215 in candles_15[instrument]:
                                    candles_15[instrument][ltt_min_15]["vol"] = tick["volume"] -candles_15[instrument][ltt_min_215][
                                                                                    "volume"]  # 3.5
                            else:
                                candles_15[instrument][ltt_min_15]["volume"] = 0
                                candles_15[instrument][ltt_min_15]["oi"] = 0
                                candles_15[instrument][ltt_min_15]["atp"] = 0

                        else:
                            #print(instrument,str(ltt_min_15),candles_15[instrument][ltt_min_15])
                            candles_15[instrument][ltt_min_15] = {}
                            candles_15[instrument][ltt_min_15]["high"] = copy(tick["last_price"])  # 8
                            candles_15[instrument][ltt_min_15]["low"] = copy(tick["last_price"])
                            candles_15[instrument][ltt_min_15]["open"] = copy(tick["last_price"])
                            candles_15[instrument][ltt_min_15]["close"] = copy(tick["last_price"])
                            candles_15[instrument][ltt_min_15]["volume"] = 0
                            candles_15[instrument][ltt_min_15]["vol"] = 0
                            candles_15[instrument][ltt_min_15]["oi"] = 0
                            candles_15[instrument][ltt_min_15]["atp"] = 0
                            if ltt_min_215 in candles_15[instrument]:
                                #print(candles_15)
                                candle = {"token": instrument, "Time": ltt_min_215,
                                          "open": candles_15[instrument][ltt_min_215]["open"],
                                          "high": candles_15[instrument][ltt_min_215]["high"],
                                          "low": candles_15[instrument][ltt_min_215]["low"],
                                          "close": candles_15[instrument][ltt_min_215]["close"],
                                          "volume": candles_15[instrument][ltt_min_215]["vol"],
                                          "oi": candles_15[instrument][ltt_min_215]["oi"],
                                          'atp': candles_15[instrument][ltt_min_215]['atp']
                                          }
                                candleevent = CandleEvent15Min(instrument, candle)
                                EventQ.put(candleevent)
                    else:
                        candles_15[instrument]={}
                        print("created dict for "+str(instrument))

            elif event.type=="CANDLE":
                #print(event.type)
                #print(event.symbol,event.data)
                event.data.update(token_dict[event.symbol])
                df = pd.DataFrame(event.data,index=[0])
                print(df)
            elif event.type=="15MinCANDLE":
                #print(event.symbol, event.data)
                event.data.update(token_dict[event.symbol])
                df = pd.DataFrame(event.data, index=[0])
                # print(df)
                print(df)

                #print('\n')

if __name__ == "__main__":
    main()

exact values are there in the error
[11490562, 11536898]

sure.. i will test for nifty fut and bank nifty fut .. and update tomorrow

thanks a lot . please try with bank nifty index , im trying to form a candle for bank nifty index

here is my updated script which is giving error pls see
https://gist.github.com/ashuthinks/8dbb49de9fc05a0e86b99e658503b46d

sure.. just check tomorrow at the end of the day.. i will fix whatever error is there.. let me test it in live market

detail error is -

updated. error was in definition of token_dict in line 75.. if u modify it.. it will work..

detail error is -

please check the update code

thanks bro....i m bit new in python ...can you guide what all should i change to make it for random time frames..like 12 min....7 miin etc

best way it to form it from 1 min candles..
Store the 1 min candles in database and then process them using another code.

or u have to define a new event for each of time frame and copy paste the code for various time frames
for 1 min it starts from

Forming 1 min candles

for 15 min it starts from

Forming 15 min candles

thnks

let me try :) thanks

one small question i have added code for 5 minutes candle i will check that close value is greater that CPR (pivot point) if yes then trade,
cpr calculation and value check like greater or less can i add after this line of code?
elif event.type == "5MinCANDLE":
i'm bit new in python (a .net developer)
thanks

yes.. event.data contains the candle data

i mean i can do calculation and take trade there is a good design no thread issue?

yes .. you can.. it wont interfere with the candle formation

great thanks 👍

sorry for silly question btw what is the purpose if this event.data.update(token_dict[event.symbol])
i only took data from event.data

right.. the candle stick data is present in event.data
But the symbol is not present. It has only zerodha's instrument token
If u insert this candlestick in database, u would need to query by symbol
Thats why that line adds the symbol.

Ohh I see thank you for your inputs

how to use candles_1 dict in TA-Lib to get some indicator values?
Do i need to covert this dict to dataframe? I believe I can use candles_1 dict directly? could you please guide
I need to access each candles sequentially to place/exit trade, i think it would be ordered by timestamp already?

can someone post the output of print(df) for multiple symbols?

Hey, thanks for the code. I have a doubt though about the latency here, if the processing of ticks is slower than the speed of the incoming ticks, then at one point, there will be significant delays and the memory usage will also shoot up. For example , hypothetically if the ticks are coming in at every 1 second, and the tick processing code takes 2 seconds, then after like say 1 hour or so, we would still be processing 30 min ago ticks, also in our queue all these previous 30 min ticks would be stored, waiting to be processed. I chose some arbitrary numbers to only explain my doubt. And I don't have kiteconnect API to test this out. So kindly shed some light on this

why doesn't it work for the stocks only for indices
getting following error:

[4430593]
created dict for 4430593
created dict for 4430593
Traceback (most recent call last):
File "/Users/dhruv/Desktop/Stock-TA/ACML/ticker.py", line 382, in
main()
File "/Users/dhruv/Desktop/Stock-TA/ACML/ticker.py", line 251, in main
candles_1[instrument][ltt_min_1]["high"] = max(candles_1[instrument][ltt_min_1]["high"],
TypeError: max() takes 1 positional argument but 2 were given

Hi @oldmonkABA , does this work in real time candle formation ? Thanks in advance.