Skip to content

Instantly share code, notes, and snippets.

@adityajn105
Last active November 11, 2020 04:19
Show Gist options
  • Save adityajn105/fd4aec6637c9b42c111fa8210c461419 to your computer and use it in GitHub Desktop.
Save adityajn105/fd4aec6637c9b42c111fa8210c461419 to your computer and use it in GitHub Desktop.
Important Python Functionalities and Libraries

1. lambda Function

It is an anonymous function which can be used as a parameter function with sorted/sort,bisect,map,filter,reduce

>> lambda x: x*2;
>> lambda (x,y) : x+y

2. map

It is used to perform some operations on all items of sequence in one line and get a map object is iterable and can be converted to sequence

>> ls = [1,2,3,4,5,6,7]
>> sqrls = map( lambda x: x**2, ls )
<map object at 0x7f8848ff3048>
>> list(sqrls)
[1, 4, 9, 16, 25, 36, 49]<br>

3. filter

It is used to filter some items from a sequence to form a new sequence

>> ls = [1,4,6,7,9,11,15,16,17,19,21]
>> mod3 = filter(lambda x: x%3==0, ls )
>> mod3
<filter object at 0x7f5d424fd588>
>> mod3 = list(mod3)
>> mod3
[6, 9, 15, 21]

4. reduce

It is used to get a single value result from a whole sequence

>> ls = [1,4,6,7,9,11,15,16,17,19,21]
>> from functools import reduce
>> no = reduce(lambda x,y : x+y, ls)
>> no
126

5. zip

It combines n list of equal sizes and form a iterable of same size containing n sized tuples

>> questions = ['name', 'quest', 'favorite color']
>> answers = ['lancelot', 'the holy grail', 'blue']
>> for q, a in zip(questions, answers):
...     print('What is your {0}?  It is {1}.'.format(q, a))

6. PriorityQueue

It is a queue that keep the elements sorted

>> from queue import PriorityQueue
>> q = PriorityQueue()
>> ls = [ (34,'b'), (41,'d'), (11,'a'), (34,'c'), (36,'e') ]
>> for i in range(5):
...     q.put(ls[i]) # sort by first element than second, only true for integers
>> q.queue
[(11, 'a'), (34, 'c'), (34, 'b'), (41, 'd'), (36, 'e')]
>> len(q.queue)
5
>> q.get()  #pop first element
(11, 'a')

7. Generators

Function that returns an object(iterator) which can be iterated over, we can create it by using yield statement. Methods like __iter__() and __next__() are implemented automatically. We can use next().

>>> def my_gen():
...     i=0
...     while True:
...          yield i
...          i+=1
>>> a = my_gen()
>>> for i in range(5):
...     print(next(a))
0
1
2
3
4

8. Multiprocessing

* Using Joblib :

```
    >> from joblib import Parallel, delayed
    >> import requests
    >> def getImage(index, link ):
       ...response = requests.get(link, stream=True)
       ...with open(f"images/{index}.jpg", 'wb') as f:
       ......for data in response:
       .........f.write(data)
    >> Parallel(n_jobs=-2, prefer="threads")(delayed(getImage)(index, link) for index, link in index_link)  
```

9. Decorators

It takes some function and add some more functionality to it and return it. Decorators acts as a wrapper to that function.

>>> def smart_divide(func):
...   def inner(a,b):
...     print("We are dividing",a,"&",b)
...     if b==0:
...       print("Whoops!!")
...       return
...     return func(a,b)
...   return inner
>>> @smart_divide
... def divide(a,b):
...     return a/b
>>> divide(6,2)
We are dividing 6 & 2
3.0
>>> divide(6,0)
We are dividing 6 & 0
Whoops!!

10. Magic Functions

Operator Overloading, Callable object, iterable object, property decorator, method overloading

class Person:
    def __init__(self,name="xyz",marks=0,dob="0/0/0",sub_marks=[]):
        #constructor
        self.name=name
        self.marks=marks
        self.dob=dob
        self.sub_marks = sub_marks
        self.index = 0
        
    def __int__(self):
        #implement int() method, | __str__ "str()", __long__ "long()", __float__ "float()", __hex__ "hex()"  
        return self.marks
    
    def __repr__(self):
        #represent object as string
        return f"Name:{self.name}, Marks:{self.marks}, Date of Birth:{self.dob}"
    
    def __add__(self,person):
        #operator overloading "+", | __mul__ "*", __sub__ "-", __mod__ "%", __pow__ "**", __floordiv__ "//"
        return Person(self.name+' and '+person.name,self.marks+person.marks)
    
    def __ge__(self,person):
        #operator overloading ">=", | __lt__ "<", __le__ "<=", __eq__ "=", __ne__ "!="
        return self.marks >= person.marks
    
    def __eq__(self,person):
        #operator overloading "==", | __ne__ "!="
        return self.marks == person.marks
    
    def __and__(self, person):
        #operator overloading "&", | __xor__ "^", __or__ "|"
        return self.ispassed & person.ispassed
    
    def __pos__(self):
        #urinary operator "+", | __neg__ "-", __abs__ "abs()", __invert__ "~"
        if self.marks <= 33: return "D"
        elif self.marks > 75: return "A"
        elif self.marks > 50: return "B"
        else: return "C"
    
    def __totalsub(self):
        #private function
        return len(self.sub_marks)
    
    @property
    def ispassed(self):
        #@property decorator is built in attribute to give property using function
        return self.marks > 33
    
    def __bool__(self):
        #builtin bool() method to return True or False. can be used in if, while etc
        return self.ispassed
    
    def __call__(self,new_marks):
        #__call__ method can be used to turn the instances of the class into callables
        self.sub_marks.append(new_marks)
    
    def __getitem__(self, index):
        #behave class as a list
        return self.sub_marks[index]  
    
    def __len__(self):
        #implement len() attr of class
        return len(self.sub_marks)
        
    @staticmethod
    def about():
        #can be called without object
        return "student class"
        
>> person1 = Person("John Doe",43,'2001/02/11',sub_marks=[10,3,5,8,4,5,8])
>> person2 = Person("Paul Woe",21,'1995/01/22')
>> person1, person2
(Name:John Doe, Marks:43, Date of Birth:2001/02/11,
 Name:Paul Woe, Marks:21, Date of Birth:1995/01/22)

>> person1+person2
Name:John Doe and Paul Woe, Marks:64, Date of Birth:0/0/0

>> person1 & person2, person1.ispassed, person2.ispassed, bool(person1),bool(person2), person1==person2
(False, True, False, True, False, False)

>> for m in person1:
   ... print(m, end=" ")
10 3 5 8 4 5 8 

>> person1(11)
>> print(list(person1))
[10, 3, 5, 8, 4, 5, 8, 11]
>> sum(person1),len(person1), person1[4],person1[1]
(54, 8, 4, 3)

>> +person1
'C'

>> Person.about()
'student class'

Collections

1. OrderedDict

With an OrderedDict, the order of insertion is maintained when key and values are inserted into the dictionary. If we try to insert a key again, this will overwrite the previous value for that key.

>> from collections import OrderedDict
>> od = OrderedDict([ (11, 'a'), (9, 'c'),(17, 'b')])

>> for key, value in roll_no.items():
   ...print(key, value)
(11, 'a')
(9, 'c')
(17, 'b')

2. defaultdict

The default dictionary can contain duplicate keys. The advantage of using the default dictionary is that we can collect items which belong to the same key.

>> from collections import defaultdict
>> marks = [ ('a', 5), ('b', 92), ('c', 99), ('c', 98) ]
>> dict_marks = defaultdict(list)
>> for key, value in marks:
   ...dict_marks[key].append(value)
>> print(list(dict_marks.items()))
[('a', [89]), ('b', [92]), ('c', [99, 98])]

3. Counter

The Counter collections allow us to keep a count of all the items which are inserted into the collection with the keys.

>> from collections import Counter
>> Counter(['a','b','a','c','d','a','c','b','e'])
Counter({'a': 3, 'b': 2, 'c': 2, 'd': 1, 'e': 1})

4. namedtuple

>> from collections import namedtuple
>> User = namedtuple('User', ['name', 'age', 'gender'])
>> aj = User(name='Aditya', age=23, gender='M')
>> aj
User(name='Aditya', age=23, gender='M')
>> aj.name,aj.age
('Aditya', 23)

5. deque

A Deque is a double-ended queue which allows us to add and remove elements from both the ends. This enhances the capabilities of a stack or a queue.

>> dq = deque(['a','b','a','c','d','a','c','b','e'])
>> dq.popleft(),dq.pop()
('a', 'e')
>> dq.append('a')
>> dq.appendleft('e')
>> dq.insert(5,'j')
>> dq
deque(['e', 'b', 'a', 'c', 'd', 'j', 'a', 'c', 'b', 'a'])

Datetime package

1. Get Current DateTime

Get Day, Month, Year, hour, minute, second, microseconds

>> import datetime
>> now = datetime.datetime.now()
>> now
datetime.datetime(2019, 9, 28, 0, 21, 9, 69874)

>> print(now.day,now.month,now.year,now.hour,now.minute,now.second,now.microsecond)
28 9 2019 0 21 9 69874

>> now.isoformat()
'2019-09-28T00:21:09.069874'

2. From TimeStamp

Timestamp is a float value which represents datetime. It is number of seconds after 1 January 1970. Through timestamp it is easy to perform various operations on datetime.

>> now.timestamp() #seconds.microseconds
1569610269.069874

>> datetime.datetime.fromtimestamp( now.timestamp() )
datetime.datetime(2019, 9, 28, 0, 21, 9, 69874)

>> td = datetime.timedelta(days = 5, hours = 1, seconds = 1, microseconds = 2)
>> td
datetime.timedelta(5, 3601, 2)

>> datetime.datetime.fromtimestamp(now.timestamp()+td.total_seconds()) #adding timedelta as no of seconds
datetime.datetime(2019, 10, 3, 1, 21, 10, 69876)

3. Dealing with only dates

>> today = datetime.date.today()
>> today
datetime.date(2019, 9, 28)

>> today.day,today.month,today.year
(28, 9, 2019)

>> datetime.date.fromtimestamp(now.timestamp())
datetime.date(2019, 9, 28)

>> today.isoformat()
'2019-09-28'

4. Difference of two datetime - TimeDelta

A better way to manipulate datetime is to use TimeDelta, it is much faster and easier to use.

>> td = now-datetime.datetime(year=1996,month=5,day=10)
>> td
datetime.timedelta(8541, 1269, 69874)

>> td.days, td.total_seconds(), td.days/365
(8541, 737943669.069874, 23.4)

>> datetime.timedelta(days=5,hours=1,minutes=30,seconds=1,microseconds=2) #define timedelta (only below days allowed)
datetime.timedelta(5, 5401, 2)

>> now + datetime.timedelta(days=2,hours=6)
datetime.datetime(2019, 9, 30, 6, 21, 9, 69874)

5. strftime : Displaying datetime in readable/string formats

  • %Y : year (0001,..., 2018, 2019,..., 9999), %y : (01,...,18,19,...,99)
  • %m : month (01, 02, ..., 11, 12), %B : (January, February,...,December), %b : (Jan, Feb,...,Dec)
  • %d : day (01, 02, ..., 30, 31)
  • %H : hour (00, 01, ..., 22, 23), %I : (01, 02,..., 12)
  • %M : minute (00, 01, ..., 58, 59)
  • %S : second (00, 01, ..., 58, 59)
  • %p : AM/PM, %P : am/pm
  • %D : date "%m/%d/%y" ( 09/28/19 )
  • %T : time "%H:%M:%S" ( 00:21:09 ), %r : "%I:%M:%S %p" (12:21:09 AM)
  • %A : day ( Monday, Tuesday,...,Sunday ), %a : (Mon, Tue,...,Sun)
>> now.strftime("%d %B %Y, %I:%M:%S %p")
'28 September 2019, 12:21:09 AM'

>> now.strftime("%A on %Y/%m/%d at %H hours %M minutes %S seconds")
'Saturday on 2019/09/28 at 00 hours 21 minutes 09 seconds'

6. strptime : Convert string datetime into datetime for computation

>> datetime_string = '28 September 2019, 12:21:09 AM'
>> now = datetime.datetime.strptime(datetime_string, "%d %B %Y, %I:%M:%S %p")
>> now
datetime.datetime(2019, 9, 28, 0, 21, 9)

>> nextday = now + datetime.timedelta(days=1)
>> nextday.strftime("%d %B %Y, %I:%M:%S %p")
'29 September 2019, 12:21:09 AM'

Itertools Package

Functions in itertools “operate” on iterators to produce more complex iterators.

>> iterable1 = [ 'a','b','c','d' ]
>> iterable2 = [ 1, 2, 3, 4 ]
>> iterable3 = [ 'a','b' ]

1. chain(*iterables)

returns a iterator which combination of multiple iterator

>> list(chain(iterable1,iterable2,iterable2,iterable3))
['a', 'b', 'c', 'd', 1, 2, 3, 4, 1, 2, 3, 4, 'a', 'b']

2. combinations(iterable, r)

return all possible 'n' sized combinations of iterator (also combinations_with_replacement is available)

>> list(combinations(iterable1,3))
[('a', 'b', 'c'), ('a', 'b', 'd'), ('a', 'c', 'd'), ('b', 'c', 'd')]

3. permutations(iterable, r)

return all possible 'n' sized permutations of iterator

>> list(permutations(iterable1,2))
[('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c')]

4. count(start=0, step=1)

return iterator object which is iterable infinitly takes start and step as parameter

>> iterator = count(5,2)
>> for i in range(5):
   ...print( next(iterator), end=" " )
5 7 9 11 13 

5. cycle(iterable)

Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely

>> iterator = cycle(iterable2)
>> for i in range(15):
   ...print( next(iterator), end=" " )
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 

6. product( *iterables, repeat=1)

Cartesian product of input iterables. Equivalent to nested for-loops.

>> list(product( iterable2,iterable3))
[(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b'), (3, 'a'), (3, 'b'), (4, 'a'), (4, 'b')]

7. repeat(object, times=None)

create an iterator which returns the object for the specified number of times. If not specified, returns the object endlessly.

>> list(repeat(iterable3,4))
[['a', 'b'], ['a', 'b'], ['a', 'b'], ['a', 'b']]

8. islice(iterable, start, stop, step=1)

Return an iterator whose next() metod returns selected values from an iterable.

>> list(islice(['a','b','c','d','e','f','j'],2,10,2))
['c', 'e', 'j']

9. compress(data, selectors)

Return data elements corresponding to true selector elements.

>> list(compress(iterable2,[True,False,True,False]))
[1, 3]

10. zip_longest( *iters, fillvalue=None)

Same as zip but when the shorter iterables are exhausted, the fillvalue is substituted in their place.

>> list(zip_longest(iterable2,iterable3,fillvalue='x'))
[(1, 'a'), (2, 'b'), (3, 'x'), (4, 'x')]

11. accumulate(iterable, func)

Return series of accumulated sums (or other binary function results).

>> list(accumulate(iterable2, lambda x,y: x+y))
[1, 3, 6, 10]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment