Created
April 27, 2018 02:14
-
-
Save sasikanth513/7bf1fc736b9142bb1bb57973017ee84e to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ | |
| { | |
| "Author": "stackoverflow", | |
| "url": "https://stackoverflow.com", | |
| "format": "html", | |
| "tips": [ | |
| { | |
| "title": "What does the "yield" keyword do?", | |
| "body": "<p>To understand what <code>yield</code> does, you must understand what <em>generators</em> are. And before generators come <em>iterables</em>.</p>\n\n<h2>Iterables</h2>\n\n<p>When you create a list, you can read its items one by one. Reading its items one by one is called iteration:</p>\n\n<pre><code>>>> mylist = [1, 2, 3]\n>>> for i in mylist:\n... print(i)\n1\n2\n3\n</code></pre>\n\n<p><code>mylist</code> is an <em>iterable</em>. When you use a list comprehension, you create a list, and so an iterable:</p>\n\n<pre><code>>>> mylist = [x*x for x in range(3)]\n>>> for i in mylist:\n... print(i)\n0\n1\n4\n</code></pre>\n\n<p>Everything you can use \"<code>for... in...</code>\" on is an iterable; <code>lists</code>, <code>strings</code>, files...</p>\n\n<p>These iterables are handy because you can read them as much as you wish, but you store all the values in memory and this is not always what you want when you have a lot of values.</p>\n\n<h2>Generators</h2>\n\n<p>Generators are iterators, a kind of iterable <strong>you can only iterate over once</strong>. Generators do not store all the values in memory, <strong>they generate the values on the fly</strong>:</p>\n\n<pre><code>>>> mygenerator = (x*x for x in range(3))\n>>> for i in mygenerator:\n... print(i)\n0\n1\n4\n</code></pre>\n\n<p>It is just the same except you used <code>()</code> instead of <code>[]</code>. BUT, you <strong>cannot</strong> perform <code>for i in mygenerator</code> a second time since generators can only be used once: they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one.</p>\n\n<h2>Yield</h2>\n\n<p><code>yield</code> is a keyword that is used like <code>return</code>, except the function will return a generator.</p>\n\n<pre><code>>>> def createGenerator():\n... mylist = range(3)\n... for i in mylist:\n... yield i*i\n...\n>>> mygenerator = createGenerator() # create a generator\n>>> print(mygenerator) # mygenerator is an object!\n<generator object createGenerator at 0xb7555c34>\n>>> for i in mygenerator:\n... print(i)\n0\n1\n4\n</code></pre>\n\n<p>Here it's a useless example, but it's handy when you know your function will return a huge set of values that you will only need to read once.</p>\n\n<p>To master <code>yield</code>, you must understand that <strong>when you call the function, the code you have written in the function body does not run.</strong> The function only returns the generator object, this is a bit tricky :-)</p>\n\n<p>Then, your code will be run each time the <code>for</code> uses the generator.</p>\n\n<p>Now the hard part:</p>\n\n<p>The first time the <code>for</code> calls the generator object created from your function, it will run the code in your function from the beginning until it hits <code>yield</code>, then it'll return the first value of the loop. Then, each other call will run the loop you have written in the function one more time, and return the next value, until there is no value to return.</p>\n\n<p>The generator is considered empty once the function runs but does not hit <code>yield</code> anymore. It can be because the loop had come to an end, or because you do not satisfy an <code>\"if/else\"</code> anymore.</p>\n\n<hr>\n\n<h2>Your code explained</h2>\n\n<p>Generator:</p>\n\n<pre><code># Here you create the method of the node object that will return the generator\ndef _get_child_candidates(self, distance, min_dist, max_dist):\n\n # Here is the code that will be called each time you use the generator object:\n\n # If there is still a child of the node object on its left\n # AND if distance is ok, return the next child\n if self._leftchild and distance - max_dist < self._median:\n yield self._leftchild\n\n # If there is still a child of the node object on its right\n # AND if distance is ok, return the next child\n if self._rightchild and distance + max_dist >= self._median:\n yield self._rightchild\n\n # If the function arrives here, the generator will be considered empty\n # there is no more than two values: the left and the right children\n</code></pre>\n\n<p>Caller:</p>\n\n<pre><code># Create an empty list and a list with the current object reference\nresult, candidates = list(), [self]\n\n# Loop on candidates (they contain only one element at the beginning)\nwhile candidates:\n\n # Get the last candidate and remove it from the list\n node = candidates.pop()\n\n # Get the distance between obj and the candidate\n distance = node._get_dist(obj)\n\n # If distance is ok, then you can fill the result\n if distance <= max_dist and distance >= min_dist:\n result.extend(node._values)\n\n # Add the children of the candidate in the candidates list\n # so the loop will keep running until it will have looked\n # at all the children of the children of the children, etc. of the candidate\n candidates.extend(node._get_child_candidates(distance, min_dist, max_dist))\n\nreturn result\n</code></pre>\n\n<p>This code contains several smart parts:</p>\n\n<ul>\n<li><p>The loop iterates on a list but the list expands while the loop is being iterated :-) It's a concise way to go through all these nested data even if it's a bit dangerous since you can end up with an infinite loop. In this case, <code>candidates.extend(node._get_child_candidates(distance, min_dist, max_dist))</code> exhausts all the values of the generator, but <code>while</code> keeps creating new generator objects which will produce different values from the previous ones since it's not applied on the same node.</p></li>\n<li><p>The <code>extend()</code> method is a list object method that expects an iterable and adds its values to the list.</p></li>\n</ul>\n\n<p>Usually we pass a list to it:</p>\n\n<pre><code>>>> a = [1, 2]\n>>> b = [3, 4]\n>>> a.extend(b)\n>>> print(a)\n[1, 2, 3, 4]\n</code></pre>\n\n<p>But in your code it gets a generator, which is good because:</p>\n\n<ol>\n<li>You don't need to read the values twice.</li>\n<li>You may have a lot of children and you don't want them all stored in memory.</li>\n</ol>\n\n<p>And it works because Python does not care if the argument of a method is a list or not. Python expects iterables so it will work with strings, lists, tuples and generators! This is called duck typing and is one of the reason why Python is so cool. But this is another story, for another question...</p>\n\n<p>You can stop here, or read a little bit to see an advanced use of a generator:</p>\n\n<h2>Controlling a generator exhaustion</h2>\n\n<pre><code>>>> class Bank(): # let's create a bank, building ATMs\n... crisis = False\n... def create_atm(self):\n... while not self.crisis:\n... yield \"$100\"\n>>> hsbc = Bank() # when everything's ok the ATM gives you as much as you want\n>>> corner_street_atm = hsbc.create_atm()\n>>> print(corner_street_atm.next())\n$100\n>>> print(corner_street_atm.next())\n$100\n>>> print([corner_street_atm.next() for cash in range(5)])\n['$100', '$100', '$100', '$100', '$100']\n>>> hsbc.crisis = True # crisis is coming, no more money!\n>>> print(corner_street_atm.next())\n<type 'exceptions.StopIteration'>\n>>> wall_street_atm = hsbc.create_atm() # it's even true for new ATMs\n>>> print(wall_street_atm.next())\n<type 'exceptions.StopIteration'>\n>>> hsbc.crisis = False # trouble is, even post-crisis the ATM remains empty\n>>> print(corner_street_atm.next())\n<type 'exceptions.StopIteration'>\n>>> brand_new_atm = hsbc.create_atm() # build a new one to get back in business\n>>> for cash in brand_new_atm:\n... print cash\n$100\n$100\n$100\n$100\n$100\n$100\n$100\n$100\n$100\n...\n</code></pre>\n\n<p><strong>Note:</strong> For Python3 use<code>print(corner_street_atm.__next__())</code> or <code>print(next(corner_street_atm))</code></p>\n\n<p>It can be useful for various things like controlling access to a resource.</p>\n\n<h2>Itertools, your best friend</h2>\n\n<p>The itertools module contains special functions to manipulate iterables. Ever wish to duplicate a generator?\nChain two generators? Group values in a nested list with a one liner? <code>Map / Zip</code> without creating another list?</p>\n\n<p>Then just <code>import itertools</code>.</p>\n\n<p>An example? Let's see the possible orders of arrival for a 4 horse race:</p>\n\n<pre><code>>>> horses = [1, 2, 3, 4]\n>>> races = itertools.permutations(horses)\n>>> print(races)\n<itertools.permutations object at 0xb754f1dc>\n>>> print(list(itertools.permutations(horses)))\n[(1, 2, 3, 4),\n (1, 2, 4, 3),\n (1, 3, 2, 4),\n (1, 3, 4, 2),\n (1, 4, 2, 3),\n (1, 4, 3, 2),\n (2, 1, 3, 4),\n (2, 1, 4, 3),\n (2, 3, 1, 4),\n (2, 3, 4, 1),\n (2, 4, 1, 3),\n (2, 4, 3, 1),\n (3, 1, 2, 4),\n (3, 1, 4, 2),\n (3, 2, 1, 4),\n (3, 2, 4, 1),\n (3, 4, 1, 2),\n (3, 4, 2, 1),\n (4, 1, 2, 3),\n (4, 1, 3, 2),\n (4, 2, 1, 3),\n (4, 2, 3, 1),\n (4, 3, 1, 2),\n (4, 3, 2, 1)]\n</code></pre>\n\n<h2>Understanding the inner mechanisms of iteration</h2>\n\n<p>Iteration is a process implying iterables (implementing the <code>__iter__()</code> method) and iterators (implementing the <code>__next__()</code> method).\nIterables are any objects you can get an iterator from. Iterators are objects that let you iterate on iterables.</p>\n\n<p>More about it in this article about <a href=\"http://effbot.org/zone/python-for-statement.htm\" rel=\"noreferrer\">how for loops work</a>.</p>\n", | |
| "source": "so", | |
| "questionId": 231767 | |
| }, | |
| { | |
| "title": "What are metaclasses in Python?", | |
| "body": "<p>A metaclass is the class of a class. Like a class defines how an instance of the class behaves, a metaclass defines how a class behaves. A class is an instance of a metaclass.</p>\n\n<p><a href=\"https://i.stack.imgur.com/QQ0OK.png\" rel=\"noreferrer\"><img src=\"https://i.stack.imgur.com/QQ0OK.png\" alt=\"metaclass diagram\"></a></p>\n\n<p>While in Python you can use arbitrary callables for metaclasses (like <a href=\"https://stackoverflow.com/questions/100003/what-is-a-metaclass-in-python/100037#100037\">Jerub</a> shows), the more useful approach is actually to make it an actual class itself. <code>type</code> is the usual metaclass in Python. In case you're wondering, yes, <code>type</code> is itself a class, and it is its own type. You won't be able to recreate something like <code>type</code> purely in Python, but Python cheats a little. To create your own metaclass in Python you really just want to subclass <code>type</code>.</p>\n\n<p>A metaclass is most commonly used as a class-factory. Like you create an instance of the class by calling the class, Python creates a new class (when it executes the 'class' statement) by calling the metaclass. Combined with the normal <code>__init__</code> and <code>__new__</code> methods, metaclasses therefore allow you to do 'extra things' when creating a class, like registering the new class with some registry, or even replace the class with something else entirely.</p>\n\n<p>When the <code>class</code> statement is executed, Python first executes the body of the <code>class</code> statement as a normal block of code. The resulting namespace (a dict) holds the attributes of the class-to-be. The metaclass is determined by looking at the baseclasses of the class-to-be (metaclasses are inherited), at the <code>__metaclass__</code> attribute of the class-to-be (if any) or the <code>__metaclass__</code> global variable. The metaclass is then called with the name, bases and attributes of the class to instantiate it.</p>\n\n<p>However, metaclasses actually define the <em>type</em> of a class, not just a factory for it, so you can do much more with them. You can, for instance, define normal methods on the metaclass. These metaclass-methods are like classmethods, in that they can be called on the class without an instance, but they are also not like classmethods in that they cannot be called on an instance of the class. <code>type.__subclasses__()</code> is an example of a method on the <code>type</code> metaclass. You can also define the normal 'magic' methods, like <code>__add__</code>, <code>__iter__</code> and <code>__getattr__</code>, to implement or change how the class behaves.</p>\n\n<p>Here's an aggregated example of the bits and pieces:</p>\n\n<pre><code>def make_hook(f):\n \"\"\"Decorator to turn 'foo' method into '__foo__'\"\"\"\n f.is_hook = 1\n return f\n\nclass MyType(type):\n def __new__(mcls, name, bases, attrs):\n\n if name.startswith('None'):\n return None\n\n # Go over attributes and see if they should be renamed.\n newattrs = {}\n for attrname, attrvalue in attrs.iteritems():\n if getattr(attrvalue, 'is_hook', 0):\n newattrs['__%s__' % attrname] = attrvalue\n else:\n newattrs[attrname] = attrvalue\n\n return super(MyType, mcls).__new__(mcls, name, bases, newattrs)\n\n def __init__(self, name, bases, attrs):\n super(MyType, self).__init__(name, bases, attrs)\n\n # classregistry.register(self, self.interfaces)\n print \"Would register class %s now.\" % self\n\n def __add__(self, other):\n class AutoClass(self, other):\n pass\n return AutoClass\n # Alternatively, to autogenerate the classname as well as the class:\n # return type(self.__name__ + other.__name__, (self, other), {})\n\n def unregister(self):\n # classregistry.unregister(self)\n print \"Would unregister class %s now.\" % self\n\nclass MyObject:\n __metaclass__ = MyType\n\n\nclass NoneSample(MyObject):\n pass\n\n# Will print \"NoneType None\"\nprint type(NoneSample), repr(NoneSample)\n\nclass Example(MyObject):\n def __init__(self, value):\n self.value = value\n @make_hook\n def add(self, other):\n return self.__class__(self.value + other.value)\n\n# Will unregister the class\nExample.unregister()\n\ninst = Example(10)\n# Will fail with an AttributeError\n#inst.unregister()\n\nprint inst + inst\nclass Sibling(MyObject):\n pass\n\nExampleSibling = Example + Sibling\n# ExampleSibling is now a subclass of both Example and Sibling (with no\n# content of its own) although it will believe it's called 'AutoClass'\nprint ExampleSibling\nprint ExampleSibling.__mro__\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 100003 | |
| }, | |
| { | |
| "title": "Does Python have a ternary conditional operator?", | |
| "body": "<p>Yes, it was <a href=\"https://mail.python.org/pipermail/python-dev/2005-September/056846.html\" rel=\"noreferrer\" title=\"[Python-Dev] Conditional Expression Resolution\">added</a> in version 2.5.<br>\nThe syntax is:</p>\n\n<pre><code>a if condition else b\n</code></pre>\n\n<p>First <code>condition</code> is evaluated, then either <code>a</code> or <code>b</code> is returned based on the <a href=\"https://en.wikipedia.org/wiki/Boolean_data_type\" rel=\"noreferrer\" title=\"Boolean data type\">Boolean</a> value of <code>condition</code><br>\nIf <code>condition</code> evaluates to <em>True</em> <code>a</code> is returned, else <code>b</code> is returned. </p>\n\n<p>For example:</p>\n\n<pre><code>>>> 'true' if True else 'false'\n'true'\n>>> 'true' if False else 'false'\n'false'\n</code></pre>\n\n<p>Note that conditionals are an <em>expression</em>, not a <em>statement</em>. This means you can't use assignments or <code>pass</code> or other statements in a conditional:</p>\n\n<pre><code>>>> pass if False else x = 3\n File \"<stdin>\", line 1\n pass if False else x = 3\n ^\nSyntaxError: invalid syntax\n</code></pre>\n\n<p>In such a case, you have to use a normal <code>if</code> statement instead of a conditional.</p>\n\n<hr>\n\n<p>Keep in mind that it's frowned upon by some Pythonistas for several reasons:</p>\n\n<ul>\n<li>The order of the arguments is different from many other languages (such as C, Ruby, Java, etc.), which may lead to bugs when people unfamiliar with Python's \"surprising\" behaviour use it (they may reverse the order).</li>\n<li>Some find it \"unwieldy\", since it goes contrary to the normal flow of thought (thinking of the condition first and then the effects).</li>\n<li>Stylistic reasons.</li>\n</ul>\n\n<p>If you're having trouble remembering the order, then remember that if you read it out loud, you (almost) say what you mean. For example, <code>x = 4 if b > 8 else 9</code> is read aloud as <code>x will be 4 if b is greater than 8 otherwise 9</code>.</p>\n\n<p>Official documentation:</p>\n\n<ul>\n<li><a href=\"https://docs.python.org/3/reference/expressions.html#conditional-expressions\" rel=\"noreferrer\" title=\"Conditional expressions\">Conditional expressions</a></li>\n<li><a href=\"https://docs.python.org/3.3/faq/programming.html#is-there-an-equivalent-of-c-s-ternary-operator\" rel=\"noreferrer\" title=\"Is there an equivalent of C’s ”?:” ternary operator?\">Is there an equivalent of C’s ”?:” ternary operator?</a></li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 394809 | |
| }, | |
| { | |
| "title": "How to check whether a file exists?", | |
| "body": "<p>If the reason you're checking is so you can do something like <code>if file_exists: open_it()</code>, it's safer to use a <code>try</code> around the attempt to open it. Checking and then opening risks the file being deleted or moved or something between when you check and when you try to open it.</p>\n\n<p>If you're not planning to open the file immediately, you can use <a href=\"https://docs.python.org/2/library/os.path.html#os.path.isfile\" rel=\"noreferrer\"><code>os.path.isfile</code></a></p>\n\n<blockquote>\n <p>Return <code>True</code> if path is an existing regular file. This follows symbolic links, so both <a href=\"https://docs.python.org/2/library/os.path.html#os.path.islink\" rel=\"noreferrer\">islink()</a> and <a href=\"https://docs.python.org/2/library/os.path.html#os.path.isfile\" rel=\"noreferrer\">isfile()</a> can be true for the same path.</p>\n</blockquote>\n\n<pre><code>import os.path\nos.path.isfile(fname) \n</code></pre>\n\n<p>if you need to be sure it's a file.</p>\n\n<p>Starting with Python 3.4, the <a href=\"https://docs.python.org/3/library/pathlib.html#pathlib.Path.is_file\" rel=\"noreferrer\"><code>pathlib</code> module</a> offers an object-oriented approach (backported to <code>pathlib2</code> in Python 2.7):</p>\n\n<pre><code>from pathlib import Path\n\nmy_file = Path(\"/path/to/file\")\nif my_file.is_file():\n # file exists\n</code></pre>\n\n<p>To check a directory, do:</p>\n\n<pre><code>if my_file.is_dir():\n # directory exists\n</code></pre>\n\n<p>To check whether a <code>Path</code> object exists independently of whether is it a file or directory, use <code>exists()</code>:</p>\n\n<pre><code>if my_file.exists():\n # path exists\n</code></pre>\n\n<p>You can also use <code>resolve()</code> in a <code>try</code> block:</p>\n\n<pre><code>try:\n my_abs_path = my_file.resolve():\nexcept FileNotFoundError:\n # doesn't exist\nelse:\n # exists\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 82831 | |
| }, | |
| { | |
| "title": "What does if __name__ == "__main__": do?", | |
| "body": "<p>When the Python interpreter reads a source file, it executes all of the code found in it. </p>\n\n<p>Before executing the code, it will define a few special variables. For example, if the python interpreter is running that module (the source file) as the main program, it sets the special <code>__name__</code> variable to have a value <code>\"__main__\"</code>. If this file is being imported from another module, <code>__name__</code> will be set to the module's name.</p>\n\n<p>In the case of your script, let's assume that it's executing as the main function, e.g. you said something like</p>\n\n<pre><code>python threading_example.py\n</code></pre>\n\n<p>on the command line. After setting up the special variables, it will execute the <code>import</code> statement and load those modules. It will then evaluate the <code>def</code> block, creating a function object and creating a variable called <code>myfunction</code> that points to the function object. It will then read the <code>if</code> statement and see that <code>__name__</code> does equal <code>\"__main__\"</code>, so it will execute the block shown there.</p>\n\n<p>One reason for doing this is that sometimes you write a module (a <code>.py</code> file) where it can be executed directly. Alternatively, it can also be imported and used in another module. By doing the main check, you can have that code only execute when you want to run the module as a program and not have it execute when someone just wants to import your module and call your functions themselves.</p>\n\n<p>See <a href=\"http://ibiblio.org/g2swap/byteofpython/read/module-name.html\" rel=\"noreferrer\">this page</a> for some extra details.</p>\n", | |
| "source": "so", | |
| "questionId": 419163 | |
| }, | |
| { | |
| "title": "Calling an external command in Python", | |
| "body": "<p>Look at the <a href=\"https://docs.python.org/2/library/subprocess.html\" rel=\"noreferrer\">subprocess module</a> in the standard library:</p>\n\n<pre><code>from subprocess import call\ncall([\"ls\", \"-l\"])\n</code></pre>\n\n<p>The advantage of <strong>subprocess</strong> vs <strong>system</strong> is that it is more flexible (you can get the stdout, stderr, the \"real\" status code, better error handling, etc...). </p>\n\n<p>The <a href=\"https://docs.python.org/library/os.html#os.system\" rel=\"noreferrer\">official docs</a> recommend the <strong>subprocess</strong> module over the alternative os.system():</p>\n\n<blockquote>\n <p>The <strong>subprocess</strong> module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function [<a href=\"https://docs.python.org/library/os.html#os.system\" rel=\"noreferrer\"><code>os.system()</code></a>].</p>\n</blockquote>\n\n<p>The \"<a href=\"https://docs.python.org/2/library/subprocess.html#replacing-older-functions-with-the-subprocess-module\" rel=\"noreferrer\">Replacing Older Functions with the subprocess Module</a>\" section in the <strong>subprocess</strong> documentation may have some helpful recipes.</p>\n\n<p>Official documentation on the <strong>subprocess</strong> module:</p>\n\n<ul>\n<li><a href=\"https://docs.python.org/2/library/subprocess.html#module-subprocess\" rel=\"noreferrer\">Python 2 - subprocess</a></li>\n<li><a href=\"https://docs.python.org/3/library/subprocess.html#module-subprocess\" rel=\"noreferrer\">Python 3 - subprocess</a></li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 89228 | |
| }, | |
| { | |
| "title": "How to merge two dictionaries in a single expression?", | |
| "body": "<blockquote>\n <h1>How can I merge two Python dictionaries in a single expression?</h1>\n</blockquote>\n\n<p>For dictionaries <code>x</code> and <code>y</code>, <code>z</code> becomes a merged dictionary with values from <code>y</code> replacing those from <code>x</code>.</p>\n\n<ul>\n<li><p>In Python 3.5 or greater, :</p>\n\n<pre><code>z = {**x, **y}\n</code></pre></li>\n<li><p>In Python 2, (or 3.4 or lower) write a function:</p>\n\n<pre><code>def merge_two_dicts(x, y):\n z = x.copy() # start with x's keys and values\n z.update(y) # modifies z with y's keys and values & returns None\n return z\n</code></pre>\n\n<p>and</p>\n\n<pre><code>z = merge_two_dicts(x, y)\n</code></pre></li>\n</ul>\n\n<h2>Explanation</h2>\n\n<p>Say you have two dicts and you want to merge them into a new dict without altering the original dicts:</p>\n\n<pre><code>x = {'a': 1, 'b': 2}\ny = {'b': 3, 'c': 4}\n</code></pre>\n\n<p>The desired result is to get a new dictionary (<code>z</code>) with the values merged, and the second dict's values overwriting those from the first.</p>\n\n<pre><code>>>> z\n{'a': 1, 'b': 3, 'c': 4}\n</code></pre>\n\n<p>A new syntax for this, proposed in <a href=\"https://www.python.org/dev/peps/pep-0448\" rel=\"noreferrer\">PEP 448</a> and <a href=\"https://mail.python.org/pipermail/python-dev/2015-February/138564.html\" rel=\"noreferrer\">available as of Python 3.5</a>, is </p>\n\n<pre><code>z = {**x, **y}\n</code></pre>\n\n<p>And it is indeed a single expression. It is now showing as implemented in the <a href=\"https://www.python.org/dev/peps/pep-0478/#features-for-3-5\" rel=\"noreferrer\">release schedule for 3.5, PEP 478</a>, and it has now made its way into <a href=\"https://docs.python.org/dev/whatsnew/3.5.html#pep-448-additional-unpacking-generalizations\" rel=\"noreferrer\">What's New in Python 3.5</a> document.</p>\n\n<p>However, since many organizations are still on Python 2, you may wish to do this in a backwards compatible way. The classically Pythonic way, available in Python 2 and Python 3.0-3.4, is to do this as a two-step process:</p>\n\n<pre><code>z = x.copy()\nz.update(y) # which returns None since it mutates z\n</code></pre>\n\n<p>In both approaches, <code>y</code> will come second and its values will replace <code>x</code>'s values, thus <code>'b'</code> will point to <code>3</code> in our final result.</p>\n\n<h1>Not yet on Python 3.5, but want a <em>single expression</em></h1>\n\n<p>If you are not yet on Python 3.5, or need to write backward-compatible code, and you want this in a <em>single expression</em>, the most performant while correct approach is to put it in a function:</p>\n\n<pre><code>def merge_two_dicts(x, y):\n \"\"\"Given two dicts, merge them into a new dict as a shallow copy.\"\"\"\n z = x.copy()\n z.update(y)\n return z\n</code></pre>\n\n<p>and then you have a single expression:</p>\n\n<pre><code>z = merge_two_dicts(x, y)\n</code></pre>\n\n<p>You can also make a function to merge an undefined number of dicts, from zero to a very large number:</p>\n\n<pre><code>def merge_dicts(*dict_args):\n \"\"\"\n Given any number of dicts, shallow copy and merge into a new dict,\n precedence goes to key value pairs in latter dicts.\n \"\"\"\n result = {}\n for dictionary in dict_args:\n result.update(dictionary)\n return result\n</code></pre>\n\n<p>This function will work in Python 2 and 3 for all dicts. e.g. given dicts <code>a</code> to <code>g</code>:</p>\n\n<pre><code>z = merge_dicts(a, b, c, d, e, f, g) \n</code></pre>\n\n<p>and key value pairs in <code>g</code> will take precedence over dicts <code>a</code> to <code>f</code>, and so on.</p>\n\n<h1>Critiques of Other Answers</h1>\n\n<p>Don't use what you see in the formerly accepted answer:</p>\n\n<pre><code>z = dict(x.items() + y.items())\n</code></pre>\n\n<p>In Python 2, you create two lists in memory for each dict, create a third list in memory with length equal to the length of the first two put together, and then discard all three lists to create the dict. <strong>In Python 3, this will fail</strong> because you're adding two <code>dict_items</code> objects together, not two lists - </p>\n\n<pre><code>>>> c = dict(a.items() + b.items())\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\nTypeError: unsupported operand type(s) for +: 'dict_items' and 'dict_items'\n</code></pre>\n\n<p>and you would have to explicitly create them as lists, e.g. <code>z = dict(list(x.items()) + list(y.items()))</code>. This is a waste of resources and computation power. </p>\n\n<p>Similarly, taking the union of <code>items()</code> in Python 3 (<code>viewitems()</code> in Python 2.7) will also fail when values are unhashable objects (like lists, for example). Even if your values are hashable, <strong>since sets are semantically unordered, the behavior is undefined in regards to precedence. So don't do this:</strong></p>\n\n<pre><code>>>> c = dict(a.items() | b.items())\n</code></pre>\n\n<p>This example demonstrates what happens when values are unhashable:</p>\n\n<pre><code>>>> x = {'a': []}\n>>> y = {'b': []}\n>>> dict(x.items() | y.items())\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\nTypeError: unhashable type: 'list'\n</code></pre>\n\n<p>Here's an example where y should have precedence, but instead the value from x is retained due to the arbitrary order of sets:</p>\n\n<pre><code>>>> x = {'a': 2}\n>>> y = {'a': 1}\n>>> dict(x.items() | y.items())\n{'a': 2}\n</code></pre>\n\n<p>Another hack you should not use:</p>\n\n<pre><code>z = dict(x, **y)\n</code></pre>\n\n<p>This uses the <code>dict</code> constructor, and is very fast and memory efficient (even slightly more-so than our two-step process) but unless you know precisely what is happening here (that is, the second dict is being passed as keyword arguments to the dict constructor), it's difficult to read, it's not the intended usage, and so it is not Pythonic. </p>\n\n<p>Here's an example of the usage being <a href=\"https://code.djangoproject.com/attachment/ticket/13357/django-pypy.2.diff\" rel=\"noreferrer\">remediated in django</a>.</p>\n\n<p>Dicts are intended to take hashable keys (e.g. frozensets or tuples), but <strong>this method fails in Python 3 when keys are not strings.</strong></p>\n\n<pre><code>>>> c = dict(a, **b)\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\nTypeError: keyword arguments must be strings\n</code></pre>\n\n<p>From the <a href=\"https://mail.python.org/pipermail/python-dev/2010-April/099459.html\" rel=\"noreferrer\">mailing list</a>, Guido van Rossum, the creator of the language, wrote:</p>\n\n<blockquote>\n <p>I am fine with\n declaring dict({}, **{1:3}) illegal, since after all it is abuse of\n the ** mechanism.</p>\n</blockquote>\n\n<p>and </p>\n\n<blockquote>\n <p>Apparently dict(x, **y) is going around as \"cool hack\" for \"call\n x.update(y) and return x\". Personally I find it more despicable than\n cool.</p>\n</blockquote>\n\n<p>It is my understanding (as well as the understanding of the <a href=\"https://mail.python.org/pipermail/python-dev/2010-April/099485.html\" rel=\"noreferrer\">creator of the language</a>) that the intended usage for <code>dict(**y)</code> is for creating dicts for readability purposes, e.g.:</p>\n\n<pre><code>dict(a=1, b=10, c=11)\n</code></pre>\n\n<p>instead of </p>\n\n<pre><code>{'a': 1, 'b': 10, 'c': 11}\n</code></pre>\n\n<h2>Response to comments</h2>\n\n<blockquote>\n <p>Despite what Guido says, <code>dict(x, **y)</code> is in line with the dict specification, which btw. works for both Python 2 and 3. The fact that this only works for string keys is a direct consequence of how keyword parameters work and not a short-comming of dict. Nor is using the ** operator in this place an abuse of the mechanism, in fact ** was designed precisely to pass dicts as keywords. </p>\n</blockquote>\n\n<p>Again, it doesn't work for 3 when keys are non-strings. The implicit calling contract is that namespaces take ordinary dicts, while users must only pass keyword arguments that are strings. All other callables enforced it. <code>dict</code> broke this consistency in Python 2:</p>\n\n<pre><code>>>> foo(**{('a', 'b'): None})\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\nTypeError: foo() keywords must be strings\n>>> dict(**{('a', 'b'): None})\n{('a', 'b'): None}\n</code></pre>\n\n<p>This inconsistency was bad given other implementations of Python (Pypy, Jython, IronPython). Thus it was fixed in Python 3, as this usage could be a breaking change.</p>\n\n<p>I submit to you that it is malicious incompetence to intentionally write code that only works in one version of a language or that only works given certain arbitrary constraints.</p>\n\n<p>Another comment:</p>\n\n<blockquote>\n <p><code>dict(x.items() + y.items())</code> is still the most readable solution for Python 2. Readability counts. </p>\n</blockquote>\n\n<p>My response: <code>merge_two_dicts(x, y)</code> actually seems much clearer to me, if we're actually concerned about readability. And it is not forward compatible, as Python 2 is increasingly deprecated.</p>\n\n<h1>Less Performant But Correct Ad-hocs</h1>\n\n<p>These approaches are less performant, but they will provide correct behavior.\nThey will be <em>much less</em> performant than <code>copy</code> and <code>update</code> or the new unpacking because they iterate through each key-value pair at a higher level of abstraction, but they <em>do</em> respect the order of precedence (latter dicts have precedence)</p>\n\n<p>You can also chain the dicts manually inside a dict comprehension:</p>\n\n<pre><code>{k: v for d in dicts for k, v in d.items()} # iteritems in Python 2.7\n</code></pre>\n\n<p>or in python 2.6 (and perhaps as early as 2.4 when generator expressions were introduced):</p>\n\n<pre><code>dict((k, v) for d in dicts for k, v in d.items())\n</code></pre>\n\n<p><code>itertools.chain</code> will chain the iterators over the key-value pairs in the correct order:</p>\n\n<pre><code>import itertools\nz = dict(itertools.chain(x.iteritems(), y.iteritems()))\n</code></pre>\n\n<h1>Performance Analysis</h1>\n\n<p>I'm only going to do the performance analysis of the usages known to behave correctly. </p>\n\n<pre><code>import timeit\n</code></pre>\n\n<p>The following is done on Ubuntu 14.04</p>\n\n<p>In Python 2.7 (system Python):</p>\n\n<pre><code>>>> min(timeit.repeat(lambda: merge_two_dicts(x, y)))\n0.5726828575134277\n>>> min(timeit.repeat(lambda: {k: v for d in (x, y) for k, v in d.items()} ))\n1.163769006729126\n>>> min(timeit.repeat(lambda: dict(itertools.chain(x.iteritems(), y.iteritems()))))\n1.1614501476287842\n>>> min(timeit.repeat(lambda: dict((k, v) for d in (x, y) for k, v in d.items())))\n2.2345519065856934\n</code></pre>\n\n<p>In Python 3.5 (deadsnakes PPA):</p>\n\n<pre><code>>>> min(timeit.repeat(lambda: {**x, **y}))\n0.4094954460160807\n>>> min(timeit.repeat(lambda: merge_two_dicts(x, y)))\n0.7881555100320838\n>>> min(timeit.repeat(lambda: {k: v for d in (x, y) for k, v in d.items()} ))\n1.4525277839857154\n>>> min(timeit.repeat(lambda: dict(itertools.chain(x.items(), y.items()))))\n2.3143140770262107\n>>> min(timeit.repeat(lambda: dict((k, v) for d in (x, y) for k, v in d.items())))\n3.2069112799945287\n</code></pre>\n\n<h2>Resources on Dictionaries</h2>\n\n<ul>\n<li><a href=\"https://stackoverflow.com/questions/327311/how-are-pythons-built-in-dictionaries-implemented/44509302#44509302\">My explanation of Python's <strong>dictionary implementation</strong>, updated for 3.6.</a></li>\n<li><a href=\"https://stackoverflow.com/questions/1024847/add-new-keys-to-a-dictionary/27208535#27208535\">Answer on how to add new keys to a dictionary</a></li>\n<li><a href=\"https://stackoverflow.com/questions/209840/map-two-lists-into-a-dictionary-in-python/33737067#33737067\">Mapping two lists into a dictionary</a></li>\n<li>The official Python <a href=\"https://docs.python.org/3/tutorial/datastructures.html#dictionaries\" rel=\"noreferrer\">docs on dictionaries</a> </li>\n<li><a href=\"https://www.youtube.com/watch?v=66P5FMkWoVU\" rel=\"noreferrer\">The Dictionary Even Mightier</a> - talk by Brandon Rhodes at Pycon 2017</li>\n<li><a href=\"https://www.youtube.com/watch?v=npw4s1QTmPg\" rel=\"noreferrer\">Modern Python Dictionaries, A Confluence of Great Ideas</a> - talk by Raymond Hettinger at Pycon 2017</li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 38987 | |
| }, | |
| { | |
| "title": "How can I create a directory if it does not exist?", | |
| "body": "<p>I see two answers with good qualities, each with a small flaw, so I will give my take on it:</p>\n\n<p>Try <a href=\"https://docs.python.org/2/library/os.path.html#os.path.exists\" rel=\"noreferrer\"><code>os.path.exists</code></a>, and consider <a href=\"https://docs.python.org/2/library/os.html#os.makedirs\" rel=\"noreferrer\"><code>os.makedirs</code></a> for the creation.</p>\n\n<pre><code>import os\nif not os.path.exists(directory):\n os.makedirs(directory)\n</code></pre>\n\n<p>As noted in comments and elsewhere, there's a race condition - if the directory is created between the <code>os.path.exists</code> and the <code>os.makedirs</code> calls, the <code>os.makedirs</code> will fail with an <code>OSError</code>. Unfortunately, blanket-catching <code>OSError</code> and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc.</p>\n\n<p>One option would be to trap the <code>OSError</code> and examine the embedded error code (see <a href=\"https://stackoverflow.com/questions/273698/is-there-a-cross-platform-way-of-getting-information-from-pythons-oserror\">Is there a cross-platform way of getting information from Python’s OSError</a>):</p>\n\n<pre><code>import os, errno\n\ntry:\n os.makedirs(directory)\nexcept OSError as e:\n if e.errno != errno.EEXIST:\n raise\n</code></pre>\n\n<p>Alternatively, there could be a second <code>os.path.exists</code>, but suppose another created the directory after the first check, then removed it before the second one - we could still be fooled. </p>\n\n<p>Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular application being developed and its expected environment before choosing an implementation.</p>\n", | |
| "source": "so", | |
| "questionId": 273192 | |
| }, | |
| { | |
| "title": "How do I sort a dictionary by value?", | |
| "body": "<p>It is not possible to sort a dictionary, only to get a representation of a dictionary that is sorted. Dictionaries are inherently orderless, but other types, such as lists and tuples, are not. So you need an ordered data type to represent sorted values, which will be a list—probably a list of tuples.</p>\n\n<p>For instance,</p>\n\n<pre><code>import operator\nx = {1: 2, 3: 4, 4: 3, 2: 1, 0: 0}\nsorted_x = sorted(x.items(), key=operator.itemgetter(1))\n</code></pre>\n\n<p><code>sorted_x</code> will be a list of tuples sorted by the second element in each tuple. <code>dict(sorted_x) == x</code>.</p>\n\n<p>And for those wishing to sort on keys instead of values:</p>\n\n<pre><code>import operator\nx = {1: 2, 3: 4, 4: 3, 2: 1, 0: 0}\nsorted_x = sorted(x.items(), key=operator.itemgetter(0))\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 613183 | |
| }, | |
| { | |
| "title": "Does Python have a string 'contains' substring method?", | |
| "body": "<p>You can use the <a href=\"https://docs.python.org/reference/expressions.html#membership-test-details\" rel=\"noreferrer\"><code>in</code> operator</a>:</p>\n\n<pre><code>if \"blah\" not in somestring: \n continue\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 3437059 | |
| }, | |
| { | |
| "title": "How do I list all files of a directory?", | |
| "body": "<p><a href=\"https://docs.python.org/2/library/os.html#os.listdir\" rel=\"noreferrer\" title=\"os.listdir\"><code>os.listdir()</code></a> will get you everything that's in a directory - files and directories.</p>\n\n<p>If you want <em>just</em> files, you could either filter this down using <a href=\"https://docs.python.org/2/library/os.path.html#module-os.path\" rel=\"noreferrer\"><code>os.path</code></a>:</p>\n\n<pre><code>from os import listdir\nfrom os.path import isfile, join\nonlyfiles = [f for f in listdir(mypath) if isfile(join(mypath, f))]\n</code></pre>\n\n<p>or you could use <a href=\"https://docs.python.org/2/library/os.html#os.walk\" rel=\"noreferrer\" title=\"os.walk\"><code>os.walk()</code></a> which will yield two lists for each directory it visits - splitting into files and dirs for you. If you only want the top directory you can just break the first time it yields</p>\n\n<pre><code>from os import walk\n\nf = []\nfor (dirpath, dirnames, filenames) in walk(mypath):\n f.extend(filenames)\n break\n</code></pre>\n\n<p>And lastly, as that example shows, adding one list to another you can either use <a href=\"https://docs.python.org/3/tutorial/datastructures.html#more-on-lists\" rel=\"noreferrer\"><code>.extend()</code></a> or </p>\n\n<pre><code>>>> q = [1, 2, 3]\n>>> w = [4, 5, 6]\n>>> q = q + w\n>>> q\n[1, 2, 3, 4, 5, 6]\n</code></pre>\n\n<p>Personally, I prefer <code>.extend()</code></p>\n", | |
| "source": "so", | |
| "questionId": 3207219 | |
| }, | |
| { | |
| "title": "How do I check if a list is empty?", | |
| "body": "<pre><code>if not a:\n print(\"List is empty\")\n</code></pre>\n\n<p>Using the implicit booleanness of the empty list is quite pythonic.</p>\n", | |
| "source": "so", | |
| "questionId": 53513 | |
| }, | |
| { | |
| "title": "Difference between append vs. extend list methods in Python", | |
| "body": "<p><a href=\"https://docs.python.org/2/library/array.html?#array.array.append\" rel=\"noreferrer\"><code>append</code></a>: Appends object at end.</p>\n\n<pre><code>x = [1, 2, 3]\nx.append([4, 5])\nprint (x)\n</code></pre>\n\n<p>gives you: <code>[1, 2, 3, [4, 5]]</code></p>\n\n<hr>\n\n<p><a href=\"https://docs.python.org/2/library/array.html?#array.array.extend\" rel=\"noreferrer\"><code>extend</code></a>: Extends list by appending elements from the iterable.</p>\n\n<pre><code>x = [1, 2, 3]\nx.extend([4, 5])\nprint (x)\n</code></pre>\n\n<p>gives you: <code>[1, 2, 3, 4, 5]</code></p>\n", | |
| "source": "so", | |
| "questionId": 252703 | |
| }, | |
| { | |
| "title": "What is the difference between @staticmethod and @classmethod in Python?", | |
| "body": "<p>Maybe a bit of example code will help: Notice the difference in the call signatures of <code>foo</code>, <code>class_foo</code> and <code>static_foo</code>:</p>\n\n<pre><code>class A(object):\n def foo(self,x):\n print \"executing foo(%s,%s)\"%(self,x)\n\n @classmethod\n def class_foo(cls,x):\n print \"executing class_foo(%s,%s)\"%(cls,x)\n\n @staticmethod\n def static_foo(x):\n print \"executing static_foo(%s)\"%x \n\na=A()\n</code></pre>\n\n<p>Below is the usual way an object instance calls a method. The object instance, <code>a</code>, is implicitly passed as the first argument.</p>\n\n<pre><code>a.foo(1)\n# executing foo(<__main__.A object at 0xb7dbef0c>,1)\n</code></pre>\n\n<hr>\n\n<p><strong>With classmethods</strong>, the class of the object instance is implicitly passed as the first argument instead of <code>self</code>.</p>\n\n<pre><code>a.class_foo(1)\n# executing class_foo(<class '__main__.A'>,1)\n</code></pre>\n\n<p>You can also call <code>class_foo</code> using the class. In fact, if you define something to be\na classmethod, it is probably because you intend to call it from the class rather than from a class instance. <code>A.foo(1)</code> would have raised a TypeError, but <code>A.class_foo(1)</code> works just fine:</p>\n\n<pre><code>A.class_foo(1)\n# executing class_foo(<class '__main__.A'>,1)\n</code></pre>\n\n<p>One use people have found for class methods is to create <a href=\"https://stackoverflow.com/a/1950927/190597\">inheritable alternative constructors</a>.</p>\n\n<hr>\n\n<p><strong>With staticmethods</strong>, neither <code>self</code> (the object instance) nor <code>cls</code> (the class) is implicitly passed as the first argument. They behave like plain functions except that you can call them from an instance or the class:</p>\n\n<pre><code>a.static_foo(1)\n# executing static_foo(1)\n\nA.static_foo('hi')\n# executing static_foo(hi)\n</code></pre>\n\n<p>Staticmethods are used to group functions which have some logical connection with a class to the class.</p>\n\n<hr>\n\n<p><code>foo</code> is just a function, but when you call <code>a.foo</code> you don't just get the function,\nyou get a \"partially applied\" version of the function with the object instance <code>a</code> bound as the first argument to the function. <code>foo</code> expects 2 arguments, while <code>a.foo</code> only expects 1 argument.</p>\n\n<p><code>a</code> is bound to <code>foo</code>. That is what is meant by the term \"bound\" below:</p>\n\n<pre><code>print(a.foo)\n# <bound method A.foo of <__main__.A object at 0xb7d52f0c>>\n</code></pre>\n\n<p>With <code>a.class_foo</code>, <code>a</code> is not bound to <code>class_foo</code>, rather the class <code>A</code> is bound to <code>class_foo</code>.</p>\n\n<pre><code>print(a.class_foo)\n# <bound method type.class_foo of <class '__main__.A'>>\n</code></pre>\n\n<p>Here, with a staticmethod, even though it is a method, <code>a.static_foo</code> just returns\na good 'ole function with no arguments bound. <code>static_foo</code> expects 1 argument, and\n<code>a.static_foo</code> expects 1 argument too.</p>\n\n<pre><code>print(a.static_foo)\n# <function static_foo at 0xb7d479cc>\n</code></pre>\n\n<p>And of course the same thing happens when you call <code>static_foo</code> with the class <code>A</code> instead.</p>\n\n<pre><code>print(A.static_foo)\n# <function static_foo at 0xb7d479cc>\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 136097 | |
| }, | |
| { | |
| "title": "Accessing the index in 'for' loops?", | |
| "body": "<p>Using an additional state variable, such as an index variable (which you would normally use in languages such as C or PHP), is considered non-pythonic.</p>\n\n<p>The better option is to use the built-in function <a href=\"https://docs.python.org/3/library/functions.html#enumerate\" rel=\"noreferrer\" title=\"enumerate\"><code>enumerate()</code></a>, available in both Python 2 and 3:</p>\n\n<pre><code>for idx, val in enumerate(ints):\n print(idx, val)\n</code></pre>\n\n<p>Check out <a href=\"https://www.python.org/dev/peps/pep-0279/\" rel=\"noreferrer\" title=\"PEP 279\">PEP 279</a> for more.</p>\n", | |
| "source": "so", | |
| "questionId": 522563 | |
| }, | |
| { | |
| "title": "Using global variables in a function other than the one that created them", | |
| "body": "<p>You can use a global variable in other functions by declaring it as <code>global</code> in each function that assigns to it:</p>\n\n<pre><code>globvar = 0\n\ndef set_globvar_to_one():\n global globvar # Needed to modify global copy of globvar\n globvar = 1\n\ndef print_globvar():\n print(globvar) # No need for global declaration to read value of globvar\n\nset_globvar_to_one()\nprint_globvar() # Prints 1\n</code></pre>\n\n<p>I imagine the reason for it is that, since global variables are so dangerous, Python wants to make sure that you really know that's what you're playing with by explicitly requiring the <code>global</code> keyword.</p>\n\n<p>See other answers if you want to share a global variable across modules.</p>\n", | |
| "source": "so", | |
| "questionId": 423379 | |
| }, | |
| { | |
| "title": "How to make a chain of function decorators?", | |
| "body": "<p>Check out <a href=\"http://docs.python.org/reference/compound_stmts.html#function\" rel=\"noreferrer\">the documentation</a> to see how decorators work. Here is what you asked for:</p>\n\n<pre><code>def makebold(fn):\n def wrapped():\n return \"<b>\" + fn() + \"</b>\"\n return wrapped\n\ndef makeitalic(fn):\n def wrapped():\n return \"<i>\" + fn() + \"</i>\"\n return wrapped\n\n@makebold\n@makeitalic\ndef hello():\n return \"hello world\"\n\nprint hello() ## returns \"<b><i>hello world</i></b>\"\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 739654 | |
| }, | |
| { | |
| "title": "How do I install pip on Windows?", | |
| "body": "<h2>Python 2.7.9+ and 3.4+</h2>\n\n<p>Good news! <a href=\"https://docs.python.org/3/whatsnew/3.4.html\" rel=\"noreferrer\">Python 3.4</a> (released March 2014) and <a href=\"https://docs.python.org/2/whatsnew/2.7.html#pep-477-backport-ensurepip-pep-453-to-python-2-7\" rel=\"noreferrer\">Python 2.7.9</a> (released December 2014) ship with Pip. This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup. In shipping with a package manager, Python joins <a href=\"http://en.wikipedia.org/wiki/Ruby_%28programming_language%29\" rel=\"noreferrer\">Ruby</a>, <a href=\"http://en.wikipedia.org/wiki/Node.js\" rel=\"noreferrer\">Node.js</a>, <a href=\"http://en.wikipedia.org/wiki/Haskell_%28programming_language%29\" rel=\"noreferrer\">Haskell</a>, <a href=\"http://en.wikipedia.org/wiki/Perl\" rel=\"noreferrer\">Perl</a>, <a href=\"http://en.wikipedia.org/wiki/Go_%28programming_language%29\" rel=\"noreferrer\">Go</a>--almost every other contemporary language with a majority open-source community. Thank you Python.</p>\n\n<p>Of course, that doesn't mean Python packaging is problem solved. The experience remains frustrating. I discuss this <a href=\"https://stackoverflow.com/questions/2436731/does-python-have-a-package-module-management-system/13445719#13445719\">in Stack Overflow question <em>Does Python have a package/module management system?</em></a>.</p>\n\n<p>And, alas for everyone using Python 2.7.8 or earlier (a sizable portion of the community). There's no plan to ship Pip to you. Manual instructions follow.</p>\n\n<h2>Python 2 ≤ 2.7.8 and Python 3 ≤ 3.3</h2>\n\n<p>Flying in the face of its <a href=\"http://www.python.org/about/\" rel=\"noreferrer\">'batteries included'</a> motto, Python ships without a package manager. To make matters worse, Pip was--until recently--ironically difficult to install.</p>\n\n<h3>Official instructions</h3>\n\n<p>Per <a href=\"https://pip.pypa.io/en/stable/installing/#do-i-need-to-install-pip\" rel=\"noreferrer\">https://pip.pypa.io/en/stable/installing/#do-i-need-to-install-pip</a>:</p>\n\n<p>Download <a href=\"https://bootstrap.pypa.io/get-pip.py\" rel=\"noreferrer\"><code>get-pip.py</code></a>, being careful to save it as a <code>.py</code> file rather than <code>.txt</code>. Then, run it from the command prompt:</p>\n\n<pre><code>python get-pip.py\n</code></pre>\n\n<p>You possibly need an administrator command prompt to do this. Follow <em><a href=\"http://technet.microsoft.com/en-us/library/cc947813(v=ws.10).aspx\" rel=\"noreferrer\">Start a Command Prompt as an Administrator</a></em> (Microsoft TechNet).</p>\n\n<p>This installs the pip package, which (in Windows) contains ...\\Scripts\\pip.exe that path must be in PATH environment variable to use pip from the command line (see the second part of 'Alternative Instructions' for adding it to your PATH,</p>\n\n<h3>Alternative instructions</h3>\n\n<p>The official documentation tells users to install Pip and each of its dependencies from source. That's tedious for the experienced and prohibitively difficult for newbies.</p>\n\n<p>For our sake, Christoph Gohlke prepares Windows installers (<code>.msi</code>) for popular Python packages. He builds installers for all Python versions, both 32 and 64 bit. You need to:</p>\n\n<ol>\n<li><a href=\"http://www.lfd.uci.edu/~gohlke/pythonlibs/#setuptools\" rel=\"noreferrer\">Install setuptools</a></li>\n<li><a href=\"http://www.lfd.uci.edu/~gohlke/pythonlibs/#pip\" rel=\"noreferrer\">Install pip</a></li>\n</ol>\n\n<p>For me, this installed Pip at <code>C:\\Python27\\Scripts\\pip.exe</code>. Find <code>pip.exe</code> on your computer, then add its folder (for example, <code>C:\\Python27\\Scripts</code>) to your path (Start / Edit environment variables). Now you should be able to run <code>pip</code> from the command line. Try installing a package:</p>\n\n<pre><code>pip install httpie\n</code></pre>\n\n<p>There you go (hopefully)! Solutions for common problems are given below:</p>\n\n<h3>Proxy problems</h3>\n\n<p>If you work in an office, you might be behind an HTTP proxy. If so, set the environment variables <a href=\"http://docs.python.org/2/library/urllib.html\" rel=\"noreferrer\"><code>http_proxy</code> and <code>https_proxy</code></a>. Most Python applications (and other free software) respect these. Example syntax:</p>\n\n<pre><code>http://proxy_url:port\nhttp://username:password@proxy_url:port\n</code></pre>\n\n<p>If you're really unlucky, your proxy might be a Microsoft <a href=\"https://en.wikipedia.org/wiki/NT_LAN_Manager\" rel=\"noreferrer\">NTLM</a> proxy. Free software can't cope. The only solution is to install a free software friendly proxy that forwards to the nasty proxy. <a href=\"http://cntlm.sourceforge.net/\" rel=\"noreferrer\">http://cntlm.sourceforge.net/</a></p>\n\n<h3>Unable to find vcvarsall.bat</h3>\n\n<p>Python modules can be partly written in C or C++. Pip tries to compile from source. If you don't have a C/C++ compiler installed and configured, you'll see this cryptic error message.</p>\n\n<blockquote>\n <p>Error: Unable to find vcvarsall.bat</p>\n</blockquote>\n\n<p>You can fix that by <a href=\"https://stackoverflow.com/questions/2817869/error-unable-to-find-vcvarsall-bat\">installing a C++ compiler</a> such as <a href=\"http://en.wikipedia.org/wiki/MinGW\" rel=\"noreferrer\">MinGW</a> or <a href=\"http://en.wikipedia.org/wiki/Visual_C%2B%2B#32-bit_versions\" rel=\"noreferrer\">Visual C++</a>. Microsoft actually ships one specifically for use with Python. Or try <em><a href=\"http://aka.ms/vcpython27\" rel=\"noreferrer\">Microsoft Visual C++ Compiler for Python 2.7</a></em>.</p>\n\n<p>Often though it's easier to check <a href=\"http://www.lfd.uci.edu/~gohlke/pythonlibs/\" rel=\"noreferrer\">Christoph's site</a> for your package.</p>\n", | |
| "source": "so", | |
| "questionId": 4750806 | |
| }, | |
| { | |
| "title": "Understanding Python's slice notation", | |
| "body": "<p>It's pretty simple really:</p>\n\n<pre><code>a[start:end] # items start through end-1\na[start:] # items start through the rest of the array\na[:end] # items from the beginning through end-1\na[:] # a copy of the whole array\n</code></pre>\n\n<p>There is also the <code>step</code> value, which can be used with any of the above:</p>\n\n<pre><code>a[start:end:step] # start through not past end, by step\n</code></pre>\n\n<p>The key point to remember is that the <code>:end</code> value represents the first value that is <em>not</em> in the selected slice. So, the difference beween <code>end</code> and <code>start</code> is the number of elements selected (if <code>step</code> is 1, the default).</p>\n\n<p>The other feature is that <code>start</code> or <code>end</code> may be a <em>negative</em> number, which means it counts from the end of the array instead of the beginning. So:</p>\n\n<pre><code>a[-1] # last item in the array\na[-2:] # last two items in the array\na[:-2] # everything except the last two items\n</code></pre>\n\n<p>Similarly, <code>step</code> may be a negative number:</p>\n\n<pre><code>a[::-1] # all items in the array, reversed\na[1::-1] # the first two items, reversed\na[:-3:-1] # the last two items, reversed\na[-3::-1] # everything except the last two items, reversed\n</code></pre>\n\n<p>Python is kind to the programmer if there are fewer items than you ask for. For example, if you ask for <code>a[:-2]</code> and <code>a</code> only contains one element, you get an empty list instead of an error. Sometimes you would prefer the error, so you have to be aware that this may happen.</p>\n", | |
| "source": "so", | |
| "questionId": 509211 | |
| }, | |
| { | |
| "title": "Finding the index of an item given a list containing it in Python", | |
| "body": "<pre><code>>>> [\"foo\", \"bar\", \"baz\"].index(\"bar\")\n1\n</code></pre>\n\n<p>Reference: <a href=\"http://docs.python.org/2/tutorial/datastructures.html#more-on-lists\" rel=\"noreferrer\">Data Structures > More on Lists</a></p>\n", | |
| "source": "so", | |
| "questionId": 176918 | |
| }, | |
| { | |
| "title": "Check if a given key already exists in a dictionary", | |
| "body": "<p><code>in</code> is the intended way to test for the existence of a key in a <code>dict</code>.</p>\n\n<pre><code>d = dict()\n\nfor i in xrange(100):\n key = i % 10\n if key in d:\n d[key] += 1\n else:\n d[key] = 1\n</code></pre>\n\n<p>If you wanted a default, you can always use <code>dict.get()</code>:</p>\n\n<pre><code>d = dict()\n\nfor i in xrange(100):\n key = i % 10\n d[key] = d.get(key, 0) + 1\n</code></pre>\n\n<p>... and if you wanted to always ensure a default value for any key you can use <code>defaultdict</code> from the <code>collections</code> module, like so:</p>\n\n<pre><code>from collections import defaultdict\n\nd = defaultdict(lambda: 0)\n\nfor i in xrange(100):\n d[i % 10] += 1\n</code></pre>\n\n<p>... but in general, the <code>in</code> keyword is the best way to do it.</p>\n", | |
| "source": "so", | |
| "questionId": 1602934 | |
| }, | |
| { | |
| "title": "Difference between __str__ and __repr__?", | |
| "body": "<p>Alex summarized well but, surprisingly, was too succinct.</p>\n\n<p>First, let me reiterate the main points in Alex’s post:</p>\n\n<ul>\n<li>The default implementation is useless (it’s hard to think of one which wouldn’t be, but yeah)</li>\n<li><code>__repr__</code> goal is to be unambiguous</li>\n<li><code>__str__</code> goal is to be readable</li>\n<li>Container’s <code>__str__</code> uses contained objects’ <code>__repr__</code></li>\n</ul>\n\n<p><strong>Default implementation is useless</strong></p>\n\n<p>This is mostly a surprise because Python’s defaults tend to be fairly useful. However, in this case, having a default for <code>__repr__</code> which would act like:</p>\n\n<pre><code>return \"%s(%r)\" % (self.__class__, self.__dict__)\n</code></pre>\n\n<p>would have been too dangerous (for example, too easy to get into infinite recursion if objects reference each other). So Python cops out. Note that there is one default which is true: if <code>__repr__</code> is defined, and <code>__str__</code> is not, the object will behave as though <code>__str__=__repr__</code>.</p>\n\n<p>This means, in simple terms: almost every object you implement should have a functional <code>__repr__</code> that’s usable for understanding the object. Implementing <code>__str__</code> is optional: do that if you need a “pretty print” functionality (for example, used by a report generator).</p>\n\n<p><strong>The goal of <code>__repr__</code> is to be unambiguous</strong></p>\n\n<p>Let me come right out and say it — I do not believe in debuggers. I don’t really know how to use any debugger, and have never used one seriously. Furthermore, I believe that the big fault in debuggers is their basic nature — most failures I debug happened a long long time ago, in a galaxy far far away. This means that I do believe, with religious fervor, in logging. Logging is the lifeblood of any decent fire-and-forget server system. Python makes it easy to log: with maybe some project specific wrappers, all you need is a</p>\n\n<pre><code>log(INFO, \"I am in the weird function and a is\", a, \"and b is\", b, \"but I got a null C — using default\", default_c)\n</code></pre>\n\n<p>But you have to do the last step — make sure every object you implement has a useful repr, so code like that can just work. This is why the “eval” thing comes up: if you have enough information so <code>eval(repr(c))==c</code>, that means you know everything there is to know about <code>c</code>. If that’s easy enough, at least in a fuzzy way, do it. If not, make sure you have enough information about <code>c</code> anyway. I usually use an eval-like format: <code>\"MyClass(this=%r,that=%r)\" % (self.this,self.that)</code>. It does not mean that you can actually construct MyClass, or that those are the right constructor arguments — but it is a useful form to express “this is everything you need to know about this instance”.</p>\n\n<p>Note: I used <code>%r</code> above, not <code>%s</code>. You always want to use <code>repr()</code> [or <code>%r</code> formatting character, equivalently] inside <code>__repr__</code> implementation, or you’re defeating the goal of repr. You want to be able to differentiate <code>MyClass(3)</code> and <code>MyClass(\"3\")</code>.</p>\n\n<p><strong>The goal of <code>__str__</code> is to be readable</strong></p>\n\n<p>Specifically, it is not intended to be unambiguous — notice that <code>str(3)==str(\"3\")</code>. Likewise, if you implement an IP abstraction, having the str of it look like 192.168.1.1 is just fine. When implementing a date/time abstraction, the str can be \"2010/4/12 15:35:22\", etc. The goal is to represent it in a way that a user, not a programmer, would want to read it. Chop off useless digits, pretend to be some other class — as long is it supports readability, it is an improvement.</p>\n\n<p><strong>Container’s <code>__str__</code> uses contained objects’ <code>__repr__</code></strong></p>\n\n<p>This seems surprising, doesn’t it? It is a little, but how readable would</p>\n\n<pre><code>[moshe is, 3, hello\nworld, this is a list, oh I don't know, containing just 4 elements]\n</code></pre>\n\n<p>be? Not very. Specifically, the strings in a container would find it way too easy to disturb its string representation. In the face of ambiguity, remember, Python resists the temptation to guess. If you want the above behavior when you’re printing a list, just</p>\n\n<pre><code>print \"[\" + \", \".join(l) + \"]\"\n</code></pre>\n\n<p>(you can probably also figure out what to do about dictionaries.</p>\n\n<p><strong>Summary</strong></p>\n\n<p>Implement <code>__repr__</code> for any class you implement. This should be second nature. Implement <code>__str__</code> if you think it would be useful to have a string version which errs on the side of more readability in favor of more ambiguity.</p>\n", | |
| "source": "so", | |
| "questionId": 1436703 | |
| }, | |
| { | |
| "title": ""Least Astonishment" and the Mutable Default Argument", | |
| "body": "<p>Actually, this is not a design flaw, and it is not because of internals, or performance.<br>\nIt comes simply from the fact that functions in Python are first-class objects, and not only a piece of code.</p>\n\n<p>As soon as you get to think into this way, then it completely makes sense: a function is an object being evaluated on its definition; default parameters are kind of \"member data\" and therefore their state may change from one call to the other - exactly as in any other object.</p>\n\n<p>In any case, Effbot has a very nice explanation of the reasons for this behavior in <a href=\"http://effbot.org/zone/default-values.htm\" rel=\"noreferrer\">Default Parameter Values in Python</a>.<br>\nI found it very clear, and I really suggest reading it for a better knowledge of how function objects work.</p>\n", | |
| "source": "so", | |
| "questionId": 1132941 | |
| }, | |
| { | |
| "title": "How do I pass a variable by reference?", | |
| "body": "<p>Arguments are <a href=\"http://docs.python.org/3/faq/programming.html#how-do-i-write-a-function-with-output-parameters-call-by-reference\" rel=\"noreferrer\">passed by assignment</a>. The rationale behind this is twofold:</p>\n\n<ol>\n<li>the parameter passed in is actually a <em>reference</em> to an object (but the reference is passed by value)</li>\n<li>some data types are mutable, but others aren't</li>\n</ol>\n\n<p>So:</p>\n\n<ul>\n<li><p>If you pass a <em>mutable</em> object into a method, the method gets a reference to that same object and you can mutate it to your heart's delight, but if you rebind the reference in the method, the outer scope will know nothing about it, and after you're done, the outer reference will still point at the original object. </p></li>\n<li><p>If you pass an <em>immutable</em> object to a method, you still can't rebind the outer reference, and you can't even mutate the object.</p></li>\n</ul>\n\n<p>To make it even more clear, let's have some examples. </p>\n\n<h2>List - a mutable type</h2>\n\n<p><strong>Let's try to modify the list that was passed to a method:</strong></p>\n\n<pre><code>def try_to_change_list_contents(the_list):\n print('got', the_list)\n the_list.append('four')\n print('changed to', the_list)\n\nouter_list = ['one', 'two', 'three']\n\nprint('before, outer_list =', outer_list)\ntry_to_change_list_contents(outer_list)\nprint('after, outer_list =', outer_list)\n</code></pre>\n\n<p>Output:</p>\n\n<pre class=\"lang-none prettyprint-override\"><code>before, outer_list = ['one', 'two', 'three']\ngot ['one', 'two', 'three']\nchanged to ['one', 'two', 'three', 'four']\nafter, outer_list = ['one', 'two', 'three', 'four']\n</code></pre>\n\n<p>Since the parameter passed in is a reference to <code>outer_list</code>, not a copy of it, we can use the mutating list methods to change it and have the changes reflected in the outer scope.</p>\n\n<p><strong>Now let's see what happens when we try to change the reference that was passed in as a parameter:</strong></p>\n\n<pre><code>def try_to_change_list_reference(the_list):\n print('got', the_list)\n the_list = ['and', 'we', 'can', 'not', 'lie']\n print('set to', the_list)\n\nouter_list = ['we', 'like', 'proper', 'English']\n\nprint('before, outer_list =', outer_list)\ntry_to_change_list_reference(outer_list)\nprint('after, outer_list =', outer_list)\n</code></pre>\n\n<p>Output:</p>\n\n<pre class=\"lang-none prettyprint-override\"><code>before, outer_list = ['we', 'like', 'proper', 'English']\ngot ['we', 'like', 'proper', 'English']\nset to ['and', 'we', 'can', 'not', 'lie']\nafter, outer_list = ['we', 'like', 'proper', 'English']\n</code></pre>\n\n<p>Since the <code>the_list</code> parameter was passed by value, assigning a new list to it had no effect that the code outside the method could see. The <code>the_list</code> was a copy of the <code>outer_list</code> reference, and we had <code>the_list</code> point to a new list, but there was no way to change where <code>outer_list</code> pointed.</p>\n\n<h2>String - an immutable type</h2>\n\n<p><strong>It's immutable, so there's nothing we can do to change the contents of the string</strong></p>\n\n<p><strong>Now, let's try to change the reference</strong></p>\n\n<pre><code>def try_to_change_string_reference(the_string):\n print('got', the_string)\n the_string = 'In a kingdom by the sea'\n print('set to', the_string)\n\nouter_string = 'It was many and many a year ago'\n\nprint('before, outer_string =', outer_string)\ntry_to_change_string_reference(outer_string)\nprint('after, outer_string =', outer_string)\n</code></pre>\n\n<p>Output:</p>\n\n<pre class=\"lang-none prettyprint-override\"><code>before, outer_string = It was many and many a year ago\ngot It was many and many a year ago\nset to In a kingdom by the sea\nafter, outer_string = It was many and many a year ago\n</code></pre>\n\n<p>Again, since the <code>the_string</code> parameter was passed by value, assigning a new string to it had no effect that the code outside the method could see. The <code>the_string</code> was a copy of the <code>outer_string</code> reference, and we had <code>the_string</code> point to a new string, but there was no way to change where <code>outer_string</code> pointed.</p>\n\n<p>I hope this clears things up a little.</p>\n\n<p><strong>EDIT:</strong> It's been noted that this doesn't answer the question that @David originally asked, \"Is there something I can do to pass the variable by actual reference?\". Let's work on that.</p>\n\n<h2>How do we get around this?</h2>\n\n<p>As @Andrea's answer shows, you could return the new value. This doesn't change the way things are passed in, but does let you get the information you want back out:</p>\n\n<pre><code>def return_a_whole_new_string(the_string):\n new_string = something_to_do_with_the_old_string(the_string)\n return new_string\n\n# then you could call it like\nmy_string = return_a_whole_new_string(my_string)\n</code></pre>\n\n<p>If you really wanted to avoid using a return value, you could create a class to hold your value and pass it into the function or use an existing class, like a list:</p>\n\n<pre><code>def use_a_wrapper_to_simulate_pass_by_reference(stuff_to_change):\n new_string = something_to_do_with_the_old_string(stuff_to_change[0])\n stuff_to_change[0] = new_string\n\n# then you could call it like\nwrapper = [my_string]\nuse_a_wrapper_to_simulate_pass_by_reference(wrapper)\n\ndo_something_with(wrapper[0])\n</code></pre>\n\n<p>Although this seems a little cumbersome.</p>\n", | |
| "source": "so", | |
| "questionId": 986006 | |
| }, | |
| { | |
| "title": "Iterating over dictionaries using 'for' loops", | |
| "body": "<p><code>key</code> is just a variable name. </p>\n\n<pre><code>for key in d:\n</code></pre>\n\n<p>will simply loop over the keys in the dictionary, rather than the keys and values. To loop over both key and value you can use the following:</p>\n\n<p>For Python 2.x:</p>\n\n<pre><code>for key, value in d.iteritems():\n</code></pre>\n\n<p>For Python 3.x:</p>\n\n<pre><code>for key, value in d.items():\n</code></pre>\n\n<p>To test for yourself, change the word <code>key</code> to <code>poop</code>.</p>\n\n<p>For Python 3.x, <code>iteritems()</code> has been replaced with simply <code>items()</code>, which returns a set-like view backed by the dict, like <code>iteritems()</code> but even better. \nThis is also available in 2.7 as <code>viewitems()</code>. </p>\n\n<p>The operation <code>items()</code> will work for both 2 and 3, but in 2 it will return a list of the dictionary's <code>(key, value)</code> pairs, which will not reflect changes to the dict that happen after the <code>items()</code> call. If you want the 2.x behavior in 3.x, you can call <code>list(d.items())</code>.</p>\n", | |
| "source": "so", | |
| "questionId": 3294889 | |
| }, | |
| { | |
| "title": "Making a flat list out of list of lists in Python", | |
| "body": "<pre><code>flat_list = [item for sublist in l for item in sublist]\n</code></pre>\n\n<p>which means:</p>\n\n<pre><code>for sublist in l:\n for item in sublist:\n flat_list.append(item)\n</code></pre>\n\n<p>is faster than the shortcuts posted so far. (<code>l</code> is the list to flatten.)</p>\n\n<p>Here is a the corresponding function:</p>\n\n<pre><code>flatten = lambda l: [item for sublist in l for item in sublist]\n</code></pre>\n\n<p>For evidence, as always, you can use the <code>timeit</code> module in the standard library:</p>\n\n<pre><code>$ python -mtimeit -s'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' '[item for sublist in l for item in sublist]'\n10000 loops, best of 3: 143 usec per loop\n$ python -mtimeit -s'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' 'sum(l, [])'\n1000 loops, best of 3: 969 usec per loop\n$ python -mtimeit -s'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' 'reduce(lambda x,y: x+y,l)'\n1000 loops, best of 3: 1.1 msec per loop\n</code></pre>\n\n<p>Explanation: the shortcuts based on <code>+</code> (including the implied use in <code>sum</code>) are, of necessity, <code>O(L**2)</code> when there are L sublists -- as the intermediate result list keeps getting longer, at each step a new intermediate result list object gets allocated, and all the items in the previous intermediate result must be copied over (as well as a few new ones added at the end). So (for simplicity and without actual loss of generality) say you have L sublists of I items each: the first I items are copied back and forth L-1 times, the second I items L-2 times, and so on; total number of copies is I times the sum of x for x from 1 to L excluded, i.e., <code>I * (L**2)/2</code>.</p>\n\n<p>The list comprehension just generates one list, once, and copies each item over (from its original place of residence to the result list) also exactly once.</p>\n", | |
| "source": "so", | |
| "questionId": 952914 | |
| }, | |
| { | |
| "title": "Understanding Python super() with __init__() methods", | |
| "body": "<p><code>super()</code> lets you avoid referring to the base class explicitly, which can be nice. But the main advantage comes with multiple inheritance, where all sorts of <a href=\"http://www.artima.com/weblogs/viewpost.jsp?thread=236275\" rel=\"noreferrer\">fun stuff</a> can happen. See the <a href=\"https://docs.python.org/2/library/functions.html#super\" rel=\"noreferrer\">standard docs on super</a> if you haven't already.</p>\n\n<p>Note that <a href=\"https://docs.python.org/3.0/library/functions.html#super\" rel=\"noreferrer\">the syntax changed in Python 3.0</a>: you can just say <code>super().__init__()</code> instead of <code>super(ChildB, self).__init__()</code> which IMO is quite a bit nicer.</p>\n", | |
| "source": "so", | |
| "questionId": 576169 | |
| }, | |
| { | |
| "title": "How can I make a time delay in Python?", | |
| "body": "<pre><code>import time\ntime.sleep(5) # delays for 5 seconds. You can Also Use Float Value.\n</code></pre>\n\n<p>Here is another example where something is run approximately once a minute:</p>\n\n<pre><code>import time \nwhile True:\n print(\"This prints once a minute.\")\n time.sleep(60) # Delay for 1 minute (60 seconds).\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 510348 | |
| }, | |
| { | |
| "title": "Catch multiple exceptions in one line (except block)", | |
| "body": "<p>From <a href=\"https://docs.python.org/3/tutorial/errors.html#handling-exceptions\" rel=\"noreferrer\">Python Documentation</a>:</p>\n\n<blockquote>\n <p>An except clause may name multiple exceptions as a parenthesized tuple, for example</p>\n</blockquote>\n\n<pre><code>except (IDontLikeYouException, YouAreBeingMeanException) as e:\n pass\n</code></pre>\n\n<p>Or, for Python 2 only:</p>\n\n<pre><code>except (IDontLikeYouException, YouAreBeingMeanException), e:\n pass\n</code></pre>\n\n<p>Separating the exception from the variable with a comma will still work in Python 2.6 and 2.7, but is now deprecated and does not work in Python 3; now you should be using <code>as</code>.</p>\n", | |
| "source": "so", | |
| "questionId": 6470428 | |
| }, | |
| { | |
| "title": "How to get current time in Python?", | |
| "body": "<pre><code>>>> import datetime\n>>> datetime.datetime.now()\ndatetime(2009, 1, 6, 15, 8, 24, 78915)\n</code></pre>\n\n<p>And just the time:</p>\n\n<pre><code>>>> datetime.datetime.time(datetime.datetime.now())\ndatetime.time(15, 8, 24, 78915)\n</code></pre>\n\n<p>The same but slightly more compact:</p>\n\n<pre><code>>>> datetime.datetime.now().time()\n</code></pre>\n\n<p>See the <a href=\"https://docs.python.org/3/library/datetime.html\" rel=\"noreferrer\"><strong>documentation</strong></a> for more info.</p>\n\n<p>To save typing, you can import the <code>datetime</code> object from the <code>datetime</code> module:</p>\n\n<pre><code>>>> from datetime import datetime\n</code></pre>\n\n<p>Then remove the leading <code>datetime.</code> from all the above.</p>\n", | |
| "source": "so", | |
| "questionId": 415511 | |
| }, | |
| { | |
| "title": "Is there a way to run Python on Android?", | |
| "source": "so", | |
| "questionId": 101754 | |
| }, | |
| { | |
| "title": "Add new keys to a dictionary?", | |
| "body": "<pre><code>>>> d = {'key':'value'}\n>>> print(d)\n{'key': 'value'}\n>>> d['mynewkey'] = 'mynewvalue'\n>>> print(d)\n{'mynewkey': 'mynewvalue', 'key': 'value'}\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1024847 | |
| }, | |
| { | |
| "title": "How do I parse a string to a float or int in Python?", | |
| "body": "<pre><code>>>> a = \"545.2222\"\n>>> float(a)\n545.22220000000004\n>>> int(float(a))\n545\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 379906 | |
| }, | |
| { | |
| "title": "How do I install pip on macOS or OS X?", | |
| "body": "<p>All you need to do is</p>\n\n<pre><code>sudo easy_install pip\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 17271319 | |
| }, | |
| { | |
| "title": "In Python, how do I read a file line-by-line into a list?", | |
| "body": "<pre><code>with open(fname) as f:\n content = f.readlines()\n# you may also want to remove whitespace characters like `\\n` at the end of each line\ncontent = [x.strip() for x in content] \n</code></pre>\n\n<p>I'm guessing that you meant <a href=\"http://docs.python.org/glossary.html#term-list\" rel=\"noreferrer\"><code>list</code></a> and not array.</p>\n", | |
| "source": "so", | |
| "questionId": 3277503 | |
| }, | |
| { | |
| "title": "How to clone or copy a list?", | |
| "body": "<p>With <code>new_list = my_list</code>, you don't actually have two lists. The assignment just copies the reference to the list, not the actual list, so both <code>new_list</code> and <code>my_list</code> refer to the same list after the assignment.</p>\n\n<p>To actually copy the list, you have various possibilities:</p>\n\n<ul>\n<li><p>You can slice it: </p>\n\n<pre><code>new_list = old_list[:]\n</code></pre>\n\n<p><a href=\"https://en.wikipedia.org/wiki/Alex_Martelli\" rel=\"noreferrer\" title=\"Alex Martelli\">Alex Martelli's</a> opinion (at least <a href=\"https://www.youtube.com/watch?v=g7V89K8QfgQ\" rel=\"noreferrer\" title=\"Bay Area Python Interest Group August 2007 Meeting\">back in 2007</a>) about this is, that <em>it is a weird syntax and it does not make sense to use it ever</em>. ;) (In his opinion, the next one is more readable).</p></li>\n<li><p>You can use the built in <a href=\"https://docs.python.org/2/library/functions.html#list\" rel=\"noreferrer\" title=\"list\"><code>list()</code></a> function:</p>\n\n<pre><code>new_list = list(old_list)\n</code></pre></li>\n<li><p>You can use generic <a href=\"https://docs.python.org/2/library/copy.html#copy.copy\" rel=\"noreferrer\" title=\"copy.copy\"><code>copy.copy()</code></a>:</p>\n\n<pre><code>import copy\nnew_list = copy.copy(old_list)\n</code></pre>\n\n<p>This is a little slower than <code>list()</code> because it has to find out the datatype of <code>old_list</code> first.</p></li>\n<li><p>If the list contains objects and you want to copy them as well, use generic <a href=\"https://docs.python.org/2/library/copy.html#copy.deepcopy\" rel=\"noreferrer\" title=\"copy.deepcopy\"><code>copy.deepcopy()</code></a>:</p>\n\n<pre><code>import copy\nnew_list = copy.deepcopy(old_list)\n</code></pre>\n\n<p>Obviously the slowest and most memory-needing method, but sometimes unavoidable.</p></li>\n</ul>\n\n<p><strong>Example:</strong></p>\n\n<pre><code>import copy\n\nclass Foo(object):\n def __init__(self, val):\n self.val = val\n\n def __repr__(self):\n return str(self.val)\n\nfoo = Foo(1)\n\na = ['foo', foo]\nb = a[:]\nc = list(a)\nd = copy.copy(a)\ne = copy.deepcopy(a)\n\n# edit orignal list and instance \na.append('baz')\nfoo.val = 5\n\nprint('original: %r\\n slice: %r\\n list(): %r\\n copy: %r\\n deepcopy: %r'\n % (a, b, c, d, e))\n</code></pre>\n\n<p>Result:</p>\n\n<pre><code>original: ['foo', 5, 'baz']\nslice: ['foo', 5]\nlist(): ['foo', 5]\ncopy: ['foo', 5]\ndeepcopy: ['foo', 1]\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 2612802 | |
| }, | |
| { | |
| "title": "How to concatenate two lists in Python?", | |
| "body": "<p>You can use the <code>+</code> operator to combine them:</p>\n\n<pre><code>listone = [1,2,3]\nlisttwo = [4,5,6]\n\nmergedlist = listone + listtwo\n</code></pre>\n\n<p>Output:</p>\n\n<pre><code>>>> mergedlist\n[1,2,3,4,5,6]\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1720421 | |
| }, | |
| { | |
| "title": "Is there a way to substring a string in Python?", | |
| "body": "<pre><code>>>> x = \"Hello World!\"\n>>> x[2:]\n'llo World!'\n>>> x[:2]\n'He'\n>>> x[:-2]\n'Hello Worl'\n>>> x[-2:]\n'd!'\n>>> x[2:-2]\n'llo Worl'\n</code></pre>\n\n<p>Python calls this concept \"slicing\" and it works on more than just strings. Take a look <a href=\"https://stackoverflow.com/questions/509211/good-primer-for-python-slice-notation\">here</a> for a comprehensive introduction.</p>\n", | |
| "source": "so", | |
| "questionId": 663171 | |
| }, | |
| { | |
| "title": "How do you split a list into evenly sized chunks?", | |
| "body": "<p>Here's a generator that yields the chunks you want:</p>\n\n<pre><code>def chunks(l, n):\n \"\"\"Yield successive n-sized chunks from l.\"\"\"\n for i in range(0, len(l), n):\n yield l[i:i + n]\n</code></pre>\n\n<hr>\n\n<pre><code>import pprint\npprint.pprint(list(chunks(range(10, 75), 10)))\n[[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],\n [20, 21, 22, 23, 24, 25, 26, 27, 28, 29],\n [30, 31, 32, 33, 34, 35, 36, 37, 38, 39],\n [40, 41, 42, 43, 44, 45, 46, 47, 48, 49],\n [50, 51, 52, 53, 54, 55, 56, 57, 58, 59],\n [60, 61, 62, 63, 64, 65, 66, 67, 68, 69],\n [70, 71, 72, 73, 74]]\n</code></pre>\n\n<hr>\n\n<p>If you're using Python 2, you should use <code>xrange()</code> instead of <code>range()</code>:</p>\n\n<pre><code>def chunks(l, n):\n \"\"\"Yield successive n-sized chunks from l.\"\"\"\n for i in xrange(0, len(l), n):\n yield l[i:i + n]\n</code></pre>\n\n<hr>\n\n<p>Also you can simply use list comprehension instead of writing a function. Python 3:</p>\n\n<pre><code>[l[i:i + n] for i in range(0, len(l), n)]\n</code></pre>\n\n<p>Python 2 version:</p>\n\n<pre><code>[l[i:i + n] for i in xrange(0, len(l), n)]\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 312443 | |
| }, | |
| { | |
| "title": "What does ** (double star/asterisk) and * (star/asterisk) do for parameters?", | |
| "body": "<p>The <code>*args</code> and <code>**kwargs</code> is a common idiom to allow arbitrary number of arguments to functions as described in the section <a href=\"http://docs.python.org/dev/tutorial/controlflow.html#more-on-defining-functions\" rel=\"noreferrer\">more on defining functions</a> in the Python documentation.</p>\n\n<p>The <code>*args</code> will give you all function parameters <a href=\"https://docs.python.org/dev/tutorial/controlflow.html#arbitrary-argument-lists\" rel=\"noreferrer\">as a tuple</a>:</p>\n\n<pre><code>In [1]: def foo(*args):\n ...: for a in args:\n ...: print a\n ...: \n ...: \n\nIn [2]: foo(1)\n1\n\n\nIn [4]: foo(1,2,3)\n1\n2\n3\n</code></pre>\n\n<p>The <code>**kwargs</code> will give you all \n<strong>keyword arguments</strong> except for those corresponding to a formal parameter as a dictionary.</p>\n\n<pre><code>In [5]: def bar(**kwargs):\n ...: for a in kwargs:\n ...: print a, kwargs[a]\n ...: \n ...: \n\nIn [6]: bar(name='one', age=27)\nage 27\nname one\n</code></pre>\n\n<p>Both idioms can be mixed with normal arguments to allow a set of fixed and some variable arguments:</p>\n\n<pre><code>def foo(kind, *args, **kwargs):\n pass\n</code></pre>\n\n<p>Another usage of the <code>*l</code> idiom is to <strong>unpack argument lists</strong> when calling a function.</p>\n\n<pre><code>In [9]: def foo(bar, lee):\n ...: print bar, lee\n ...: \n ...: \n\nIn [10]: l = [1,2]\n\nIn [11]: foo(*l)\n1 2\n</code></pre>\n\n<p>In Python 3 it is possible to use <code>*l</code> on the left side of an assignment (<a href=\"http://www.python.org/dev/peps/pep-3132/\" rel=\"noreferrer\">Extended Iterable Unpacking</a>), though it gives a list instead of a tuple in this context:</p>\n\n<pre><code>first, *rest = [1,2,3,4]\nfirst, *l, last = [1,2,3,4]\n</code></pre>\n\n<p>Also Python 3 adds new semantic (refer <a href=\"https://www.python.org/dev/peps/pep-3102/\" rel=\"noreferrer\">PEP 3102</a>):</p>\n\n<pre><code>def func(arg1, arg2, arg3, *, kwarg1, kwarg2):\n pass\n</code></pre>\n\n<p>Such function accepts only 3 positional arguments, and everything after <code>*</code> can only be passed as keyword arguments.</p>\n", | |
| "source": "so", | |
| "questionId": 36901 | |
| }, | |
| { | |
| "title": "Print in terminal with colors?", | |
| "body": "<p>This somewhat depends on what platform you are on. The most common way to do this is by printing ANSI escape sequences. For a simple example, here's some python code from the <a href=\"https://svn.blender.org/svnroot/bf-blender/trunk/blender/build_files/scons/tools/bcolors.py\" rel=\"noreferrer\">blender build scripts</a>:</p>\n\n<pre><code>class bcolors:\n HEADER = '\\033[95m'\n OKBLUE = '\\033[94m'\n OKGREEN = '\\033[92m'\n WARNING = '\\033[93m'\n FAIL = '\\033[91m'\n ENDC = '\\033[0m'\n BOLD = '\\033[1m'\n UNDERLINE = '\\033[4m'\n</code></pre>\n\n<p>To use code like this, you can do something like </p>\n\n<pre><code>print bcolors.WARNING + \"Warning: No active frommets remain. Continue?\" \n + bcolors.ENDC\n</code></pre>\n\n<p>This will work on unixes including OS X, linux and windows (provided you use <a href=\"https://github.com/adoxa/ansicon\" rel=\"noreferrer\">ANSICON</a>, or in Windows 10 provided you enable <a href=\"https://msdn.microsoft.com/en-us/library/mt638032\" rel=\"noreferrer\">VT100 emulation</a>). There are ansi codes for setting the color, moving the cursor, and more.</p>\n\n<p>If you are going to get complicated with this (and it sounds like you are if you are writing a game), you should look into the \"curses\" module, which handles a lot of the complicated parts of this for you. The <a href=\"http://docs.python.org/howto/curses.html\" rel=\"noreferrer\" title=\"Python Curses howto\">Python Curses HowTO</a> is a good introduction.</p>\n\n<p>If you are not using extended ASCII (i.e. not on a PC), you are stuck with the ascii characters below 127, and '#' or '@' is probably your best bet for a block. If you can ensure your terminal is using a IBM <a href=\"http://telecom.tbi.net/asc-ibm.html\" rel=\"noreferrer\">extended ascii character set</a>, you have many more options. Characters 176, 177, 178 and 219 are the \"block characters\".</p>\n\n<p>Some modern text-based programs, such as \"Dwarf Fortress\", emulate text mode in a graphical mode, and use images of the classic PC font. You can find some of these bitmaps that you can use on the <a href=\"http://dwarffortresswiki.org/DF2014:Tilesets\" rel=\"noreferrer\">Dwarf Fortress Wiki</a> see (<a href=\"http://dwarffortresswiki.org/Tileset_repository\" rel=\"noreferrer\">user-made tilesets</a>).</p>\n\n<p>The <a href=\"http://en.wikipedia.org/wiki/TMDC\" rel=\"noreferrer\" title=\"text mode demo contest\">Text Mode Demo Contest</a> has more resources for doing graphics in text mode.</p>\n\n<p>Hmm.. I think got a little carried away on this answer. I am in the midst of planning an epic text-based adventure game, though. Good luck with your colored text!</p>\n", | |
| "source": "so", | |
| "questionId": 287871 | |
| }, | |
| { | |
| "title": "Are static class variables possible?", | |
| "body": "<p>Variables declared inside the class definition, but not inside a method are class or static variables:</p>\n\n<pre><code>>>> class MyClass:\n... i = 3\n...\n>>> MyClass.i\n3 \n</code></pre>\n\n<p>As @<a href=\"https://stackoverflow.com/questions/68645/static-class-variables-in-python#answer-69067\">millerdev</a> points out, this creates a class-level <code>i</code> variable, but this is distinct from any instance-level <code>i</code> variable, so you could have</p>\n\n<pre><code>>>> m = MyClass()\n>>> m.i = 4\n>>> MyClass.i, m.i\n>>> (3, 4)\n</code></pre>\n\n<p>This is different from C++ and Java, but not so different from C#, where a static member can't be accessed using a reference to an instance.</p>\n\n<p>See <a href=\"https://docs.python.org/2/tutorial/classes.html#class-objects\" rel=\"noreferrer\">what the Python tutorial has to say on the subject of classes and class objects</a>.</p>\n\n<p>@Steve Johnson has already answered regarding <a href=\"http://web.archive.org/web/20090214211613/http://pyref.infogami.com/staticmethod\" rel=\"noreferrer\">static methods</a>, also documented under <a href=\"https://docs.python.org/2/library/functions.html#staticmethod\" rel=\"noreferrer\">\"Built-in Functions\" in the Python Library Reference</a>.</p>\n\n<pre><code>class C:\n @staticmethod\n def f(arg1, arg2, ...): ...\n</code></pre>\n\n<p>@beidy recommends <a href=\"https://docs.python.org/2/library/functions.html#classmethod\" rel=\"noreferrer\">classmethod</a>s over staticmethod, as the method then receives the class type as the first argument, but I'm still a little fuzzy on the advantages of this approach over staticmethod. If you are too, then it probably doesn't matter.</p>\n", | |
| "source": "so", | |
| "questionId": 68645 | |
| }, | |
| { | |
| "title": "How to get the number of elements in a list in Python?", | |
| "body": "<p>The <a href=\"https://docs.python.org/2/library/functions.html#len\" rel=\"noreferrer\" title=\"len()\"><code>len()</code></a> function can be used with a lot of types in <a href=\"http://en.wikipedia.org/wiki/Python_%28programming_language%29\" rel=\"noreferrer\">Python</a> - both built-in types and library types.</p>\n\n<pre><code>>>> len([1,2,3])\n3\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1712227 | |
| }, | |
| { | |
| "title": "Hidden features of Python", | |
| "body": "<h2>Chaining comparison operators:</h2>\n\n<pre><code>>>> x = 5\n>>> 1 < x < 10\nTrue\n>>> 10 < x < 20 \nFalse\n>>> x < 10 < x*10 < 100\nTrue\n>>> 10 > x <= 9\nTrue\n>>> 5 == x > 4\nTrue\n</code></pre>\n\n<p>In case you're thinking it's doing <code>1 < x</code>, which comes out as <code>True</code>, and then comparing <code>True < 10</code>, which is also <code>True</code>, then no, that's really not what happens (see the last example.) It's really translating into <code>1 < x and x < 10</code>, and <code>x < 10 and 10 < x * 10 and x*10 < 100</code>, but with less typing and each term is only evaluated once.</p>\n", | |
| "source": "so", | |
| "questionId": 101268 | |
| }, | |
| { | |
| "title": "Manually raising (throwing) an exception in Python", | |
| "body": "<blockquote>\n <h2>How do I manually throw/raise an exception in Python?</h2>\n</blockquote>\n\n<p><a href=\"https://docs.python.org/3/library/exceptions.html#exception-hierarchy\" rel=\"noreferrer\">Use the most specific Exception constructor that semantically fits your issue</a>. </p>\n\n<p>Be specific in your message, e.g.:</p>\n\n<pre><code>raise ValueError('A very specific bad thing happened.')\n</code></pre>\n\n<h2>Don't raise generic exceptions</h2>\n\n<p>Avoid raising a generic Exception. To catch it, you'll have to catch all other more specific exceptions that subclass it.</p>\n\n<h3>Problem 1: Hiding bugs</h3>\n\n<pre><code>raise Exception('I know Python!') # Don't! If you catch, likely to hide bugs.\n</code></pre>\n\n<p>For example:</p>\n\n<pre><code>def demo_bad_catch():\n try:\n raise ValueError('Represents a hidden bug, do not catch this')\n raise Exception('This is the exception you expect to handle')\n except Exception as error:\n print('Caught this error: ' + repr(error))\n\n>>> demo_bad_catch()\nCaught this error: ValueError('Represents a hidden bug, do not catch this',)\n</code></pre>\n\n<h3>Problem 2: Won't catch</h3>\n\n<p>and more specific catches won't catch the general exception:</p>\n\n<pre><code>def demo_no_catch():\n try:\n raise Exception('general exceptions not caught by specific handling')\n except ValueError as e:\n print('we will not catch exception: Exception')\n\n\n>>> demo_no_catch()\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\n File \"<stdin>\", line 3, in demo_no_catch\nException: general exceptions not caught by specific handling\n</code></pre>\n\n<h2>Best Practices: <code>raise</code> statement</h2>\n\n<p><a href=\"https://docs.python.org/3/library/exceptions.html#exception-hierarchy\" rel=\"noreferrer\">Instead, use the most specific Exception constructor that semantically fits your issue</a>.</p>\n\n<pre><code>raise ValueError('A very specific bad thing happened')\n</code></pre>\n\n<p>which also handily allows an arbitrary number of arguments to be passed to the constructor:</p>\n\n<pre><code>raise ValueError('A very specific bad thing happened', 'foo', 'bar', 'baz') \n</code></pre>\n\n<p>These arguments are accessed by the <code>args</code> attribute on the Exception object. For example:</p>\n\n<pre><code>try:\n some_code_that_may_raise_our_value_error()\nexcept ValueError as err:\n print(err.args)\n</code></pre>\n\n<p>prints </p>\n\n<pre><code>('message', 'foo', 'bar', 'baz') \n</code></pre>\n\n<p>In Python 2.5, an actual <code>message</code> attribute was added to BaseException in favor of encouraging users to subclass Exceptions and stop using <code>args</code>, but <a href=\"http://www.python.org/dev/peps/pep-0352/#retracted-ideas\" rel=\"noreferrer\">the introduction of <code>message</code> and the original deprecation of args has been retracted</a>.</p>\n\n<h2>Best Practices: <code>except</code> clause</h2>\n\n<p>When inside an except clause, you might want to, for example, log that a specific type of error happened, and then re-raise. The best way to do this while preserving the stack trace is to use a bare raise statement. For example:</p>\n\n<pre><code>logger = logging.getLogger(__name__)\n\ntry:\n do_something_in_app_that_breaks_easily()\nexcept AppError as error:\n logger.error(error)\n raise # just this!\n # raise AppError # Don't do this, you'll lose the stack trace!\n</code></pre>\n\n<h3>Don't modify your errors... but if you insist.</h3>\n\n<p>You can preserve the stacktrace (and error value) with <code>sys.exc_info()</code>, but <strong>this is way more error prone</strong> and <strong>has compatibility problems between Python 2 and 3</strong>, prefer to use a bare <code>raise</code> to re-raise. </p>\n\n<p>To explain - the <code>sys.exc_info()</code> returns the type, value, and traceback. </p>\n\n<pre><code>type, value, traceback = sys.exc_info()\n</code></pre>\n\n<p>This is the syntax in Python 2 - note this is not compatible with Python 3:</p>\n\n<pre><code> raise AppError, error, sys.exc_info()[2] # avoid this.\n # Equivalently, as error *is* the second object:\n raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2]\n</code></pre>\n\n<p>If you want to, you can modify what happens with your new raise - e.g. setting new args for the instance:</p>\n\n<pre><code>def error():\n raise ValueError('oops!')\n\ndef catch_error_modify_message():\n try:\n error()\n except ValueError:\n error_type, error_instance, traceback = sys.exc_info()\n error_instance.args = (error_instance.args[0] + ' <modification>',)\n raise error_type, error_instance, traceback\n</code></pre>\n\n<p>And we have preserved the whole traceback while modifying the args. Note that this is <strong>not a best practice</strong> and it is <strong>invalid syntax</strong> in Python 3 (making keeping compatibility much harder to work around).</p>\n\n<pre><code>>>> catch_error_modify_message()\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\n File \"<stdin>\", line 3, in catch_error_modify_message\n File \"<stdin>\", line 2, in error\nValueError: oops! <modification>\n</code></pre>\n\n<p>In <a href=\"https://docs.python.org/3/reference/simple_stmts.html#the-raise-statement\" rel=\"noreferrer\">Python 3</a>:</p>\n\n<pre><code> raise error.with_traceback(sys.exc_info()[2])\n</code></pre>\n\n<p>Again: avoid manually manipulating tracebacks. It's <a href=\"https://docs.python.org/2/reference/simple_stmts.html#the-raise-statement\" rel=\"noreferrer\">less efficient</a> and more error prone. And if you're using threading and <code>sys.exc_info</code> you may even get the wrong traceback (especially if you're using exception handling for control flow - which I'd personally tend to avoid.)</p>\n\n<h3>Python 3, Exception chaining</h3>\n\n<p>In Python 3, you can chain Exceptions, which preserve tracebacks:</p>\n\n<pre><code> raise RuntimeError('specific message') from error\n</code></pre>\n\n<p>Be aware:</p>\n\n<ul>\n<li>this <em>does</em> allow changing the error type raised, and</li>\n<li>this is <em>not</em> compatible with Python 2.</li>\n</ul>\n\n<h3>Deprecated Methods:</h3>\n\n<p>These can easily hide and even get into production code. You want to raise an exception, and doing them will raise an exception, <strong>but not the one intended!</strong></p>\n\n<p><a href=\"http://www.python.org/dev/peps/pep-3109/\" rel=\"noreferrer\">Valid in Python 2, but not in Python 3</a> is the following:</p>\n\n<pre><code>raise ValueError, 'message' # Don't do this, it's deprecated!\n</code></pre>\n\n<p>Only <a href=\"https://docs.python.org/2/whatsnew/2.5.html#pep-352-exceptions-as-new-style-classes\" rel=\"noreferrer\">valid in much older versions of Python</a> (2.4 and lower), you may still see people raising strings:</p>\n\n<pre><code>raise 'message' # really really wrong. don't do this.\n</code></pre>\n\n<p>In all modern versions, this will actually raise a TypeError, because you're not raising a BaseException type. If you're not checking for the right exception and don't have a reviewer that's aware of the issue, it could get into production.</p>\n\n<h2>Example Usage</h2>\n\n<p>I raise Exceptions to warn consumers of my API if they're using it incorrectly:</p>\n\n<pre><code>def api_func(foo):\n '''foo should be either 'baz' or 'bar'. returns something very useful.'''\n if foo not in _ALLOWED_ARGS:\n raise ValueError('{foo} wrong, use \"baz\" or \"bar\"'.format(foo=repr(foo)))\n</code></pre>\n\n<h2>Create your own error types when apropos</h2>\n\n<blockquote>\n <p><strong>\"I want to make an error on purpose, so that it would go into the except\"</strong></p>\n</blockquote>\n\n<p>You can create your own error types, if you want to indicate something specific is wrong with your application, just subclass the appropriate point in the exception hierarchy:</p>\n\n<pre><code>class MyAppLookupError(LookupError):\n '''raise this when there's a lookup error for my app'''\n</code></pre>\n\n<p>and usage:</p>\n\n<pre><code>if important_key not in resource_dict and not ok_to_be_missing:\n raise MyAppLookupError('resource is missing, and that is not ok.')\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 2052390 | |
| }, | |
| { | |
| "title": "Find current directory and file's directory", | |
| "body": "<p>To get the full path to the directory a Python file is contained in, write this in that file:</p>\n\n<pre><code>import os \ndir_path = os.path.dirname(os.path.realpath(__file__))\n</code></pre>\n\n<p>(Note that the incantation above won't work if you've already used <code>os.chdir()</code> to change your current working directory, since the value of the <code>__file__</code> constant is relative to the current working directory and is not changed by an <code>os.chdir()</code> call.)</p>\n\n<hr>\n\n<p>To get the current working directory use </p>\n\n<pre><code>import os\ncwd = os.getcwd()\n</code></pre>\n\n<hr>\n\n<p>Documentation references for the modules, constants and functions used above:</p>\n\n<ul>\n<li>The <a href=\"https://docs.python.org/library/os.html\"><code>os</code></a> and <a href=\"https://docs.python.org/library/os.path.html#module-os.path\"><code>os.path</code></a> modules.</li>\n<li>The <a href=\"https://docs.python.org/reference/datamodel.html\"><code>__file__</code></a> constant</li>\n<li><a href=\"https://docs.python.org/library/os.path.html#os.path.realpath\"><code>os.path.realpath(path)</code></a> (returns <em>\"the canonical path of the specified filename, eliminating any symbolic links encountered in the path\"</em>)</li>\n<li><a href=\"https://docs.python.org/library/os.path.html#os.path.dirname\"><code>os.path.dirname(path)</code></a> (returns <em>\"the directory name of pathname <code>path</code>\"</em>)</li>\n<li><a href=\"https://docs.python.org/library/os.html#os.getcwd\"><code>os.getcwd()</code></a> (returns <em>\"a string representing the current working directory\"</em>)</li>\n<li><a href=\"https://docs.python.org/library/os.html#os.chdir\"><code>os.chdir(path)</code></a> (<em>\"change the current working directory to <code>path</code>\"</em>)</li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 5137497 | |
| }, | |
| { | |
| "title": "How to convert string to lowercase in Python", | |
| "body": "<pre><code>s = \"Kilometer\"\nprint(s.lower())\n</code></pre>\n\n<p>The official documentation is <a href=\"https://docs.python.org/3.4/library/stdtypes.html?highlight=str.lower#str.lower\" rel=\"noreferrer\"><code>str.lower()</code></a>.</p>\n", | |
| "source": "so", | |
| "questionId": 6797984 | |
| }, | |
| { | |
| "title": "Converting string into datetime", | |
| "body": "<pre><code>from datetime import datetime\n\ndatetime_object = datetime.strptime('Jun 1 2005 1:33PM', '%b %d %Y %I:%M%p')\n</code></pre>\n\n<p>The resulting <code>datetime</code> object is timezone-naive.</p>\n\n<p>Links:</p>\n\n<ul>\n<li><p>Python documentation for <code>strptime</code>: <a href=\"https://docs.python.org/2/library/datetime.html#datetime.datetime.strptime\" rel=\"noreferrer\" title=\"datetime.datetime.strptime\">Python 2</a>, <a href=\"https://docs.python.org/3/library/datetime.html#datetime.datetime.strptime\" rel=\"noreferrer\">Python 3</a></p></li>\n<li><p>Python documentation for <code>strftime</code> format mask: <a href=\"https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior\" rel=\"noreferrer\" title=\"strftime-and-strptime-behavior\">Python 2</a>, <a href=\"https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior\" rel=\"noreferrer\">Python 3</a></p></li>\n<li><p><a href=\"http://strftime.org/\" rel=\"noreferrer\">strftime.org</a> is also a really nice reference for strftime</p></li>\n</ul>\n\n<p>Notes:</p>\n\n<ul>\n<li><code>strptime</code> = \"string parse time\"</li>\n<li><code>strftime</code> = \"string format time\"</li>\n<li>Pronounce it out loud today & you won't have to search for it again in 6 months.</li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 466345 | |
| }, | |
| { | |
| "title": "Replacements for switch statement in Python?", | |
| "body": "<p>You could use a dictionary:</p>\n\n<pre><code>def f(x):\n return {\n 'a': 1,\n 'b': 2,\n }[x]\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 60208 | |
| }, | |
| { | |
| "title": "Upgrading all packages with pip", | |
| "body": "<p>There isn't a built-in flag yet, but you can use</p>\n\n<pre><code>pip freeze --local | grep -v '^\\-e' | cut -d = -f 1 | xargs -n1 pip install -U\n</code></pre>\n\n<p>Note: there are infinite potential variations for this. I'm trying to keep this answer short and simple, but please do suggest variations in the comments!</p>\n\n<p>Relevant edits:</p>\n\n<ul>\n<li>Added a <code>grep</code> to skip \"-e\" package definitions, as suggested by @jawache (Yes, you could replace <code>grep</code>+<code>cut</code> with <code>sed</code> or <code>awk</code> or <code>perl</code> or...).</li>\n<li><p>Newer versions of <code>pip</code> allow you to list outdated packages: </p>\n\n<pre><code>pip list --outdated --format=freeze\n</code></pre></li>\n<li><p>Added <code>-n1</code> to <code>xargs</code>, prevents stopping everything if updating one package fails (thanks <a href=\"https://stackoverflow.com/users/339505/andsens\">@andsens</a>)</p></li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 2720014 | |
| }, | |
| { | |
| "title": "How do I copy a file in python?", | |
| "body": "<p><a href=\"http://docs.python.org/2/library/shutil.html\" rel=\"noreferrer\"><code>shutil</code></a> has many methods you can use. One of which is:</p>\n\n<pre><code>from shutil import copyfile\n\ncopyfile(src, dst)\n</code></pre>\n\n<p>Copy the contents of the file named <code>src</code> to a file named <code>dst</code>. The destination location must be writable; otherwise, an <code>IOError</code> exception will be raised. If <code>dst</code> already exists, it will be replaced. Special files such as character or block devices and pipes cannot be copied with this function. <code>src</code> and <code>dst</code> are path names given as strings. </p>\n", | |
| "source": "so", | |
| "questionId": 123198 | |
| }, | |
| { | |
| "title": "Why is reading lines from stdin much slower in C++ than Python?", | |
| "body": "<p>By default, <code>cin</code> is synchronized with stdio, which causes it to avoid any input buffering. If you add this to the top of your main, you should see much better performance:</p>\n\n<pre><code>std::ios_base::sync_with_stdio(false);\n</code></pre>\n\n<p>Normally, when an input stream is buffered, instead of reading one character at a time, the stream will be read in larger chunks. This reduces the number of system calls, which are typically relatively expensive. However, since the <code>FILE*</code> based <code>stdio</code> and <code>iostreams</code> often have separate implementations and therefore separate buffers, this could lead to a problem if both were used together. For example:</p>\n\n<pre><code>int myvalue1;\ncin >> myvalue1;\nint myvalue2;\nscanf(\"%d\",&myvalue2);\n</code></pre>\n\n<p>If more input was read by <code>cin</code> than it actually needed, then the second integer value wouldn't be available for the <code>scanf</code> function, which has its own independent buffer. This would lead to unexpected results.</p>\n\n<p>To avoid this, by default, streams are synchronized with <code>stdio</code>. One common way to achieve this is to have <code>cin</code> read each character one at a time as needed using <code>stdio</code> functions. Unfortunately, this introduces a lot of overhead. For small amounts of input, this isn't a big problem, but when you are reading millions of lines, the performance penalty is significant.</p>\n\n<p>Fortunately, the library designers decided that you should also be able to disable this feature to get improved performance if you knew what you were doing, so they provided the <a href=\"http://en.cppreference.com/w/cpp/io/ios_base/sync_with_stdio\" rel=\"noreferrer\"><code>sync_with_stdio</code></a> method.</p>\n", | |
| "source": "so", | |
| "questionId": 9371238 | |
| }, | |
| { | |
| "title": "Static methods in Python?", | |
| "body": "<p>Yep, using the <a href=\"https://docs.python.org/2/library/functions.html#staticmethod\" rel=\"noreferrer\" title=\"staticmethod\">staticmethod</a> decorator</p>\n\n<pre><code>class MyClass(object):\n @staticmethod\n def the_static_method(x):\n print x\n\nMyClass.the_static_method(2) # outputs 2\n</code></pre>\n\n<p>Note that some code might use the old method of defining a static method, using <code>staticmethod</code> as a function rather than a decorator. This should only be used if you have to support ancient versions of Python (2.2 and 2.3)</p>\n\n<pre><code>class MyClass(object):\n def the_static_method(x):\n print x\n the_static_method = staticmethod(the_static_method)\n\nMyClass.the_static_method(2) # outputs 2\n</code></pre>\n\n<p>This is entirely identical to the first example (using <code>@staticmethod</code>), just not using the nice decorator syntax</p>\n\n<p>Finally, use <a href=\"https://docs.python.org/2/library/functions.html#staticmethod\" rel=\"noreferrer\" title=\"staticmethod\"><code>staticmethod()</code></a> sparingly! There are very few situations where static-methods are necessary in Python, and I've seen them used many times where a separate \"top-level\" function would have been clearer.</p>\n\n<hr>\n\n<p><a href=\"https://docs.python.org/2/library/functions.html#staticmethod\" rel=\"noreferrer\" title=\"staticmethod\">The following is verbatim from the documentation:</a>:</p>\n\n<blockquote>\n <p>A static method does not receive an implicit first argument. To declare a static method, use this idiom:</p>\n\n<pre><code>class C:\n @staticmethod\n def f(arg1, arg2, ...): ...\n</code></pre>\n \n <p>The @staticmethod form is a function <a href=\"https://docs.python.org/2/glossary.html#term-decorator\" rel=\"noreferrer\" title=\"term-decorator\"><em>decorator</em></a> – see the description of function definitions in <a href=\"https://docs.python.org/2/reference/compound_stmts.html#function\" rel=\"noreferrer\" title=\"Function definitions\"><em>Function definitions</em></a> for details.</p>\n \n <p>It can be called either on the class (such as <code>C.f()</code>) or on an instance (such as <code>C().f()</code>). The instance is ignored except for its class.</p>\n \n <p>Static methods in Python are similar to those found in Java or C++. For a more advanced concept, see <a href=\"https://docs.python.org/2/library/functions.html#classmethod\" rel=\"noreferrer\" title=\"classmethod\"><code>classmethod()</code></a>.</p>\n \n <p>For more information on static methods, consult the documentation on the standard type hierarchy in <a href=\"https://docs.python.org/2/reference/datamodel.html#types\" rel=\"noreferrer\" title=\"types\"><em>The standard type hierarchy</em></a>.</p>\n \n <p>New in version 2.2.</p>\n \n <p>Changed in version 2.4: Function decorator syntax added.</p>\n</blockquote>\n", | |
| "source": "so", | |
| "questionId": 735975 | |
| }, | |
| { | |
| "title": "How to delete a file or folder?", | |
| "body": "<p><a href=\"http://docs.python.org/library/os.html#os.remove\" rel=\"noreferrer\"><code>os.remove()</code></a> will remove a file.</p>\n\n<p><a href=\"http://docs.python.org/library/os.html#os.rmdir\" rel=\"noreferrer\"><code>os.rmdir()</code></a> will remove an empty directory.</p>\n\n<p><a href=\"http://docs.python.org/library/shutil.html#shutil.rmtree\" rel=\"noreferrer\"><code>shutil.rmtree()</code></a> will delete a directory and all its contents.</p>\n", | |
| "source": "so", | |
| "questionId": 6996603 | |
| }, | |
| { | |
| "title": "Python join: why is it string.join(list) instead of list.join(string)?", | |
| "body": "<p>It's because any iterable can be joined, not just lists, but the result and the \"joiner\" are always strings.</p>\n\n<p>E.G:</p>\n\n<pre><code>import urllib2\nprint '\\n############\\n'.join(\n urllib2.urlopen('http://data.stackexchange.com/users/7095'))\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 493819 | |
| }, | |
| { | |
| "title": "Determine the type of an object?", | |
| "body": "<p>To get the type of an object, you can use the built-in <a href=\"http://docs.python.org/3/library/functions.html#type\" rel=\"noreferrer\"><code>type()</code></a> function. Passing an object as the only parameter will return the type object of that object:</p>\n\n<pre><code>>>> type([]) is list\nTrue\n>>> type({}) is dict\nTrue\n>>> type('') is str\nTrue\n>>> type(0) is int\nTrue\n>>> type({})\n<type 'dict'>\n>>> type([])\n<type 'list'>\n</code></pre>\n\n<p>This of course also works for custom types:</p>\n\n<pre><code>>>> class Test1 (object):\n pass\n>>> class Test2 (Test1):\n pass\n>>> a = Test1()\n>>> b = Test2()\n>>> type(a) is Test1\nTrue\n>>> type(b) is Test2\nTrue\n</code></pre>\n\n<p>Note that <code>type()</code> will only return the immediate type of the object, but won’t be able to tell you about type inheritance.</p>\n\n<pre><code>>>> type(b) is Test1\nFalse\n</code></pre>\n\n<p>To cover that, you should use the <a href=\"http://docs.python.org/3/library/functions.html#isinstance\" rel=\"noreferrer\"><code>isinstance</code></a> function. This of course also works for built-in types:</p>\n\n<pre><code>>>> isinstance(b, Test1)\nTrue\n>>> isinstance(b, Test2)\nTrue\n>>> isinstance(a, Test1)\nTrue\n>>> isinstance(a, Test2)\nFalse\n>>> isinstance([], list)\nTrue\n>>> isinstance({}, dict)\nTrue\n</code></pre>\n\n<p><code>isinstance()</code> is usually the preferred way to ensure the type of an object because it will also accept derived types. So unless you actually need the type object (for whatever reason), using <code>isinstance()</code> is preferred over <code>type()</code>.</p>\n\n<p>The second parameter of <code>isinstance()</code> also accepts a tuple of types, so it’s possible to check for multiple types at once. <code>isinstance</code> will then return true, if the object is of any of those types:</p>\n\n<pre><code>>>> isinstance([], (tuple, list, set))\nTrue\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 2225038 | |
| }, | |
| { | |
| "title": "Getting the last element of a list in Python", | |
| "body": "<p><code>some_list[-1]</code> is the shortest and most Pythonic.</p>\n\n<p>In fact, you can do much more with this syntax. The <code>some_list[-n]</code> syntax gets the nth-to-last element. So <code>some_list[-1]</code> gets the last element, <code>some_list[-2]</code> gets the second to last, etc, all the way down to <code>some_list[-len(some_list)]</code>, which gives you the first element.</p>\n\n<p>You can also set list elements in this way. For instance:</p>\n\n<pre><code>>>> some_list = [1, 2, 3]\n>>> some_list[-1] = 5 # Set the last element\n>>> some_list[-2] = 3 # Set the second to last element\n>>> some_list\n[1, 3, 5]\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 930397 | |
| }, | |
| { | |
| "title": "What is __init__.py for?", | |
| "body": "<p>It's a part of a package. <a href=\"http://docs.python.org/tutorial/modules.html#packages\" rel=\"noreferrer\">Here's the documentation.</a></p>\n\n<blockquote>\n <p>The <code>__init__.py</code> files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as <code>string</code>, from unintentionally hiding valid modules that occur later (deeper) on the module search path. In the simplest case, <code>__init__.py</code> can just be an empty file, but it can also execute initialization code for the package or set the <code>__all__</code> variable, described later.</p>\n</blockquote>\n", | |
| "source": "so", | |
| "questionId": 448271 | |
| }, | |
| { | |
| "title": "How can I remove (chomp) a trailing newline in Python?", | |
| "body": "<p>Try the method <code>rstrip()</code> (see doc <a href=\"http://docs.python.org/2/library/stdtypes.html#str.rstrip\" rel=\"noreferrer\">Python 2</a> and <a href=\"https://docs.python.org/3/library/stdtypes.html#str.rstrip\" rel=\"noreferrer\">Python 3</a>)</p>\n\n<pre><code>>>> 'test string\\n'.rstrip()\n'test string'\n</code></pre>\n\n<p>Python's <code>rstrip()</code> method strips <em>all</em> kinds of trailing whitespace by default, not just one newline as Perl does with <a href=\"http://perldoc.perl.org/functions/chomp.html\" rel=\"noreferrer\"><code>chomp</code></a>.</p>\n\n<pre><code>>>> 'test string \\n \\r\\n\\n\\r \\n\\n'.rstrip()\n'test string'\n</code></pre>\n\n<p>To strip only newlines:</p>\n\n<pre><code>>>> 'test string \\n \\r\\n\\n\\r \\n\\n'.rstrip('\\n')\n'test string \\n \\r\\n\\n\\r '\n</code></pre>\n\n<p>There are also the methods <code>lstrip()</code> and <code>strip()</code>:</p>\n\n<pre><code>>>> s = \" \\n\\r\\n \\n abc def \\n\\r\\n \\n \"\n>>> s.strip()\n'abc def'\n>>> s.lstrip()\n'abc def \\n\\r\\n \\n '\n>>> s.rstrip()\n' \\n\\r\\n \\n abc def'\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 275018 | |
| }, | |
| { | |
| "title": "How to print without newline or space?", | |
| "body": "<h2>General way</h2>\n\n<pre><code>import sys\nsys.stdout.write('.')\n</code></pre>\n\n<p>You may also need to call</p>\n\n<pre><code>sys.stdout.flush()\n</code></pre>\n\n<p>to ensure <code>stdout</code> is flushed immediately.</p>\n\n<h2>Python 2.6+</h2>\n\n<p>From Python 2.6 you can import the <code>print</code> function from Python 3:</p>\n\n<pre><code>from __future__ import print_function\n</code></pre>\n\n<p>This allows you to use the Python 3 solution below.</p>\n\n<h2>Python 3</h2>\n\n<p>In Python 3, the <code>print</code> statement has been changed into a function. In Python 3, you can instead do:</p>\n\n<pre><code>print('.', end='')\n</code></pre>\n\n<p>This also works in Python 2, provided that you've used <code>from __future__ import print_function</code>.</p>\n\n<p>If you are having trouble with buffering, you can flush the output by adding <code>flush=True</code> keyword argument:</p>\n\n<pre><code>print('.', end='', flush=True)\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 493386 | |
| }, | |
| { | |
| "title": "Why is "1000000000000000 in range(1000000000000001)" so fast in Python 3?", | |
| "body": "<p>The Python 3 <code>range()</code> object doesn't produce numbers immediately; it is a smart sequence object that produces numbers <em>on demand</em>. All it contains is your start, stop and step values, then as you iterate over the object the next integer is calculated each iteration.</p>\n\n<p>The object also implements the <a href=\"https://docs.python.org/3/reference/datamodel.html#object.__contains__\"><code>object.__contains__</code> hook</a>, and <em>calculates</em> if your number is part of its range. Calculating is a O(1) constant time operation. There is never a need to scan through all possible integers in the range.</p>\n\n<p>From the <a href=\"https://docs.python.org/3/library/stdtypes.html#range\"><code>range()</code> object documentation</a>:</p>\n\n<blockquote>\n <p>The advantage of the <code>range</code> type over a regular <code>list</code> or <code>tuple</code> is that a range object will always take the same (small) amount of memory, no matter the size of the range it represents (as it only stores the <code>start</code>, <code>stop</code> and <code>step</code> values, calculating individual items and subranges as needed).</p>\n</blockquote>\n\n<p>So at a minimum, your <code>range()</code> object would do:</p>\n\n<pre><code>class my_range(object):\n def __init__(self, start, stop=None, step=1):\n if stop is None:\n start, stop = 0, start\n self.start, self.stop, self.step = start, stop, step\n if step < 0:\n lo, hi = stop, start\n else:\n lo, hi = start, stop\n self.length = ((hi - lo - 1) // abs(step)) + 1\n\n def __iter__(self):\n current = self.start\n if self.step < 0:\n while current > self.stop:\n yield current\n current += self.step\n else:\n while current < self.stop:\n yield current\n current += self.step\n\n def __len__(self):\n return self.length\n\n def __getitem__(self, i):\n if i < 0:\n i += self.length\n if 0 <= i < self.length:\n return self.start + i * self.step\n raise IndexError('Index out of range: {}'.format(i))\n\n def __contains__(self, num):\n if self.step < 0:\n if not (self.stop < num <= self.start):\n return False\n else:\n if not (self.start <= num < self.stop):\n return False\n return (num - self.start) % self.step == 0\n</code></pre>\n\n<p>This is still missing several things that a real <code>range()</code> supports (such as the <code>.index()</code> or <code>.count()</code> methods, hashing, equality testing, or slicing), but should give you an idea.</p>\n\n<p>I also simplified the <code>__contains__</code> implementation to only focus on integer tests; if you give a real <code>range()</code> object a non-integer value (including subclasses of <code>int</code>), a slow scan is initiated to see if there is a match, just as if you use a containment test against a list of all the contained values. This was done to continue to support other numeric types that just happen to support equality testing with integers but are not expected to support integer arithmetic as well. See the original <a href=\"http://bugs.python.org/issue1766304\">Python issue</a> that implemented the containment test.</p>\n", | |
| "source": "so", | |
| "questionId": 30081275 | |
| }, | |
| { | |
| "title": "How do I sort a list of dictionaries by values of the dictionary in Python?", | |
| "body": "<p>It may look cleaner using a key instead a cmp:</p>\n\n<pre class=\"lang-py prettyprint-override\"><code>newlist = sorted(list_to_be_sorted, key=lambda k: k['name']) \n</code></pre>\n\n<p>or as J.F.Sebastian and others suggested,</p>\n\n<pre class=\"lang-py prettyprint-override\"><code>from operator import itemgetter\nnewlist = sorted(list_to_be_sorted, key=itemgetter('name')) \n</code></pre>\n\n<p>For completeness (as pointed out in comments by fitzgeraldsteele), add <code>reverse=True</code> to sort descending</p>\n\n<pre class=\"lang-py prettyprint-override\"><code>newlist = sorted(l, key=itemgetter('name'), reverse=True)\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 72899 | |
| }, | |
| { | |
| "title": "How to randomly select an item from a list?", | |
| "body": "<p>Use <a href=\"https://docs.python.org/2/library/random.html#random.choice\" rel=\"noreferrer\"><code>random.choice</code></a>:</p>\n\n<pre><code>import random\n\nfoo = ['a', 'b', 'c', 'd', 'e']\nprint(random.choice(foo))\n</code></pre>\n\n<p>For cryptographically secure random choices (e.g. for generating a passphrase from a wordlist), use <a href=\"https://docs.python.org/2/library/random.html#random.SystemRandom\" rel=\"noreferrer\"><code>random.SystemRandom</code></a> class:</p>\n\n<pre><code>import random\n\nfoo = ['battery', 'correct', 'horse', 'staple']\nsecure_random = random.SystemRandom()\nprint(secure_random.choice(foo))\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 306400 | |
| }, | |
| { | |
| "title": "How do I check if a string is a number (float)?", | |
| "body": "<blockquote>\n <p>Which, not only is ugly and slow</p>\n</blockquote>\n\n<p>I'd dispute both.</p>\n\n<p>A regex or other string parsing would be uglier and slower. </p>\n\n<p>I'm not sure that anything much could be faster than the above. It calls the function and returns. Try/Catch doesn't introduce much overhead because the most common exception is caught without an extensive search of stack frames.</p>\n\n<p>The issue is that any numeric conversion function has two kinds of results</p>\n\n<ul>\n<li>A number, if the number is valid</li>\n<li>A status code (e.g., via errno) or exception to show that no valid number could be parsed.</li>\n</ul>\n\n<p>C (as an example) hacks around this a number of ways. Python lays it out clearly and explicitly.</p>\n\n<p>I think your code for doing this is perfect.</p>\n", | |
| "source": "so", | |
| "questionId": 354038 | |
| }, | |
| { | |
| "title": "Meaning of @classmethod and @staticmethod for beginner?", | |
| "body": "<p>Though <code>classmethod</code> and <code>staticmethod</code> are quite similar, there's a slight difference in usage for both entities: <code>classmethod</code> must have a reference to a class object as the first parameter, whereas <code>staticmethod</code> can have no parameters at all.</p>\n\n<h2>Example</h2>\n\n<pre><code>class Date(object):\n\n def __init__(self, day=0, month=0, year=0):\n self.day = day\n self.month = month\n self.year = year\n\n @classmethod\n def from_string(cls, date_as_string):\n day, month, year = map(int, date_as_string.split('-'))\n date1 = cls(day, month, year)\n return date1\n\n @staticmethod\n def is_date_valid(date_as_string):\n day, month, year = map(int, date_as_string.split('-'))\n return day <= 31 and month <= 12 and year <= 3999\n\ndate2 = Date.from_string('11-09-2012')\nis_date = Date.is_date_valid('11-09-2012')\n</code></pre>\n\n<h2>Explanation</h2>\n\n<p>Let's assume an example of a class, dealing with date information (this is what will be our boilerplate to cook on):</p>\n\n<pre><code>class Date(object):\n\n def __init__(self, day=0, month=0, year=0):\n self.day = day\n self.month = month\n self.year = year\n</code></pre>\n\n<p>This class obviously could be used to store information about certain dates (without timezone information; let's assume all dates are presented in UTC).</p>\n\n<p>Here we have <code>__init__</code>, a typical initializer of Python class instances, which receives arguments as a typical <code>instancemethod</code>, having the first non-optional argument (<code>self</code>) that holds reference to a newly created instance.</p>\n\n<p><strong>Class Method</strong></p>\n\n<p>We have some tasks that can be nicely done using <code>classmethod</code>s.</p>\n\n<p><em>Let's assume that we want to create a lot of <code>Date</code> class instances having date information coming from outer source encoded as a string of next format ('dd-mm-yyyy'). We have to do that in different places of our source code in project.</em></p>\n\n<p>So what we must do here is:</p>\n\n<ol>\n<li>Parse a string to receive day, month and year as three integer variables or a 3-item tuple consisting of that variable.</li>\n<li>Instantiate <code>Date</code> by passing those values to initialization call.</li>\n</ol>\n\n<p>This will look like:</p>\n\n<pre><code>day, month, year = map(int, string_date.split('-'))\ndate1 = Date(day, month, year)\n</code></pre>\n\n<p>For this purpose, C++ has such feature as overloading, but Python lacks that feature- so here's when <code>classmethod</code> applies. Lets create another \"<em>constructor</em>\".</p>\n\n<pre><code> @classmethod\n def from_string(cls, date_as_string):\n day, month, year = map(int, date_as_string.split('-'))\n date1 = cls(day, month, year)\n return date1\n\ndate2 = Date.from_string('11-09-2012')\n</code></pre>\n\n<p>Let's look more carefully at the above implementation, and review what advantages we have here:</p>\n\n<ol>\n<li>We've implemented date string parsing in one place and it's reusable now.</li>\n<li>Encapsulation works fine here (if you think that you could implement string parsing as a single function elsewhere, this solution fits OOP paradigm far better).</li>\n<li><code>cls</code> is an object that holds <strong>class itself</strong>, not an instance of the class. It's pretty cool because if we inherit our <code>Date</code> class, all children will have <code>from_string</code> defined also.</li>\n</ol>\n\n<p><strong>Static method</strong></p>\n\n<p>What about <code>staticmethod</code>? It's pretty similar to <code>classmethod</code> but doesn't take any obligatory parameters (like a class method or instance method does).</p>\n\n<p>Let's look at the next use case.</p>\n\n<p><em>We have a date string that we want to validate somehow. This task is also logically bound to <code>Date</code> class we've used so far, but still doesn't require instantiation of it.</em></p>\n\n<p>Here is where <code>staticmethod</code> can be useful. Let's look at the next piece of code:</p>\n\n<pre><code> @staticmethod\n def is_date_valid(date_as_string):\n day, month, year = map(int, date_as_string.split('-'))\n return day <= 31 and month <= 12 and year <= 3999\n\n # usage:\n is_date = Date.is_date_valid('11-09-2012')\n</code></pre>\n\n<p>So, as we can see from usage of <code>staticmethod</code>, we don't have any access to what the class is- it's basically just a function, called syntactically like a method, but without access to the object and it's internals (fields and another methods), while classmethod does.</p>\n", | |
| "source": "so", | |
| "questionId": 12179271 | |
| }, | |
| { | |
| "title": "How do I access environment variables from Python?", | |
| "body": "<p>Environment variables are accessed through <a href=\"https://docs.python.org/library/os.html#os.environ\" rel=\"noreferrer\">os.environ</a></p>\n\n<pre><code>import os\nprint(os.environ['HOME'])\n</code></pre>\n\n<p>Or you can see a list of all the environment variables using:</p>\n\n<pre><code>os.environ\n</code></pre>\n\n<p>As sometimes you might need to see a complete list!</p>\n\n<pre><code># using get will return `None` if a key is not present rather than raise a `KeyError`\nprint(os.environ.get('KEY_THAT_MIGHT_EXIST'))\n\n# os.getenv is equivalent, and can also give a default value instead of `None`\nprint(os.getenv('KEY_THAT_MIGHT_EXIST', default_value))\n</code></pre>\n\n<p><a href=\"https://docs.python.org/install/index.html#how-installation-works\" rel=\"noreferrer\">Python default installation</a> on Windows is <code>C:\\Python</code>. If you want to find out while running python you can do:</p>\n\n<pre><code>import sys\nprint(sys.prefix)\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 4906977 | |
| }, | |
| { | |
| "title": "How can I represent an 'Enum' in Python?", | |
| "body": "<p>Enums have been added to Python 3.4 as described in <a href=\"http://www.python.org/dev/peps/pep-0435/\" rel=\"noreferrer\">PEP 435</a>. It has also been <a href=\"https://pypi.python.org/pypi/enum34\" rel=\"noreferrer\">backported to 3.3, 3.2, 3.1, 2.7, 2.6, 2.5, and 2.4</a> on pypi. </p>\n\n<p>For more advanced Enum techniques try the <a href=\"https://pypi.python.org/pypi/aenum\" rel=\"noreferrer\">aenum library</a> (2.7, 3.3+, same author as <code>enum34</code>. Code is not perfectly compatible between py2 and py3, e.g. you'll need <a href=\"https://stackoverflow.com/a/25982264/57461\"><code>__order__</code> in python 2</a>).</p>\n\n<ul>\n<li>To use <code>enum34</code>, do <code>$ pip install enum34</code></li>\n<li>To use <code>aenum</code>, do <code>$ pip install aenum</code></li>\n</ul>\n\n<p>Installing <code>enum</code> (no numbers) will install a completely different and incompatible version.</p>\n\n<hr>\n\n<pre><code>from enum import Enum # for enum34, or the stdlib version\n# from aenum import Enum # for the aenum version\nAnimal = Enum('Animal', 'ant bee cat dog')\n\nAnimal.ant # returns <Animal.ant: 1>\nAnimal['ant'] # returns <Animal.ant: 1> (string lookup)\nAnimal.ant.name # returns 'ant' (inverse lookup)\n</code></pre>\n\n<p>or equivalently:</p>\n\n<pre><code>class Animal(Enum):\n ant = 1\n bee = 2\n cat = 3\n dog = 4\n</code></pre>\n\n<hr>\n\n<p>In earlier versions, one way of accomplishing enums is:</p>\n\n<pre><code>def enum(**enums):\n return type('Enum', (), enums)\n</code></pre>\n\n<p>which is used like so:</p>\n\n<pre><code>>>> Numbers = enum(ONE=1, TWO=2, THREE='three')\n>>> Numbers.ONE\n1\n>>> Numbers.TWO\n2\n>>> Numbers.THREE\n'three'\n</code></pre>\n\n<p>You can also easily support automatic enumeration with something like this:</p>\n\n<pre><code>def enum(*sequential, **named):\n enums = dict(zip(sequential, range(len(sequential))), **named)\n return type('Enum', (), enums)\n</code></pre>\n\n<p>and used like so:</p>\n\n<pre><code>>>> Numbers = enum('ZERO', 'ONE', 'TWO')\n>>> Numbers.ZERO\n0\n>>> Numbers.ONE\n1\n</code></pre>\n\n<p>Support for converting the values back to names can be added this way:</p>\n\n<pre><code>def enum(*sequential, **named):\n enums = dict(zip(sequential, range(len(sequential))), **named)\n reverse = dict((value, key) for key, value in enums.iteritems())\n enums['reverse_mapping'] = reverse\n return type('Enum', (), enums)\n</code></pre>\n\n<p>This overwrites anything with that name, but it is useful for rendering your enums in output. It will throw KeyError if the reverse mapping doesn't exist. With the first example:</p>\n\n<pre><code>>>> Numbers.reverse_mapping['three']\n'THREE'\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 36932 | |
| }, | |
| { | |
| "title": "*args and **kwargs?", | |
| "body": "<p><a href=\"http://docs.python.org/tutorial/controlflow.html#arbitrary-argument-lists\" rel=\"noreferrer\">The syntax is the <code>*</code> and <code>**</code></a>. The names <code>*args</code> and <code>**kwargs</code> are only by convention but there's no hard requirement to use them.</p>\n\n<p>You would use <code>*args</code> when you're not sure how many arguments might be passed to your function, i.e. it allows you pass an arbitrary number of arguments to your function. For example:</p>\n\n<pre><code>>>> def print_everything(*args):\n for count, thing in enumerate(args):\n... print( '{0}. {1}'.format(count, thing))\n...\n>>> print_everything('apple', 'banana', 'cabbage')\n0. apple\n1. banana\n2. cabbage\n</code></pre>\n\n<p>Similarly, <code>**kwargs</code> allows you to handle named arguments that you have not defined in advance:</p>\n\n<pre><code>>>> def table_things(**kwargs):\n... for name, value in kwargs.items():\n... print( '{0} = {1}'.format(name, value))\n...\n>>> table_things(apple = 'fruit', cabbage = 'vegetable')\ncabbage = vegetable\napple = fruit\n</code></pre>\n\n<p>You can use these along with named arguments too. The explicit arguments get values first and then everything else is passed to <code>*args</code> and <code>**kwargs</code>. The named arguments come first in the list. For example:</p>\n\n<pre><code>def table_things(titlestring, **kwargs)\n</code></pre>\n\n<p>You can also use both in the same function definition but <code>*args</code> must occur before <code>**kwargs</code>.</p>\n\n<p>You can also use the <code>*</code> and <code>**</code> syntax when calling a function. For example:</p>\n\n<pre><code>>>> def print_three_things(a, b, c):\n... print( 'a = {0}, b = {1}, c = {2}'.format(a,b,c))\n...\n>>> mylist = ['aardvark', 'baboon', 'cat']\n>>> print_three_things(*mylist)\na = aardvark, b = baboon, c = cat\n</code></pre>\n\n<p>As you can see in this case it takes the list (or tuple) of items and unpacks it. By this it matches them to the arguments in the function. Of course, you could have a <code>*</code> both in the function definition and in the function call.</p>\n", | |
| "source": "so", | |
| "questionId": 3394835 | |
| }, | |
| { | |
| "title": "How do you read from stdin in Python?", | |
| "body": "<p>You could use the <a href=\"http://docs.python.org/library/fileinput.html\" rel=\"noreferrer\"><code>fileinput</code></a> module:</p>\n\n<pre><code>import fileinput\n\nfor line in fileinput.input():\n pass\n</code></pre>\n\n<p><a href=\"http://docs.python.org/library/fileinput.html\" rel=\"noreferrer\"><code>fileinput</code></a> will loop through all the lines in the input specified as file names given in command-line arguments, or the standard input if no arguments are provided.</p>\n", | |
| "source": "so", | |
| "questionId": 1450393 | |
| }, | |
| { | |
| "title": "Python string formatting: % vs. .format", | |
| "body": "<p>To answer your first question... <code>.format</code> just seems more sophisticated in many ways. An annoying thing about <code>%</code> is also how it can either take a variable or a tuple. You'd think the following would always work:</p>\n\n<pre><code>\"hi there %s\" % name\n</code></pre>\n\n<p>yet, if <code>name</code> happens to be <code>(1, 2, 3)</code>, it will throw a <code>TypeError</code>. To guarantee that it always prints, you'd need to do</p>\n\n<pre><code>\"hi there %s\" % (name,) # supply the single argument as a single-item tuple\n</code></pre>\n\n<p>which is just ugly. <code>.format</code> doesn't have those issues. Also in the second example you gave, the <code>.format</code> example is much cleaner looking.</p>\n\n<p>Why would you not use it? </p>\n\n<ul>\n<li>not knowing about it (me before reading this)</li>\n<li>having to be compatible with Python 2.5</li>\n</ul>\n\n<hr>\n\n<p>To answer your second question, string formatting happens at the same time as any other operation - when the string formatting expression is evaluated. And Python, not being a lazy language, evaluates expressions before calling functions, so in your <code>log.debug</code> example, the expression <code>\"some debug info: %s\"%some_info</code>will first evaluate to, e.g. <code>\"some debug info: roflcopters are active\"</code>, then that string will be passed to <code>log.debug()</code>. </p>\n", | |
| "source": "so", | |
| "questionId": 5082452 | |
| }, | |
| { | |
| "title": "How do you append to a file?", | |
| "body": "<pre><code>with open(\"test.txt\", \"a\") as myfile:\n myfile.write(\"appended text\")\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 4706499 | |
| }, | |
| { | |
| "title": "Calling a function of a module by using its name (a string)", | |
| "body": "<p>Assuming module <code>foo</code> with method <code>bar</code>:</p>\n\n<pre><code>import foo\nmethod_to_call = getattr(foo, 'bar')\nresult = method_to_call()\n</code></pre>\n\n<p>As far as that goes, lines 2 and 3 can be compressed to:</p>\n\n<pre><code>result = getattr(foo, 'bar')()\n</code></pre>\n\n<p>if that makes more sense for your use case. You can use <code>getattr</code> in this fashion on class instance bound methods, module-level methods, class methods... the list goes on.</p>\n", | |
| "source": "so", | |
| "questionId": 3061 | |
| }, | |
| { | |
| "title": "Convert bytes to a string?", | |
| "body": "<p>You need to decode the bytes object to produce a string:</p>\n\n<pre><code>>>> b\"abcde\"\nb'abcde'\n\n# utf-8 is used here because it is a very common encoding, but you\n# need to use the encoding your data is actually in.\n>>> b\"abcde\".decode(\"utf-8\") \n'abcde'\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 606191 | |
| }, | |
| { | |
| "title": "How to know if an object has an attribute in Python", | |
| "body": "<p>Try <a href=\"https://docs.python.org/3/library/functions.html#hasattr\" rel=\"noreferrer\"><code>hasattr()</code></a>:</p>\n\n<pre><code>if hasattr(a, 'property'):\n a.property\n</code></pre>\n\n<p>EDIT: See <a href=\"https://stackoverflow.com/a/610923/117030\">zweiterlinde's answer</a> below, who offers good advice about asking forgiveness! A very pythonic approach! </p>\n\n<p>The general practice in python is that, if the property is likely to be there most of the time, simply call it and either let the exception propagate, or trap it with a try/except block. This will likely be faster than <code>hasattr</code>. If the property is likely to not be there most of the time, or you're not sure, using <code>hasattr</code> will probably be faster than repeatedly falling into an exception block.</p>\n", | |
| "source": "so", | |
| "questionId": 610883 | |
| }, | |
| { | |
| "title": "Find all files in a directory with extension .txt in Python", | |
| "body": "<p>You can use <a href=\"https://docs.python.org/2/library/glob.html\" rel=\"noreferrer\"><code>glob</code></a>:</p>\n\n<pre><code>import glob, os\nos.chdir(\"/mydir\")\nfor file in glob.glob(\"*.txt\"):\n print(file)\n</code></pre>\n\n<p>or simply <a href=\"https://docs.python.org/2/library/os.html#os.listdir\" rel=\"noreferrer\"><code>os.listdir</code></a>:</p>\n\n<pre><code>import os\nfor file in os.listdir(\"/mydir\"):\n if file.endswith(\".txt\"):\n print(os.path.join(\"/mydir\", file))\n</code></pre>\n\n<p>or if you want to traverse directory, use <a href=\"https://docs.python.org/2/library/os.html#os.walk\" rel=\"noreferrer\"><code>os.walk</code></a>:</p>\n\n<pre><code>import os\nfor root, dirs, files in os.walk(\"/mydir\"):\n for file in files:\n if file.endswith(\".txt\"):\n print(os.path.join(root, file))\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 3964681 | |
| }, | |
| { | |
| "title": "Limiting floats to two decimal points", | |
| "body": "<p>You are running into the old problem with floating point numbers that all numbers cannot be represented. The command line is just showing you the full floating point form from memory. In floating point your rounded version is the same number. Since computers are binary they store floating point numbers as an integer and then divide it by a power of two so 13.95 will be represented in a similar fashion to 125650429603636838/(2**53). Double precision numbers have 53 bits (16 digits) of precision and regular floats have 24 bits (8 digits) of precision. The <a href=\"http://docs.python.org/tutorial/floatingpoint.html\" rel=\"noreferrer\">floating point in python uses double precision</a> to store the values.</p>\n\n<p>for example</p>\n\n<pre><code> >>> 125650429603636838/(2**53)\n 13.949999999999999\n\n >>> 234042163/(2**24)\n 13.949999988079071\n\n >>> a=13.946\n >>> print(a)\n 13.946\n >>> print(\"%.2f\" % a)\n 13.95\n >>> round(a,2)\n 13.949999999999999\n >>> print(\"%.2f\" % round(a,2))\n 13.95\n >>> print(\"{0:.2f}\".format(a))\n 13.95\n >>> print(\"{0:.2f}\".format(round(a,2)))\n 13.95\n >>> print(\"{0:.15f}\".format(round(a,2)))\n 13.949999999999999\n</code></pre>\n\n<p>If you are after only two decimal places as in currency then you have a couple of better choices use integers and store values in cents not dollars and then divide by 100 to convert to dollars. Or use a fixed point number like <a href=\"http://docs.python.org/library/decimal.html\" rel=\"noreferrer\">decimal</a></p>\n", | |
| "source": "so", | |
| "questionId": 455612 | |
| }, | |
| { | |
| "title": "What IDE to use for Python?", | |
| "body": "\n\n<h2>Results</h2>\n\n<p><a href=\"https://docs.google.com/spreadsheets/d/1l3x94P55qoxqYbq5GosWQ7IonZ4vR-4ZyCaImiVmCSk/pubhtml\" rel=\"noreferrer\">Spreadsheet version</a></p>\n\n<p><a href=\"https://docs.google.com/spreadsheets/d/1l3x94P55qoxqYbq5GosWQ7IonZ4vR-4ZyCaImiVmCSk/pubhtml\" rel=\"noreferrer\"><img src=\"https://i.stack.imgur.com/Jrc6C.png\" alt=\"spreadsheet screenshot\"></a></p>\n\n<p>Alternatively, in plain text: (also available as a a <a href=\"https://i.stack.imgur.com/ItVKB.png\" rel=\"noreferrer\">screenshot</a>)</p>\n\n<pre class=\"lang-none prettyprint-override\"><code> Bracket Matching -. .- Line Numbering\n Smart Indent -. | | .- UML Editing / Viewing\n Source Control Integration -. | | | | .- Code Folding\n Error Markup -. | | | | | | .- Code Templates\nIntegrated Python Debugging -. | | | | | | | | .- Unit Testing\n Multi-Language Support -. | | | | | | | | | | .- GUI Designer (Qt, Eric, etc)\n Auto Code Completion -. | | | | | | | | | | | | .- Integrated DB Support\n Commercial/Free -. | | | | | | | | | | | | | | .- Rapid Application\n Cross Platform -. | | | | | | | | | | | | | | | | Development\n +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n Atom |Y |F |Y |Y*|Y |Y |Y |Y |Y |Y | |Y |Y | | | | |*many plugins\n Editra |Y |F |Y |Y | | |Y |Y |Y |Y | |Y | | | | | |\n Emacs |Y |F |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y | | | |\n Eric Ide |Y |F |Y | |Y |Y | |Y | |Y | |Y | |Y | | | |\n Geany |Y |F |Y*|Y | | | |Y |Y |Y | |Y | | | | | |*very limited\n Gedit |Y |F |Y¹|Y | | | |Y |Y |Y | | |Y²| | | | |¹with plugin; ²sort of\n Idle |Y |F |Y | |Y | | |Y |Y | | | | | | | | |\n JEdit |Y |F | |Y | | | | |Y |Y | |Y | | | | | |\n KDevelop |Y |F | |Y | | |Y |Y |Y |Y | |Y | | | | | |\n Komodo |Y |CF|Y |Y |Y |Y |Y |Y |Y |Y | |Y |Y |Y | |Y | |\n NetBeans* |Y |F |Y |Y |Y | |Y |Y |Y |Y |Y |Y |Y |Y | | |Y |*pre-v7.0\n Notepad++ |W |F |Y |Y | |Y*|Y*|Y*|Y |Y | |Y |Y*| | | | |*with plugin\n Pfaide |W |C |Y |Y | | | |Y |Y |Y | |Y |Y | | | | |\n PIDA |LW|F |Y |Y | | | |Y |Y |Y | |Y | | | | | |VIM based\n PTVS |W |F |Y |Y |Y |Y |Y |Y |Y |Y | |Y | | |Y*| |Y |*WPF bsed\n PyCharm |Y |CF|Y |Y*|Y |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y | |*JavaScript\n PyDev(Eclipse) |Y |F |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y |Y | | | |\n Pyscripter |W |F |Y | |Y |Y | |Y | |Y | | |Y |Y | | | |\n PythonWin |W |F |Y | |Y | | |Y |Y | | |Y | | | | | |\n SciTE |Y |F¹| |Y | |Y | |Y |Y |Y | |Y |Y | | | | |¹Mac version is\n ScriptDev |W |C |Y |Y |Y |Y | |Y |Y |Y | |Y |Y | | | | | commercial\n Spyder |Y |F |Y | |Y |Y | |Y |Y |Y | | | | | | | |\n Sublime Text |Y |CF|Y |Y | |Y |Y |Y |Y |Y | |Y |Y |Y*| | | |extensible w/Python,\n TextMate |M |F | |Y | | |Y |Y |Y |Y | |Y |Y | | | | | *PythonTestRunner\n UliPad |Y |F |Y |Y |Y | | |Y |Y | | | |Y |Y | | | |\n Vim |Y |F |Y |Y |Y |Y |Y |Y |Y |Y | |Y |Y |Y | | | |\n Visual Studio |W |CF|Y |Y |Y |? |Y |Y |Y |Y |? |Y |? |? |? |? |? |\n WingIde |Y |C |Y |Y*|Y |Y |Y |Y |Y |Y | |Y |Y |Y | | | |*support for C\n Zeus |W |C | | | | |Y |Y |Y |Y | |Y |Y | | | | |\n +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n Cross Platform -' | | | | | | | | | | | | | | | | Development\n Commercial/Free -' | | | | | | | | | | | | | | '- Rapid Application\n Auto Code Completion -' | | | | | | | | | | | | '- Integrated DB Support\n Multi-Language Support -' | | | | | | | | | | '- GUI Designer (Qt, Eric, etc)\nIntegrated Python Debugging -' | | | | | | | | '- Unit Testing\n Error Markup -' | | | | | | '- Code Templates\n Source Control Integration -' | | | | '- Code Folding\n Smart Indent -' | | '- UML Editing / Viewing\n Bracket Matching -' '- Line Numbering\n</code></pre>\n\n<hr>\n\n<p>Acronyms used:</p>\n\n<pre class=\"lang-none prettyprint-override\"><code> L - Linux\n W - Windows\n M - Mac\n C - Commercial\n F - Free\n CF - Commercial with Free limited edition\n ? - To be confirmed\n</code></pre>\n\n<p>I don't mention basics like syntax highlighting as I expect these by default.</p>\n\n<hr>\n\n<p>This is a just dry list reflecting your feedback and comments, I am not advocating any of these tools. I will keep updating this list as you keep posting your answers.</p>\n\n<p><strong><em>PS. Can you help me to add features of the above editors to the list (like auto-complete, debugging, etc.)?</em></strong></p>\n\n<p>We have a comprehensive wiki page for this question <a href=\"https://wiki.python.org/moin/IntegratedDevelopmentEnvironments\" rel=\"noreferrer\">https://wiki.python.org/moin/IntegratedDevelopmentEnvironments</a></p>\n\n<p><a href=\"https://docs.google.com/spreadsheets/d/1l3x94P55qoxqYbq5GosWQ7IonZ4vR-4ZyCaImiVmCSk/edit#gid=0&fvid=1960281650\" rel=\"noreferrer\" title=\"Submit edits to the spreadsheet\">Submit edits to the spreadsheet</a></p>\n", | |
| "source": "so", | |
| "questionId": 81584 | |
| }, | |
| { | |
| "title": "Reverse a string in Python", | |
| "body": "<p>How about:</p>\n\n<pre><code>>>> 'hello world'[::-1]\n'dlrow olleh'\n</code></pre>\n\n<p>This is <a href=\"http://docs.python.org/2/whatsnew/2.3.html#extended-slices\" rel=\"noreferrer\">extended slice</a> syntax. It works by doing <code>[begin:end:step]</code> - by leaving begin and end off and specifying a step of -1, it reverses a string.</p>\n", | |
| "source": "so", | |
| "questionId": 931092 | |
| }, | |
| { | |
| "title": "How to count the occurrences of a list item?", | |
| "body": "<p>If you only want one item's count, use the <code>count</code> method:</p>\n\n<pre><code>>>> [1, 2, 3, 4, 1, 4, 1].count(1)\n3\n</code></pre>\n\n<p><strong>Don't</strong> use this if you want to count multiple items. Calling <code>count</code> in a loop requires a separate pass over the list for every <code>count</code> call, which can be catastrophic for performance. If you want to count all items, or even just multiple items, use <code>Counter</code>, as explained in the other answers.</p>\n", | |
| "source": "so", | |
| "questionId": 2600191 | |
| }, | |
| { | |
| "title": "Parsing values from a JSON file?", | |
| "body": "<p>I think what Ignacio is saying is that your JSON file is incorrect. You have <code>[]</code> when you should have <code>{}</code>. <code>[]</code> are for lists, <code>{}</code> are for dictionaries.</p>\n\n<p>Here's how your JSON file should look, your JSON file wouldn't even load for me:</p>\n\n<pre><code>{\n \"maps\": [\n {\n \"id\": \"blabla\",\n \"iscategorical\": \"0\"\n },\n {\n \"id\": \"blabla\",\n \"iscategorical\": \"0\"\n }\n ],\n \"masks\": {\n \"id\": \"valore\"\n },\n \"om_points\": \"value\",\n \"parameters\": {\n \"id\": \"valore\"\n }\n}\n</code></pre>\n\n<p>Then you can use your code:</p>\n\n<pre><code>import json\nfrom pprint import pprint\n\ndata = json.load(open('data.json'))\n\npprint(data)\n</code></pre>\n\n<p>With data, you can now also find values like so:</p>\n\n<pre><code>data[\"maps\"][0][\"id\"]\ndata[\"masks\"][\"id\"]\ndata[\"om_points\"]\n</code></pre>\n\n<p>Try those out and see if it starts to make sense.</p>\n", | |
| "source": "so", | |
| "questionId": 2835559 | |
| }, | |
| { | |
| "title": "How to leave/exit/deactivate a python virtualenv?", | |
| "body": "<p>Usually, activating a virtualenv gives you a shell function named:</p>\n\n<pre><code>$ deactivate\n</code></pre>\n\n<p>which puts things back to normal.</p>\n\n<h3>Edit 1</h3>\n\n<p>I have just looked specifically again at the code for <code>virtualenvwrapper,</code> and, yes, it too supports <code>deactivate</code> as the way to escape from all virtualenvs.</p>\n\n<h3>Edit 2</h3>\n\n<p>If you are trying to leave an <em>Anaconda</em> environment, the procedure is a bit different: run the two-word command <code>source deactivate</code> since they implement deactivation using a stand-alone script.</p>\n\n<pre><code>bash-4.3$ deactivate\npyenv-virtualenv: deactivate must be sourced. Run 'source deactivate' instead of 'deactivate'\nbash-4.3$ source deactivate\npyenv-virtualenv: no virtualenv has been activated.\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 990754 | |
| }, | |
| { | |
| "title": "How do you change the size of figures drawn with matplotlib?", | |
| "body": "<p><a href=\"https://matplotlib.org/api/_as_gen/matplotlib.pyplot.figure.html#matplotlib.pyplot.figure\" rel=\"noreferrer\">figure</a> tells you the call signature:</p>\n\n<pre><code>figure(num=None, figsize=(8, 6), dpi=80, facecolor='w', edgecolor='k')\n</code></pre>\n\n<p>So <code>figure(figsize=(1,1))</code> creates an inch-by-inch image, which will be 80-by-80 pixels unless you also give a different dpi argument.</p>\n", | |
| "source": "so", | |
| "questionId": 332289 | |
| }, | |
| { | |
| "title": "Create a dictionary with list comprehension in Python", | |
| "body": "<p>In Python 2.6 and earlier, the dict constructor can receive an iterable of key/value pairs:</p>\n\n<pre><code>d = dict((key, value) for (key, value) in iterable)\n</code></pre>\n\n<p>From Python 2.7 and 3 onwards, you can just use the <a href=\"http://www.python.org/dev/peps/pep-0274/\" rel=\"noreferrer\">dict comprehension syntax</a> directly:</p>\n\n<pre><code>d = {key: value for (key, value) in iterable}\n</code></pre>\n\n<p>Of course, you can use the iterable in any way you want (tuples and lists literals, generator comprehensions, list comprehensions, generator functions, functional composition... feel creative) as long as each element is an iterable itself of two elements:</p>\n\n<pre><code>d = {value: foo(value) for value in sequence if bar(value)}\n\ndef key_value_gen(k):\n yield chr(k+65)\n yield chr((k+13)%26+65)\nd = dict(map(key_value_gen, range(26)))\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1747817 | |
| }, | |
| { | |
| "title": "Renaming columns in pandas", | |
| "body": "<p>Just assign it to the <code>.columns</code> attribute:</p>\n\n<pre><code>>>> df = pd.DataFrame({'$a':[1,2], '$b': [10,20]})\n>>> df.columns = ['a', 'b']\n>>> df\n a b\n0 1 10\n1 2 20\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 11346283 | |
| }, | |
| { | |
| "title": "How to remove an element from a list by index in Python?", | |
| "body": "<p>Use <code>del</code> and specify the element you want to delete with the index:</p>\n\n<pre><code>In [9]: a = list(range(10))\nIn [10]: a\nOut[10]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\nIn [11]: del a[-1]\nIn [12]: a\nOut[12]: [0, 1, 2, 3, 4, 5, 6, 7, 8]\n</code></pre>\n\n<p><a href=\"http://docs.python.org/tutorial/datastructures.html#the-del-statement\" rel=\"noreferrer\">Here</a> is the section from the tutorial.</p>\n", | |
| "source": "so", | |
| "questionId": 627435 | |
| }, | |
| { | |
| "title": "How to determine a Python variable's type?", | |
| "body": "<p>Python doesn't have the same types as C/C++, which appears to be your question.</p>\n\n<p>Try this:</p>\n\n<pre><code>>>> i = 123\n>>> type(i)\n<type 'int'>\n>>> type(i) is int\nTrue\n>>> i = 123456789L\n>>> type(i)\n<type 'long'>\n>>> type(i) is long\nTrue\n>>> i = 123.456\n>>> type(i)\n<type 'float'>\n>>> type(i) is float\nTrue\n</code></pre>\n\n<p>The distinction between int and long goes away in Python 3.0, though.</p>\n", | |
| "source": "so", | |
| "questionId": 402504 | |
| }, | |
| { | |
| "title": "How to remove a key from a python dictionary?", | |
| "body": "<p>Use <a href=\"http://docs.python.org/library/stdtypes.html#dict.pop\" rel=\"noreferrer\"><code>dict.pop()</code></a>:</p>\n\n<pre><code>my_dict.pop('key', None)\n</code></pre>\n\n<p>This will return <code>my_dict[key]</code> if <code>key</code> exists in the dictionary, and <code>None</code> otherwise. If the second parameter is not specified (ie. <code>my_dict.pop('key')</code>) and <code>key</code> does not exist, a <code>KeyError</code> is raised.</p>\n", | |
| "source": "so", | |
| "questionId": 11277432 | |
| }, | |
| { | |
| "title": "Nicest way to pad zeroes to string", | |
| "body": "<p>Strings:</p>\n\n<pre><code>>>> n = '4'\n>>> print n.zfill(3)\n004\n</code></pre>\n\n<p>And for numbers:</p>\n\n<pre><code>>>> n = 4\n>>> print '%03d' % n\n004\n>>> print format(n, '03') # python >= 2.6\n004\n>>> print '{0:03d}'.format(n) # python >= 2.6\n004\n>>> print '{foo:03d}'.format(foo=n) # python >= 2.6\n004\n>>> print('{:03d}'.format(n)) # python >= 2.7 + python3\n004\n>>> print('{0:03d}'.format(n)) # python 3\n004\n>>> print(f'{n:03}') # python >= 3.6\n004\n</code></pre>\n\n<p><a href=\"https://docs.python.org/2/library/string.html#formatexamples\" rel=\"noreferrer\">String formatting documentation</a>.</p>\n", | |
| "source": "so", | |
| "questionId": 339007 | |
| }, | |
| { | |
| "title": "Random string generation with upper case letters and digits in Python", | |
| "body": "<p><strong>Answer in one line:</strong></p>\n\n<pre><code>''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(N))\n</code></pre>\n\n<p>or even shorter starting with Python 3.6 using <a href=\"https://docs.python.org/3/library/random.html#random.choices\" rel=\"noreferrer\"><code>random.choices()</code></a>:</p>\n\n<pre><code>''.join(random.choices(string.ascii_uppercase + string.digits, k=N))\n</code></pre>\n\n<p><strong>A cryptographically more secure version; see <a href=\"https://stackoverflow.com/a/23728630/2213647\">https://stackoverflow.com/a/23728630/2213647</a>:</strong></p>\n\n<pre><code>''.join(random.SystemRandom().choice(string.ascii_uppercase + string.digits) for _ in range(N))\n</code></pre>\n\n<p><strong>In details, with a clean function for further reuse:</strong></p>\n\n<pre><code>>>> import string\n>>> import random\n>>> def id_generator(size=6, chars=string.ascii_uppercase + string.digits):\n... return ''.join(random.choice(chars) for _ in range(size))\n...\n>>> id_generator()\n'G5G74W'\n>>> id_generator(3, \"6793YUIO\")\n'Y3U'\n</code></pre>\n\n<p><strong>How does it work ?</strong></p>\n\n<p>We import <code>string</code>, a module that contains sequences of common ASCII characters, and <code>random</code>, a module that deals with random generation.</p>\n\n<p><code>string.ascii_uppercase + string.digits</code> just concatenates the list of characters representing uppercase ASCII chars and digits:</p>\n\n<pre><code>>>> string.ascii_uppercase\n'ABCDEFGHIJKLMNOPQRSTUVWXYZ'\n>>> string.digits\n'0123456789'\n>>> string.ascii_uppercase + string.digits\n'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'\n</code></pre>\n\n<p>Then we use a list comprehension to create a list of 'n' elements:</p>\n\n<pre><code>>>> range(4) # range create a list of 'n' numbers\n[0, 1, 2, 3]\n>>> ['elem' for _ in range(4)] # we use range to create 4 times 'elem'\n['elem', 'elem', 'elem', 'elem']\n</code></pre>\n\n<p>In the example above, we use <code>[</code> to create the list, but we don't in the <code>id_generator</code> function so Python doesn't create the list in memory, but generates the elements on the fly, one by one (more about this <a href=\"https://stackoverflow.com/questions/231767/the-python-yield-keyword-explained/231855#231855\">here</a>).</p>\n\n<p>Instead of asking to create 'n' times the string <code>elem</code>, we will ask Python to create 'n' times a random character, picked from a sequence of characters:</p>\n\n<pre><code>>>> random.choice(\"abcde\")\n'a'\n>>> random.choice(\"abcde\")\n'd'\n>>> random.choice(\"abcde\")\n'b'\n</code></pre>\n\n<p>Therefore <code>random.choice(chars) for _ in range(size)</code> really is creating a sequence of <code>size</code> characters. Characters that are randomly picked from <code>chars</code>:</p>\n\n<pre><code>>>> [random.choice('abcde') for _ in range(3)]\n['a', 'b', 'b']\n>>> [random.choice('abcde') for _ in range(3)]\n['e', 'b', 'e']\n>>> [random.choice('abcde') for _ in range(3)]\n['d', 'a', 'c']\n</code></pre>\n\n<p>Then we just join them with an empty string so the sequence becomes a string:</p>\n\n<pre><code>>>> ''.join(['a', 'b', 'b'])\n'abb'\n>>> [random.choice('abcde') for _ in range(3)]\n['d', 'c', 'b']\n>>> ''.join(random.choice('abcde') for _ in range(3))\n'dac'\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 2257441 | |
| }, | |
| { | |
| "title": "Getting the class name of an instance in Python", | |
| "body": "<p>Have you tried the <code>__name__</code> attribute of the class? ie <code>type(x).__name__</code> will give you the name of the class, which I think is what you want.</p>\n\n<pre><code>>>> import itertools\n>>> x = itertools.count(0)\n>>> type(x).__name__\n'count'\n</code></pre>\n\n<p>This method works with <a href=\"https://wiki.python.org/moin/NewClassVsClassicClass\" rel=\"noreferrer\">new-style classes</a> only. Your code might use some old-style classes. The following works for both:</p>\n\n<pre><code>x.__class__.__name__\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 510972 | |
| }, | |
| { | |
| "title": "How can you profile a script?", | |
| "body": "<p>Python includes a profiler called <a href=\"https://docs.python.org/3/library/profile.html#module-cProfile\" rel=\"noreferrer\">cProfile</a>. It not only gives the total running time, but also times each function separately, and tells you how many times each function was called, making it easy to determine where you should make optimizations.</p>\n\n<p>You can call it from within your code, or from the interpreter, like this:</p>\n\n<pre><code>import cProfile\ncProfile.run('foo()')\n</code></pre>\n\n<p>Even more usefully, you can invoke the cProfile when running a script:</p>\n\n<pre><code>python -m cProfile myscript.py\n</code></pre>\n\n<p>To make it even easier, I made a little batch file called 'profile.bat':</p>\n\n<pre><code>python -m cProfile %1\n</code></pre>\n\n<p>So all I have to do is run:</p>\n\n<pre><code>profile euler048.py\n</code></pre>\n\n<p>And I get this:</p>\n\n<pre class=\"lang-none prettyprint-override\"><code>1007 function calls in 0.061 CPU seconds\n\nOrdered by: standard name\nncalls tottime percall cumtime percall filename:lineno(function)\n 1 0.000 0.000 0.061 0.061 <string>:1(<module>)\n 1000 0.051 0.000 0.051 0.000 euler048.py:2(<lambda>)\n 1 0.005 0.005 0.061 0.061 euler048.py:2(<module>)\n 1 0.000 0.000 0.061 0.061 {execfile}\n 1 0.002 0.002 0.053 0.053 {map}\n 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler objects}\n 1 0.000 0.000 0.000 0.000 {range}\n 1 0.003 0.003 0.003 0.003 {sum}\n</code></pre>\n\n<p>EDIT: Updated link to a good video resource from PyCon 2013 titled \n<a href=\"http://lanyrd.com/2013/pycon/scdywg/\" rel=\"noreferrer\"><strong><em>Python Profiling</em></strong></a>.</p>\n", | |
| "source": "so", | |
| "questionId": 582336 | |
| }, | |
| { | |
| "title": "How to print to stderr in Python?", | |
| "body": "<p>I found this to be the only one short + flexible + portable + readable:</p>\n\n<pre><code>from __future__ import print_function\nimport sys\n\ndef eprint(*args, **kwargs):\n print(*args, file=sys.stderr, **kwargs)\n</code></pre>\n\n<p>The function <code>eprint</code> can be used in the same way as the standard <code>print</code> function:</p>\n\n<pre><code>>>> print(\"Test\")\nTest\n>>> eprint(\"Test\")\nTest\n>>> eprint(\"foo\", \"bar\", \"baz\", sep=\"---\")\nfoo---bar---baz\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 5574702 | |
| }, | |
| { | |
| "title": "Converting integer to string in Python?", | |
| "body": "<pre><code>>>> str(10)\n'10'\n>>> int('10')\n10\n</code></pre>\n\n<p>Links to the documentation:</p>\n\n<ul>\n<li><a href=\"https://docs.python.org/2/library/functions.html#int\" rel=\"noreferrer\"><code>int()</code></a> </li>\n<li><a href=\"https://docs.python.org/2/library/functions.html#str\" rel=\"noreferrer\"><code>str()</code></a></li>\n</ul>\n\n<p>The problem seems to come from this line: <code>d.str()</code>. </p>\n\n<p>Conversion to a string is done with the builtin <code>str()</code> function, which basically calls the <code>__str__()</code> method of its parameter.</p>\n\n<p>Also, it shouldn't be necessary to call <code>pow()</code>. Try using the <code>**</code> operator.</p>\n", | |
| "source": "so", | |
| "questionId": 961632 | |
| }, | |
| { | |
| "title": "What is the meaning of a single- and a double-underscore before an object name?", | |
| "body": "<h2>Single Underscore</h2>\n\n<p>Names, in a class, with a leading underscore are simply to indicate to other programmers that the attribute or method is intended to be private. However, nothing special is done with the name itself.</p>\n\n<p>To quote <a href=\"http://www.python.org/dev/peps/pep-0008/\" rel=\"noreferrer\">PEP-8</a>:</p>\n\n<blockquote>\n <p>_single_leading_underscore: weak \"internal use\" indicator. E.g. <code>from M import *</code> does not import objects whose name starts with an underscore.</p>\n</blockquote>\n\n<h2>Double Underscore (Name Mangling)</h2>\n\n<p>From <a href=\"http://docs.python.org/tutorial/classes.html#private-variables-and-class-local-references\" rel=\"noreferrer\">the Python docs</a>:</p>\n\n<blockquote>\n <p>Any identifier of the form <code>__spam</code> (at least two leading underscores, at most one trailing underscore) is textually replaced with <code>_classname__spam</code>, where <code>classname</code> is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, so it can be used to define class-private instance and class variables, methods, variables stored in globals, and even variables stored in instances. private to this class on instances of other classes. </p>\n</blockquote>\n\n<p>And a warning from the same page:</p>\n\n<blockquote>\n <p>Name mangling is intended to give classes an easy way to define “private” instance variables and methods, without having to worry about instance variables defined by derived classes, or mucking with instance variables by code outside the class. Note that the mangling rules are designed mostly to avoid accidents; it still is possible for a determined soul to access or modify a variable that is considered private.</p>\n</blockquote>\n\n<h2>Example</h2>\n\n<pre><code>>>> class MyClass():\n... def __init__(self):\n... self.__superprivate = \"Hello\"\n... self._semiprivate = \", world!\"\n...\n>>> mc = MyClass()\n>>> print mc.__superprivate\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\nAttributeError: myClass instance has no attribute '__superprivate'\n>>> print mc._semiprivate\n, world!\n>>> print mc.__dict__\n{'_MyClass__superprivate': 'Hello', '_semiprivate': ', world!'}\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1301346 | |
| }, | |
| { | |
| "title": "How do I connect to a MySQL Database in Python?", | |
| "body": "\n\n<h2>Connecting to MYSQL with Python in 3 steps</h2>\n\n<p><strong>1 - Setting</strong></p>\n\n<p>You must install a MySQL driver before doing anything. Unlike PHP, only the SQLite driver is installed by default with Python. The most used package to do so is <a href=\"http://pypi.python.org/pypi/MySQL-python/\" rel=\"noreferrer\">MySQLdb</a> but it's hard to install it using easy_install.</p>\n\n<p>For Windows user, you can get an <a href=\"http://sourceforge.net/project/showfiles.php?group_id=22307\" rel=\"noreferrer\">exe of MySQLdb</a>. </p>\n\n<p>For Linux, this is a casual package (python-mysqldb). (You can use <code>sudo apt-get install python-mysqldb</code> (for debian based distros), <code>yum install MySQL-python</code> (for rpm-based), or <code>dnf install python-mysql</code> (for modern fedora distro) in command line to download.)</p>\n\n<p>For Mac, you can <a href=\"https://stackoverflow.com/questions/1448429/how-to-install-mysqldb-python-data-access-library-to-mysql-on-mac-os-x#1448476\">install MySQLdb using Macport</a>.</p>\n\n<p><strong>2 - Usage</strong></p>\n\n<p>After installing, reboot. This is not mandatory, but will prevent me from answering 3 or 4 other questions in this post if something goes wrong. So please reboot.</p>\n\n<p>Then it is just like using another package :</p>\n\n<pre class=\"lang-py prettyprint-override\"><code>#!/usr/bin/python\nimport MySQLdb\n\ndb = MySQLdb.connect(host=\"localhost\", # your host, usually localhost\n user=\"john\", # your username\n passwd=\"megajonhy\", # your password\n db=\"jonhydb\") # name of the data base\n\n# you must create a Cursor object. It will let\n# you execute all the queries you need\ncur = db.cursor()\n\n# Use all the SQL you like\ncur.execute(\"SELECT * FROM YOUR_TABLE_NAME\")\n\n# print all the first cell of all the rows\nfor row in cur.fetchall():\n print row[0]\n\ndb.close()\n</code></pre>\n\n<p>Of course, there are thousand of possibilities and options; this is a very basic example. You will have to look at the documentation. <a href=\"http://www.mikusa.com/python-mysql-docs/\" rel=\"noreferrer\">A good starting point</a>.</p>\n\n<p><strong>3 - More advanced usage</strong></p>\n\n<p>Once you know how it works, you may want to use an <a href=\"https://en.wikipedia.org/wiki/Object-Relational_Mapping\" rel=\"noreferrer\">ORM</a> to avoid writting SQL manually and manipulate your tables as they were Python objects. The most famous ORM in the Python community is <a href=\"http://www.sqlalchemy.org/\" rel=\"noreferrer\">SQLAlchemy</a>. </p>\n\n<p>I strongly advise you to use it: your life is going to be much easier.</p>\n\n<p>I recently discovered another jewel in the Python world: <a href=\"http://peewee.readthedocs.org/en/latest/index.html\" rel=\"noreferrer\">peewee</a>. It's a very lite ORM, really easy and fast to setup then use. It makes my day for small projects or stand alone apps, where using big tools like SQLAlchemy or Django is overkill :</p>\n\n<pre class=\"lang-py prettyprint-override\"><code>import peewee\nfrom peewee import *\n\ndb = MySQLDatabase('jonhydb', user='john', passwd='megajonhy')\n\nclass Book(peewee.Model):\n author = peewee.CharField()\n title = peewee.TextField()\n\n class Meta:\n database = db\n\nBook.create_table()\nbook = Book(author=\"me\", title='Peewee is cool')\nbook.save()\nfor book in Book.filter(author=\"me\"):\n print book.title\n</code></pre>\n\n<p>This example works out of the box. Nothing other than having peewee (<code>pip install peewee</code>) is required.</p>\n", | |
| "source": "so", | |
| "questionId": 372885 | |
| }, | |
| { | |
| "title": "What are the differences between type() and isinstance()?", | |
| "body": "<p>To summarize the contents of other (already good!) answers, <code>isinstance</code> caters for inheritance (an instance of a derived class <em>is an</em> instance of a base class, too), while checking for equality of <code>type</code> does not (it demands identity of types and rejects instances of subtypes, AKA subclasses).</p>\n\n<p>Normally, in Python, you want your code to support inheritance, of course (since inheritance is so handy, it would be bad to stop code using yours from using it!), so <code>isinstance</code> is less bad than checking identity of <code>type</code>s because it seamlessly supports inheritance.</p>\n\n<p>It's not that <code>isinstance</code> is <em>good</em>, mind you—it's just <em>less bad</em> than checking equality of types. The normal, Pythonic, preferred solution is almost invariably \"duck typing\": try using the argument <em>as if</em> it was of a certain desired type, do it in a <code>try</code>/<code>except</code> statement catching all exceptions that could arise if the argument was not in fact of that type (or any other type nicely duck-mimicking it;-), and in the <code>except</code> clause, try something else (using the argument \"as if\" it was of some other type).</p>\n\n<p><code>basestring</code> <strong>is</strong>, however, quite a special case—a builtin type that exists <strong>only</strong> to let you use <code>isinstance</code> (both <code>str</code> and <code>unicode</code> subclass <code>basestring</code>). Strings are sequences (you could loop over them, index them, slice them, ...), but you generally want to treat them as \"scalar\" types—it's somewhat incovenient (but a reasonably frequent use case) to treat all kinds of strings (and maybe other scalar types, i.e., ones you can't loop on) one way, all containers (lists, sets, dicts, ...) in another way, and <code>basestring</code> plus <code>isinstance</code> helps you do that—the overall structure of this idiom is something like:</p>\n\n<pre><code>if isinstance(x, basestring)\n return treatasscalar(x)\ntry:\n return treatasiter(iter(x))\nexcept TypeError:\n return treatasscalar(x)\n</code></pre>\n\n<p>You could say that <code>basestring</code> is an <em>Abstract Base Class</em> (\"ABC\")—it offers no concrete functionality to subclasses, but rather exists as a \"marker\", mainly for use with <code>isinstance</code>. The concept is obviously a growing one in Python, since <a href=\"http://www.python.org/dev/peps/pep-3119/\" rel=\"noreferrer\">PEP 3119</a>, which introduces a generalization of it, was accepted and has been implemented starting with Python 2.6 and 3.0.</p>\n\n<p>The PEP makes it clear that, while ABCs can often substitute for duck typing, there is generally no big pressure to do that (see <a href=\"http://www.python.org/dev/peps/pep-3119/#abcs-vs-duck-typing\" rel=\"noreferrer\">here</a>). ABCs as implemented in recent Python versions do however offer extra goodies: <code>isinstance</code> (and <code>issubclass</code>) can now mean more than just \"[an instance of] a derived class\" (in particular, any class can be \"registered\" with an ABC so that it will show as a subclass, and its instances as instances of the ABC); and ABCs can also offer extra convenience to actual subclasses in a very natural way via Template Method design pattern applications (see <a href=\"http://en.wikipedia.org/wiki/Template_method_pattern\" rel=\"noreferrer\">here</a> and <a href=\"http://www.catonmat.net/blog/learning-python-design-patterns-through-video-lectures/\" rel=\"noreferrer\">here</a> [[part II]] for more on the TM DP, in general and specifically in Python, independent of ABCs).</p>\n\n<p>For the underlying mechanics of ABC support as offered in Python 2.6, see <a href=\"http://docs.python.org/library/abc.html\" rel=\"noreferrer\">here</a>; for their 3.1 version, very similar, see <a href=\"http://docs.python.org/3.1/library/abc.html\" rel=\"noreferrer\">here</a>. In both versions, standard library module <a href=\"http://docs.python.org/3.1/library/collections.html#module-collections\" rel=\"noreferrer\">collections</a> (that's the 3.1 version—for the very similar 2.6 version, see <a href=\"http://docs.python.org/library/collections.html#module-collections\" rel=\"noreferrer\">here</a>) offers several useful ABCs.</p>\n\n<p>For the purpose of this answer, the key thing to retain about ABCs (beyond an arguably more natural placement for TM DP functionality, compared to the classic Python alternative of mixin classes such as <a href=\"http://docs.python.org/library/userdict.html?highlight=userdict#UserDict.DictMixin\" rel=\"noreferrer\">UserDict.DictMixin</a>) is that they make <code>isinstance</code> (and <code>issubclass</code>) much more attractive and pervasive (in Python 2.6 and going forward) than they used to be (in 2.5 and before), and therefore, by contrast, make checking type equality an even worse practice in recent Python versions than it already used to be.</p>\n", | |
| "source": "so", | |
| "questionId": 1549801 | |
| }, | |
| { | |
| "title": "UnicodeEncodeError: 'ascii' codec can't encode character u'\\xa0' in position 20: ordinal not in range(128)", | |
| "body": "<p>You need to read the Python <a href=\"https://docs.python.org/2.7/howto/unicode.html\" rel=\"noreferrer\">Unicode HOWTO</a>. This error is the <a href=\"https://docs.python.org/2.7/howto/unicode.html#the-unicode-type\" rel=\"noreferrer\">very first example</a>.</p>\n\n<p>Basically, stop using <code>str</code> to convert from unicode to encoded text / bytes.</p>\n\n<p>Instead, properly use <a href=\"http://docs.python.org/library/stdtypes.html#str.encode\" rel=\"noreferrer\"><code>.encode()</code></a> to encode the string:</p>\n\n<pre><code>p.agent_info = u' '.join((agent_contact, agent_telno)).encode('utf-8').strip()\n</code></pre>\n\n<p>or work entirely in unicode.</p>\n", | |
| "source": "so", | |
| "questionId": 9942594 | |
| }, | |
| { | |
| "title": "How can I print literal curly-brace characters in python string and also use .format on it?", | |
| "body": "<p>You need to double the <code>{{</code> and <code>}}</code>:</p>\n\n<pre><code>>>> x = \" {{ Hello }} {0} \"\n>>> print x.format(42)\n' { Hello } 42 '\n</code></pre>\n\n<p>Here's the relevant part of the <a href=\"http://docs.python.org/library/string.html#formatstrings\">Python documentation for format string syntax</a>:</p>\n\n<blockquote>\n <p>Format strings contain “replacement fields” surrounded by curly braces <code>{}</code>. Anything that is not contained in braces is considered literal text, which is copied unchanged to the output. If you need to include a brace character in the literal text, it can be escaped by doubling: <code>{{</code> and <code>}}</code>.</p>\n</blockquote>\n", | |
| "source": "so", | |
| "questionId": 5466451 | |
| }, | |
| { | |
| "title": "How to use threading in Python?", | |
| "body": "<p>Since this question was asked in 2010, there has been real simplification in how to do simple multithreading with python with <strong><a href=\"https://docs.python.org/2/library/functions.html#map\" rel=\"noreferrer\">map</a></strong> and <strong><a href=\"https://docs.python.org/2/library/multiprocessing.html\" rel=\"noreferrer\">pool</a></strong>.</p>\n\n<p>The code below comes from an article/blog post that you should definitely check out (no affiliation) - <strong><a href=\"http://chriskiehl.com/article/parallelism-in-one-line/\" rel=\"noreferrer\">Parallelism in one line:\nA Better Model for Day to Day Threading Tasks</a></strong>. I'll summarize below - it ends up being just a few lines of code:</p>\n\n<pre><code>from multiprocessing.dummy import Pool as ThreadPool \npool = ThreadPool(4) \nresults = pool.map(my_function, my_array)\n</code></pre>\n\n<p>Which is the multithreaded version of:</p>\n\n<pre><code>results = []\nfor item in my_array:\n results.append(my_function(item))\n</code></pre>\n\n<hr>\n\n<p><strong>Description</strong></p>\n\n<blockquote>\n <p>Map is a cool little function, and the key to easily injecting parallelism into your Python code. For those unfamiliar, map is something lifted from functional languages like Lisp. It is a function which maps another function over a sequence.</p>\n \n <p>Map handles the iteration over the sequence for us, applies the function, and stores all of the results in a handy list at the end.</p>\n</blockquote>\n\n<p><img src=\"https://i.stack.imgur.com/Yq37m.png\" alt=\"enter image description here\"></p>\n\n<hr>\n\n<p><strong>Implementation</strong></p>\n\n<blockquote>\n <p>Parallel versions of the map function are provided by two libraries:multiprocessing, and also its little known, but equally fantastic step child:multiprocessing.dummy.</p>\n</blockquote>\n\n<pre><code>import urllib2 \nfrom multiprocessing.dummy import Pool as ThreadPool \n\nurls = [\n 'http://www.python.org', \n 'http://www.python.org/about/',\n 'http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html',\n 'http://www.python.org/doc/',\n 'http://www.python.org/download/',\n 'http://www.python.org/getit/',\n 'http://www.python.org/community/',\n 'https://wiki.python.org/moin/',\n ]\n\n# make the Pool of workers\npool = ThreadPool(4) \n\n# open the urls in their own threads\n# and return the results\nresults = pool.map(urllib2.urlopen, urls)\n\n# close the pool and wait for the work to finish \npool.close() \npool.join() \n</code></pre>\n\n<p>And the timing results:</p>\n\n<pre><code>Single thread: 14.4 seconds\n 4 Pool: 3.1 seconds\n 8 Pool: 1.4 seconds\n 13 Pool: 1.3 seconds\n</code></pre>\n\n<hr>\n\n<p><strong>Passing multiple arguments</strong> (works like this only in Python 3.3 and later): (<a href=\"https://stackoverflow.com/a/28975239/2327328\">source</a>):</p>\n\n<p>To pass multiple arrays:</p>\n\n<pre><code>results = pool.starmap(function, zip(list_a, list_b))\n</code></pre>\n\n<p>or to pass a constant and an array:</p>\n\n<pre><code>results = pool.starmap(function, zip(itertools.repeat(constant), list_a))\n</code></pre>\n\n<p>If you are using an earlier version of Python, you can pass multiple arguments via <a href=\"https://stackoverflow.com/a/5443941/1893275\">this workaround</a>.</p>\n\n<p>(Thanks to <a href=\"https://stackoverflow.com/users/2441026/user136036\">user136036</a> for the helpful comment)</p>\n", | |
| "source": "so", | |
| "questionId": 2846653 | |
| }, | |
| { | |
| "title": "Why use pip over easy_install?", | |
| "body": "<p>Many of the answers here are out of date for 2015 (although <a href=\"https://stackoverflow.com/a/3220572/908494\">the initially accepted one from Daniel Roseman</a> is not). Here's the current state of things:</p>\n\n<ul>\n<li>Binary packages are now distributed as wheels (<code>.whl</code> files)—not just on PyPI, but in third-party repositories like <a href=\"http://www.lfd.uci.edu/~gohlke/pythonlibs/\" rel=\"noreferrer\">Christoph Gohlke's Extension Packages for Windows</a>. <code>pip</code> can handle wheels; <code>easy_install</code> cannot.</li>\n<li>Virtual environments (which come built-in with 3.4, or can be added to 2.6+/3.1+ with <a href=\"https://pypi.python.org/pypi/virtualenv\" rel=\"noreferrer\"><code>virtualenv</code></a>) have become a very important and prominent tool (and recommended in <a href=\"https://docs.python.org/3/installing/index.html\" rel=\"noreferrer\">the official docs</a>); they include <code>pip</code> out of the box, but don't even work properly with <code>easy_install</code>.</li>\n<li>The <code>distribute</code> package that included <code>easy_install</code> is no longer maintained. Its improvements over <code>setuptools</code> got merged back into <code>setuptools</code>. Trying to install <code>distribute</code> will just install <code>setuptools</code> instead.</li>\n<li><code>easy_install</code> itself is only quasi-maintained.</li>\n<li>All of the cases where <code>pip</code> used to be inferior to <code>easy_install</code>—installing from an unpacked source tree, from a DVCS repo, etc.—are long-gone; you can <code>pip install .</code>, <code>pip install git+https://</code>.</li>\n<li><code>pip</code> comes with the official Python 2.7 and 3.4+ packages from python.org, and a <code>pip</code> bootstrap is included by default if you build from source.</li>\n<li>The various incomplete bits of documentation on installing, using, and building packages have been replaced by the <a href=\"https://packaging.python.org/\" rel=\"noreferrer\">Python Packaging User Guide</a>. Python's own documentation on <a href=\"https://docs.python.org/3/installing/index.html\" rel=\"noreferrer\">Installing Python Modules</a> now defers to this user guide, and explicitly calls out <code>pip</code> as \"the preferred installer program\".</li>\n<li>Other new features have been added to <code>pip</code> over the years that will never be in <code>easy_install</code>. For example, <code>pip</code> makes it easy to clone your site-packages by building a requirements file and then installing it with a single command on each side. Or to convert your requirements file to a local repo to use for in-house development. And so on.</li>\n</ul>\n\n<p>The only good reason that I know of to use <code>easy_install</code> in 2015 is the special case of using Apple's pre-installed Python versions with OS X 10.5-10.8. Since 10.5, Apple has included <code>easy_install</code>, but as of 10.10 they still don't include <code>pip</code>. With 10.9+, you should still just use <code>get-pip.py</code>, but for 10.5-10.8, this has some problems, so it's easier to <code>sudo easy_install pip</code>. (In general, <code>easy_install pip</code> is a bad idea; it's only for OS X 10.5-10.8 that you want to do this.) Also, 10.5-10.8 include <code>readline</code> in a way that <code>easy_install</code> knows how to kludge around but <code>pip</code> doesn't, so you also want to <code>sudo easy_install readline</code> if you want to upgrade that.</p>\n", | |
| "source": "so", | |
| "questionId": 3220404 | |
| }, | |
| { | |
| "title": "How do I trim whitespace?", | |
| "body": "<p>Whitespace on both sides:</p>\n\n<pre><code>s = \" \\t a string example\\t \"\ns = s.strip()\n</code></pre>\n\n<p>Whitespace on the right side:</p>\n\n<pre><code>s = s.rstrip()\n</code></pre>\n\n<p>Whitespace on the left side:</p>\n\n<pre><code>s = s.lstrip()\n</code></pre>\n\n<p>As <a href=\"https://stackoverflow.com/users/84380/thedz\">thedz</a> points out, you can provide an argument to strip arbitrary characters to any of these functions like this:</p>\n\n<pre><code>s = s.strip(' \\t\\n\\r')\n</code></pre>\n\n<p>This will strip any space, <code>\\t</code>, <code>\\n</code>, or <code>\\r</code> characters from the left-hand side, right-hand side, or both sides of the string. </p>\n\n<p>The examples above only remove strings from the left-hand and right-hand sides of strings. If you want to also remove characters from the middle of a string, try <code>re.sub</code>:</p>\n\n<pre><code>import re\nprint re.sub('[\\s+]', '', s)\n</code></pre>\n\n<p>That should print out:</p>\n\n<pre><code>astringexample\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1185524 | |
| }, | |
| { | |
| "title": "How do I trim whitespace from a Python string?", | |
| "body": "<p>Just one space, or all such spaces? If the second, then strings already have a <code>.strip()</code> method:</p>\n\n<pre><code>>>> ' Hello '.strip()\n'Hello'\n>>> ' Hello'.strip()\n'Hello'\n>>> 'Bob has a cat'.strip()\n'Bob has a cat'\n>>> ' Hello '.strip() # ALL spaces at ends removed\n'Hello'\n</code></pre>\n\n<p>If you need only to remove one space however, you could do it with:</p>\n\n<pre><code>def strip_one_space(s):\n if s.endswith(\" \"): s = s[:-1]\n if s.startswith(\" \"): s = s[1:]\n return s\n\n>>> strip_one_space(\" Hello \")\n' Hello'\n</code></pre>\n\n<p>Also, note that <code>str.strip()</code> removes other whitespace characters as well (e.g. tabs and newlines). To remove only spaces, you can specify the character to remove as an argument to <code>strip</code>, i.e.:</p>\n\n<pre><code>>>> \" Hello\\n\".strip(\" \")\n'Hello\\n'\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 761804 | |
| }, | |
| { | |
| "title": "Does Django scale?", | |
| "body": "<ol>\n<li><p><strong>\"What are the largest sites built on Django today?\"</strong></p>\n\n<p>There isn't any single place that collects information about traffic on Django built sites, so I'll have to take a stab at it using data from various locations. First, we have a list of Django sites on the front page of <a href=\"http://www.djangoproject.com/\" rel=\"noreferrer\">the main Django project page</a> and then a list of Django built sites at <a href=\"http://www.djangosites.org/\" rel=\"noreferrer\">djangosites.org</a>. Going through the lists and picking some that I know have decent traffic we see:</p>\n\n<ul>\n<li><p><strong><a href=\"http://instagram.com\" rel=\"noreferrer\">Instagram</a></strong>: <a href=\"http://instagram-engineering.tumblr.com/post/13649370142/what-powers-instagram-hundreds-of-instances\" rel=\"noreferrer\">What Powers Instagram: Hundreds of Instances, Dozens of Technologies</a>.</p></li>\n<li><p><strong><a href=\"http://pinterest.com/\" rel=\"noreferrer\">Pinterest</a></strong>: <a href=\"http://www.alexa.com/siteinfo/Pinterest.com\" rel=\"noreferrer\">Alexa rank 37 (21.4.2015)</a> and 70 Million users in 2013</p></li>\n<li><p><strong><a href=\"http://bitbucket.org/\" rel=\"noreferrer\">Bitbucket</a></strong>: <a href=\"https://blog.bitbucket.org/2015/02/05/bitbucket-2014-in-review/\" rel=\"noreferrer\">200TB of Code and 2.500.000 Users</a></p></li>\n<li><p><strong><a href=\"http://disqus.com\" rel=\"noreferrer\">Disqus</a></strong>: <a href=\"http://pyvideo.org/video/418/pycon-2011--disqus--serving-400-million-people-wi\" rel=\"noreferrer\">Serving 400 million people with Python</a>.</p></li>\n<li><p><strong><a href=\"http://curse.com/\" rel=\"noreferrer\">curse.com</a></strong>: <a href=\"http://www.quantcast.com/curse.com\" rel=\"noreferrer\">600k daily visits</a>.</p></li>\n<li><p><strong><a href=\"http://tabblo.com/\" rel=\"noreferrer\">tabblo.com</a></strong>: <a href=\"http://www.quantcast.com/tabblo.com\" rel=\"noreferrer\">44k daily visits</a>, see Ned Batchelder's posts <a href=\"http://nedbatchelder.com/blog/200902/infrastructure_for_modern_web_sites.html\" rel=\"noreferrer\">Infrastructure for modern web sites</a>.</p></li>\n<li><p><strong><a href=\"http://chesspark.com/\" rel=\"noreferrer\">chesspark.com</a></strong>: <a href=\"http://www.alexa.com/siteinfo/chesspark.com\" rel=\"noreferrer\">Alexa</a> rank about 179k.</p></li>\n<li><p><strong><a href=\"http://pownce.com/\" rel=\"noreferrer\">pownce.com</a></strong> (no longer active): <a href=\"http://www.alexa.com/siteinfo/pownce.com\" rel=\"noreferrer\">alexa</a> rank about 65k.\nMike Malone of Pownce, in his EuroDjangoCon presentation on <strong><a href=\"http://www.slideshare.net/road76/scaling-django\" rel=\"noreferrer\">Scaling Django Web Apps</a></strong> says \"hundreds of hits per second\". This is a very good presentation on how to scale Django, and makes some good points including (current) shortcomings in Django scalability.</p></li>\n<li><p>HP had a site built with Django 1.5: <a href=\"http://www.eprintcenter.com\" rel=\"noreferrer\">ePrint center</a>. However, as for novemer/2015 the entire website was migrated and this link is just a redirect. This website was a world-wide service attending subscription to Instant Ink and related services HP offered (*).</p></li>\n</ul></li>\n<li><p><strong>\"Can Django deal with 100,000 users daily, each visiting the site for a couple of hours?\"</strong></p>\n\n<p>Yes, see above.</p></li>\n<li><p><strong>\"Could a site like Stack Overflow run on Django?\"</strong></p>\n\n<p>My gut feeling is yes but, as others answered and Mike Malone mentions in his presentation, database design is critical. Strong proof might also be found at www.cnprog.com if we can find any reliable traffic stats. Anyway, it's not just something that will happen by throwing together a bunch of Django models :)</p></li>\n</ol>\n\n<p>There are, of course, many more sites and bloggers of interest, but I have got to stop somewhere!</p>\n\n<hr>\n\n<p>Blog post about <a href=\"http://web.archive.org/web/20130307032621/http://concentricsky.com/blog/2009/oct/michaelmoorecom\" rel=\"noreferrer\">Using Django to build high-traffic site michaelmoore.com</a> described as a <a href=\"http://www.alexa.com/siteinfo/http%3A%2F%2Fmichaelmoore.com\" rel=\"noreferrer\">top 10,000 website</a>. <a href=\"http://www.quantcast.com/michaelmoore.com\" rel=\"noreferrer\">Quantcast stats</a> and <a href=\"http://siteanalytics.compete.com/michaelmoore.com/\" rel=\"noreferrer\">compete.com stats</a>.</p>\n\n<hr>\n\n<p><sub>(*) The author of the edit, including such reference, used to work as outsourced developer in that project.</sub></p>\n", | |
| "source": "so", | |
| "questionId": 886221 | |
| }, | |
| { | |
| "title": "Most elegant way to check if the string is empty in Python?", | |
| "body": "<p>Empty strings are <a href=\"http://docs.python.org/2/library/stdtypes.html#truth-value-testing\" rel=\"noreferrer\">\"falsy\"</a> which means they are considered false in a Boolean context, so you can just do this:</p>\n\n<pre><code>if not myString:\n</code></pre>\n\n<p>This is the preferred way if you know that your variable is a string. If your variable could also be some other type then you should use <code>myString == \"\"</code>. See the documentation on <a href=\"http://docs.python.org/library/stdtypes.html#truth-value-testing\" rel=\"noreferrer\">Truth Value Testing</a> for other values that are false in Boolean contexts.</p>\n", | |
| "source": "so", | |
| "questionId": 9573244 | |
| }, | |
| { | |
| "title": "What's the canonical way to check for type in Python?", | |
| "body": "<p>To check if <code>o</code> is an instance of <code>str</code> or any subclass of <code>str</code> (this would be the \"canonical\" way):</p>\n\n<pre><code>if isinstance(o, str):\n</code></pre>\n\n<p>To check if the type of <code>o</code> is exactly <code>str</code>:</p>\n\n<pre><code>if type(o) is str:\n</code></pre>\n\n<p>The following also works, and can be useful in some cases:</p>\n\n<pre><code>if issubclass(type(o), str):\n\nif type(o) in ([str] + str.__subclasses__()):\n</code></pre>\n\n<p>See <a href=\"http://docs.python.org/2/library/functions.html\" rel=\"noreferrer\">Built-in Functions</a> in the Python Library Reference for relevant information.</p>\n\n<p>One more note: in this case, you may actually want to use:</p>\n\n<pre><code>if isinstance(o, basestring):\n</code></pre>\n\n<p>because this will also catch Unicode strings (<code>unicode</code> is not a subclass of <code>str</code>; both <code>str</code> and <code>unicode</code> are subclasses of <code>basestring</code>).</p>\n\n<p>Alternatively, <code>isinstance</code> accepts a tuple of classes. This will return True if x is an instance of any subclass of any of (str, unicode):</p>\n\n<pre><code>if isinstance(o, (str, unicode)):\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 152580 | |
| }, | |
| { | |
| "title": "What is the purpose of self?", | |
| "body": "<p>The reason you need to use <code>self.</code> is because Python does not use the <code>@</code> syntax to refer to instance attributes. Python decided to do methods in a way that makes the instance to which the method belongs be <em>passed</em> automatically, but not <em>received</em> automatically: the first parameter of methods is the instance the method is called on. That makes methods entirely the same as functions, and leaves the actual name to use up to you (although <code>self</code> is the convention, and people will generally frown at you when you use something else.) <code>self</code> is not special to the code, it's just another object.</p>\n\n<p>Python could have done something else to distinguish normal names from attributes -- special syntax like Ruby has, or requiring declarations like C++ and Java do, or perhaps something yet more different -- but it didn't. Python's all for making things explicit, making it obvious what's what, and although it doesn't do it entirely everywhere, it does do it for instance attributes. That's why assigning to an instance attribute needs to know what instance to assign to, and that's why it needs <code>self.</code>.</p>\n", | |
| "source": "so", | |
| "questionId": 2709821 | |
| }, | |
| { | |
| "title": "Why does comparing strings in Python using either '==' or 'is' sometimes produce a different result?", | |
| "body": "<p><code>is</code> is identity testing, <code>==</code> is equality testing. what happens in your code would be emulated in the interpreter like this:</p>\n\n<pre><code>>>> a = 'pub'\n>>> b = ''.join(['p', 'u', 'b'])\n>>> a == b\nTrue\n>>> a is b\nFalse\n</code></pre>\n\n<p>so, no wonder they're not the same, right?</p>\n\n<p>In other words: <code>is</code> is the <code>id(a) == id(b)</code></p>\n", | |
| "source": "so", | |
| "questionId": 1504717 | |
| }, | |
| { | |
| "title": "Extracting extension from filename in Python", | |
| "body": "<p>Yes. Use <a href=\"https://docs.python.org/2/library/os.path.html#os.path.splitext\" rel=\"noreferrer\" title=\"os.path.splitext\"><code>os.path.splitext</code></a>:</p>\n\n<pre><code>>>> import os\n>>> filename, file_extension = os.path.splitext('/path/to/somefile.ext')\n>>> filename\n'/path/to/somefile'\n>>> file_extension\n'.ext'\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 541390 | |
| }, | |
| { | |
| "title": "Proper way to declare custom exceptions in modern Python?", | |
| "body": "<p>Maybe I missed the question, but why not:</p>\n\n<pre><code>class MyException(Exception):\n pass\n</code></pre>\n\n<p><strong>Edit:</strong> to override something (or pass extra args), do this:</p>\n\n<pre><code>class ValidationError(Exception):\n def __init__(self, message, errors):\n\n # Call the base class constructor with the parameters it needs\n super(ValidationError, self).__init__(message)\n\n # Now for your custom code...\n self.errors = errors\n</code></pre>\n\n<p>That way you could pass dict of error messages to the second param, and get to it later with <code>e.errors</code></p>\n\n<hr>\n\n<p><strong>Python 3 Update:</strong> In Python 3+, you can use this slightly more compact use of <code>super()</code>:</p>\n\n<pre><code>class ValidationError(Exception):\n def __init__(self, message, errors):\n\n # Call the base class constructor with the parameters it needs\n super().__init__(message)\n\n # Now for your custom code...\n self.errors = errors\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1319615 | |
| }, | |
| { | |
| "title": "Generate random integers between 0 and 9", | |
| "body": "<p>Try:</p>\n\n<pre><code>from random import randint\nprint(randint(0, 9))\n</code></pre>\n\n<p>More info: <a href=\"https://docs.python.org/3/library/random.html#random.randint\" rel=\"noreferrer\">https://docs.python.org/3/library/random.html#random.randint</a></p>\n", | |
| "source": "so", | |
| "questionId": 3996904 | |
| }, | |
| { | |
| "title": "How to flush output of Python print?", | |
| "body": "<pre><code>import sys\nsys.stdout.flush()\n</code></pre>\n\n<p>Print by default prints to <code>sys.stdout</code>.</p>\n\n<p>References:</p>\n\n<ul>\n<li><a href=\"http://docs.python.org/reference/simple_stmts.html#the-print-statement\" rel=\"noreferrer\">http://docs.python.org/reference/simple_stmts.html#the-print-statement</a></li>\n<li><a href=\"http://docs.python.org/library/sys.html\" rel=\"noreferrer\">http://docs.python.org/library/sys.html</a></li>\n<li><a href=\"http://docs.python.org/library/stdtypes.html#file-objects\" rel=\"noreferrer\">http://docs.python.org/library/stdtypes.html#file-objects</a></li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 230751 | |
| }, | |
| { | |
| "title": "What is the difference between old style and new style classes in Python?", | |
| "body": "<p>From <a href=\"http://docs.python.org/2/reference/datamodel.html#new-style-and-classic-classes\" rel=\"noreferrer\">http://docs.python.org/2/reference/datamodel.html#new-style-and-classic-classes</a> :</p>\n\n<blockquote>\n <p>Up to Python 2.1, old-style classes were the only flavour available to the user. The concept of (old-style) class is unrelated to the concept of type: if <code>x</code> is an instance of an old-style class, then <code>x.__class__</code> designates the class of <code>x</code>, but <code>type(x)</code> is always <code><type 'instance'></code>. This reflects the fact that all old-style instances, independently of their class, are implemented with a single built-in type, called instance. </p>\n \n <p><strong>New-style classes were introduced in Python 2.2 to unify the concepts of class and type</strong>. A new-style class is simply a user-defined type, no more, no less. If x is an instance of a new-style class, then <code>type(x)</code> is typically the same as <code>x.__class__</code> (although this is not guaranteed – a new-style class instance is permitted to override the value returned for <code>x.__class__</code>).</p>\n \n <p><strong>The major motivation for introducing new-style classes is to provide a unified object model with a full meta-model</strong>. It also has a number of immediate benefits, like the ability to subclass most built-in types, or the introduction of \"descriptors\", which enable computed properties. </p>\n \n <p><strong>For compatibility reasons, classes are still old-style by default</strong>. New-style classes are created by specifying another new-style class (i.e. a type) as a parent class, or the \"top-level type\" object if no other parent is needed. The behaviour of new-style classes differs from that of old-style classes in a number of important details in addition to what type returns. Some of these changes are fundamental to the new object model, like the way special methods are invoked. Others are \"fixes\" that could not be implemented before for compatibility concerns, like the method resolution order in case of multiple inheritance. </p>\n \n <p><strong>Python 3 only has new-style classes</strong>. No matter if you subclass from <code>object</code> or not, classes are new-style in Python 3.</p>\n</blockquote>\n", | |
| "source": "so", | |
| "questionId": 54867 | |
| }, | |
| { | |
| "title": "What's the difference between lists and tuples?", | |
| "body": "<p>Apart from tuples being immutable there is also a semantic distinction that should guide their usage. Tuples are heterogeneous data structures (i.e., their entries have different meanings), while lists are homogeneous sequences. <strong>Tuples have structure, lists have order.</strong> </p>\n\n<p>Using this distinction makes code more explicit and understandable.</p>\n\n<p>One example would be pairs of page and line number to reference locations in a book, e.g.:</p>\n\n<pre><code>my_location = (42, 11) # page number, line number\n</code></pre>\n\n<p>You can then use this as a key in a dictionary to store notes on locations. A list on the other hand could be used to store multiple locations. Naturally one might want to add or remove locations from the list, so it makes sense that lists are mutable. On the other hand it doesn't make sense to add or remove items from an existing location - hence tuples are immutable.</p>\n\n<p>There might be situations where you want to change items within an existing location tuple, for example when iterating through the lines of a page. But tuple immutability forces you to create a new location tuple for each new value. This seems inconvenient on the face of it, but using immutable data like this is a cornerstone of value types and functional programming techniques, which can have substantial advantages.</p>\n\n<p>There are some interesting articles on this issue, e.g. <a href=\"http://jtauber.com/blog/2006/04/15/python_tuples_are_not_just_constant_lists/\" rel=\"noreferrer\">\"Python Tuples are Not Just Constant Lists\"</a> or <a href=\"http://news.e-scribe.com/397\" rel=\"noreferrer\">\"Understanding tuples vs. lists in Python\"</a>. The official Python documentation <a href=\"http://docs.python.org/2/tutorial/datastructures.html#tuples-and-sequences\" rel=\"noreferrer\">also mentions this</a> (<em>\"Tuples are immutable, and usually contain an heterogeneous sequence ...\"</em>).</p>\n\n<p>In a statically typed language like <em>Haskell</em> the values in a tuple generally have different types and the length of the tuple must be fixed. In a list the values all have the same type and the length is not fixed. So the difference is very obvious.</p>\n\n<p>Finally there is the <a href=\"http://docs.python.org/dev/library/collections.html#collections.namedtuple\" rel=\"noreferrer\">namedtuple</a> in Python, which makes sense because a tuple is already supposed to have structure. This underlines the idea that tuples are a light-weight alternative to classes and instances.</p>\n", | |
| "source": "so", | |
| "questionId": 626759 | |
| }, | |
| { | |
| "title": "error: Unable to find vcvarsall.bat", | |
| "body": "<p><strong><em>Update</strong>: Comments point out that the instructions here may be dangerous. Consider using the Visual C++ 2008 Express edition or the purpose-built <a href=\"http://www.microsoft.com/en-us/download/details.aspx?id=44266\" rel=\"noreferrer\">Microsoft Visual C++ Compiler for Python</a> (<a href=\"/a/26127562/2778484\">details</a>) and <strong>NOT</strong> using the original answer below. Original error message means the required version of Visual C++ is not installed.</em></p>\n\n<hr>\n\n<p>For Windows installations:</p>\n\n<p>While running setup.py for package installations, Python 2.7 searches for an installed Visual Studio 2008. You can trick Python to use a newer Visual Studio by setting the correct path in <code>VS90COMNTOOLS</code> environment variable before calling <code>setup.py</code>.</p>\n\n<p>Execute the following command based on the version of Visual Studio installed:</p>\n\n<ul>\n<li>Visual Studio 2010 (VS10): <code>SET VS90COMNTOOLS=%VS100COMNTOOLS%</code></li>\n<li>Visual Studio 2012 (VS11): <code>SET VS90COMNTOOLS=%VS110COMNTOOLS%</code></li>\n<li>Visual Studio 2013 (VS12): <code>SET VS90COMNTOOLS=%VS120COMNTOOLS%</code></li>\n<li>Visual Studio 2015 (VS14): <code>SET VS90COMNTOOLS=%VS140COMNTOOLS%</code></li>\n</ul>\n\n<hr>\n\n<p>WARNING: As noted below, this answer is unlikely to work if you are trying to compile python modules.</p>\n\n<p>See <a href=\"https://stackoverflow.com/questions/3047542\">Building lxml for Python 2.7 on Windows</a> for details.</p>\n", | |
| "source": "so", | |
| "questionId": 2817869 | |
| }, | |
| { | |
| "title": "Way to create multiline comments in Python?", | |
| "body": "<p>You can use triple-quoted strings. When they're not a docstring (first thing in a class/function/module), they are ignored. </p>\n\n<pre><code>'''\nThis is a multiline\ncomment.\n'''\n</code></pre>\n\n<p>(Make sure to indent the leading <code>'''</code> appropriately to avoid an <code>IndentationError</code>.)</p>\n\n<p>Guido van Rossum (creator of Python) <a href=\"https://twitter.com/gvanrossum/status/112670605505077248\" rel=\"noreferrer\">tweeted this</a> as a \"pro tip\".</p>\n\n<p>However, Python's style guide, PEP8, <a href=\"http://www.python.org/dev/peps/pep-0008/#block-comments\" rel=\"noreferrer\">favors using consecutive single-line comments</a>, and this is also what you'll find in many projects. Editors usually have a shortcut to do this easily.</p>\n", | |
| "source": "so", | |
| "questionId": 7696924 | |
| }, | |
| { | |
| "title": "How do I check what version of Python is running my script?", | |
| "body": "<p>This information is available in the <a href=\"http://docs.python.org/2/library/sys.html#sys.version\" rel=\"noreferrer\">sys.version</a> string in the <a href=\"http://docs.python.org/2/library/sys.html\" rel=\"noreferrer\">sys</a> module:</p>\n\n<pre><code>>>> import sys\n</code></pre>\n\n<p>Human readable:</p>\n\n<pre><code>>>> print (sys.version) #parentheses necessary in python 3. \n2.5.2 (r252:60911, Jul 31 2008, 17:28:52) \n[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)]\n</code></pre>\n\n<p>For further processing:</p>\n\n<pre><code>>>> sys.version_info\n(2, 5, 2, 'final', 0)\n# or\n>>> sys.hexversion\n34014192\n</code></pre>\n\n<p>To ensure a script runs with a minimal version requirement of the Python interpreter add this to your code:</p>\n\n<pre><code>assert sys.version_info >= (2,5)\n</code></pre>\n\n<p>This compares major and minor version information. Add micro (=<code>0</code>, <code>1</code>, etc) and even releaselevel (=<code>'alpha'</code>,<code>'final'</code>, etc) to the tuple as you like. Note however, that it is almost always better to \"duck\" check if a certain feature is there, and if not, workaround (or bail out). Sometimes features go away in newer releases, being replaced by others.</p>\n", | |
| "source": "so", | |
| "questionId": 1093322 | |
| }, | |
| { | |
| "title": "Why do people write the #!/usr/bin/env python shebang on the first line of a Python script?", | |
| "body": "<p>If you have several versions of Python installed, <code>/usr/bin/env</code> will ensure the interpreter used is the first one on your environment's <code>$PATH</code>. The alternative would be to hardcode something like <code>#!/usr/bin/python</code>; that's ok, but less flexible.</p>\n\n<p>In Unix, an <em>executable</em> file that's meant to be interpreted can indicate what interpreter to use by having a <code>#!</code> at the start of the first line, followed by the interpreter (and any flags it may need).</p>\n\n<p>If you're talking about other platforms, of course, this rule does not apply (but that \"shebang line\" does no harm, and will help if you ever copy that script to a platform <em>with</em> a Unix base, such as Linux, Mac, etc).</p>\n", | |
| "source": "so", | |
| "questionId": 2429511 | |
| }, | |
| { | |
| "title": "Why are Python lambdas useful?", | |
| "body": "<p>Are you talking about <a href=\"https://docs.python.org/3.5/tutorial/controlflow.html#lambda-expressions\" rel=\"noreferrer\">lambda functions</a>? Like</p>\n\n<pre><code>lambda x: x**2 + 2*x - 5\n</code></pre>\n\n<p>Those things are actually quite useful. Python supports a style of programming called <em>functional programming</em> where you can pass functions to other functions to do stuff. Example:</p>\n\n<pre><code>mult3 = filter(lambda x: x % 3 == 0, [1, 2, 3, 4, 5, 6, 7, 8, 9])\n</code></pre>\n\n<p>sets <code>mult3</code> to <code>[3, 6, 9]</code>, those elements of the original list that are multiples of 3. This is shorter (and, one could argue, clearer) than</p>\n\n<pre><code>def filterfunc(x):\n return x % 3 == 0\nmult3 = filter(filterfunc, [1, 2, 3, 4, 5, 6, 7, 8, 9])\n</code></pre>\n\n<p>Of course, in this particular case, you could do the same thing as a list comprehension:</p>\n\n<pre><code>mult3 = [x for x in [1, 2, 3, 4, 5, 6, 7, 8, 9] if x % 3 == 0]\n</code></pre>\n\n<p>(or even as <code>range(3,10,3)</code>), but there are many other, more sophisticated use cases where you can't use a list comprehension and a lambda function may be the shortest way to write something out.</p>\n\n<ul>\n<li><p>Returning a function from another function</p>\n\n<pre><code>>>> def transform(n):\n... return lambda x: x + n\n...\n>>> f = transform(3)\n>>> f(4)\n7\n</code></pre>\n\n<p>This is often used to create function wrappers, such as Python's decorators.</p></li>\n<li><p>Combining elements of an iterable sequence with <code>reduce()</code></p>\n\n<pre><code>>>> reduce(lambda a, b: '{}, {}'.format(a, b), [1, 2, 3, 4, 5, 6, 7, 8, 9])\n'1, 2, 3, 4, 5, 6, 7, 8, 9'\n</code></pre></li>\n<li><p>Sorting by an alternate key</p>\n\n<pre><code>>>> sorted([1, 2, 3, 4, 5, 6, 7, 8, 9], key=lambda x: abs(5-x))\n[5, 4, 6, 3, 7, 2, 8, 1, 9]\n</code></pre></li>\n</ul>\n\n<p>I use lambda functions on a regular basis. It took me a while to get used to them, but eventually I came to understand that they're a very valuable part of the language.</p>\n", | |
| "source": "so", | |
| "questionId": 890128 | |
| }, | |
| { | |
| "title": "Installing specific package versions with pip", | |
| "body": "<p>First, I see two issues with what you're trying to do. Since you already have an installed version, you should either uninstall the current existing driver or use <code>pip install -I MySQL_python==1.2.2</code></p>\n\n<p>However, you'll soon find out that this doesn't work. If you look at pip's installation log, or if you do a <code>pip install -Iv MySQL_python==1.2.2</code> you'll find that the PyPI URL link does not work for MySQL_python v1.2.2. You can verify this here: <a href=\"http://pypi.python.org/pypi/MySQL-python/1.2.2\" rel=\"noreferrer\">http://pypi.python.org/pypi/MySQL-python/1.2.2</a></p>\n\n<p>The download link 404s and the fallback URL links are re-directing infinitely due to sourceforge.net's recent upgrade and PyPI's stale URL.</p>\n\n<p>So to properly install the driver, you can follow these steps:</p>\n\n<pre><code>pip uninstall MySQL_python\npip install -Iv http://sourceforge.net/projects/mysql-python/files/mysql-python/1.2.2/MySQL-python-1.2.2.tar.gz/download\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 5226311 | |
| }, | |
| { | |
| "title": "Delete an item from a dictionary", | |
| "body": "<p>The <a href=\"http://docs.python.org/reference/simple_stmts.html#grammar-token-del_stmt\" rel=\"noreferrer\"><code>del</code> statement</a> removes an element:</p>\n\n<pre><code>del d[key]\n</code></pre>\n\n<p>However, this mutates the existing dictionary so the contents of the dictionary changes for anybody else who has a reference to the same instance. To return a <em>new</em> dictionary, make a copy of the dictionary:</p>\n\n<pre><code>def removekey(d, key):\n r = dict(d)\n del r[key]\n return r\n</code></pre>\n\n<p>The <code>dict()</code> constructor makes a <em>shallow copy</em>. To make a deep copy, see the <a href=\"https://docs.python.org/library/copy.html\" rel=\"noreferrer\"><code>copy</code> module</a>.</p>\n", | |
| "source": "so", | |
| "questionId": 5844672 | |
| }, | |
| { | |
| "title": "In Python, how do I determine if an object is iterable?", | |
| "body": "<ol>\n<li><p>Checking for <code>__iter__</code> works on sequence types, but it would fail on e.g. strings <strong>in Python 2</strong>. I would like to know the right answer too, until then, here is one possibility (which would work on strings, too):</p>\n\n<pre><code>try:\n some_object_iterator = iter(some_object)\nexcept TypeError, te:\n print some_object, 'is not iterable'\n</code></pre>\n\n<p>The <code>iter</code> built-in checks for the <code>__iter__</code> method or in the case of strings the <code>__getitem__</code> method.</p></li>\n<li><p>Another general pythonic approach is to assume an iterable, then fail gracefully if it does not work on the given object. The Python glossary:</p>\n\n<blockquote>\n <p>Pythonic programming style that determines an object's type by inspection of its method or attribute signature rather than by explicit relationship to some type object (\"If it looks like a <strong>duck</strong> and quacks like a <strong>duck</strong>, it must be a <strong>duck</strong>.\") By emphasizing interfaces rather than specific types, well-designed code improves its flexibility by allowing polymorphic substitution. Duck-typing avoids tests using type() or isinstance(). <strong>Instead, it typically employs the EAFP (Easier to Ask Forgiveness than Permission) style of programming.</strong></p>\n \n <p>...</p>\n\n<pre><code>try:\n _ = (e for e in my_object)\nexcept TypeError:\n print my_object, 'is not iterable'\n</code></pre>\n</blockquote></li>\n<li><p>The <a href=\"http://docs.python.org/library/collections.html#abcs-abstract-base-classes\" rel=\"noreferrer\"><code>collections</code></a> module provides some abstract base classes, which allow to ask classes or instances if they provide particular functionality, for example:</p>\n\n<pre><code>import collections\n\nif isinstance(e, collections.Iterable):\n # e is iterable\n</code></pre>\n\n<p>However, this does not check for classes that are iterable through <code>__getitem__</code>.</p></li>\n</ol>\n", | |
| "source": "so", | |
| "questionId": 1952464 | |
| }, | |
| { | |
| "title": "Delete column from pandas DataFrame using del df.column_name", | |
| "body": "<p>It's difficult to make <code>del df.column_name</code> work simply as the result of syntactic limitations in Python. <code>del df[name]</code> gets translated to <code>df.__delitem__(name)</code> under the covers by Python.</p>\n", | |
| "source": "so", | |
| "questionId": 13411544 | |
| }, | |
| { | |
| "title": "How can I do a line break (line continuation) in Python?", | |
| "body": "<p>What is the line? You can just have arguments on the next line without any problems:</p>\n\n<pre><code>a = dostuff(blahblah1, blahblah2, blahblah3, blahblah4, blahblah5, \n blahblah6, blahblah7)\n</code></pre>\n\n<p>Otherwise you can do something like this:</p>\n\n<pre><code>if a == True and \\\n b == False\n</code></pre>\n\n<p>Check the <a href=\"http://www.python.org/dev/peps/pep-0008/\" rel=\"noreferrer\">style guide</a> for more information.</p>\n\n<p>From your example line:</p>\n\n<pre><code>a = '1' + '2' + '3' + \\\n '4' + '5'\n</code></pre>\n\n<p>Or:</p>\n\n<pre><code>a = ('1' + '2' + '3' +\n '4' + '5')\n</code></pre>\n\n<p>Note that the style guide says that using the implicit continuation with parentheses is preferred, but in this particular case just adding parentheses around your expression is probably the wrong way to go.</p>\n", | |
| "source": "so", | |
| "questionId": 53162 | |
| }, | |
| { | |
| "title": "If Python is interpreted, what are .pyc files?", | |
| "body": "<p>They contain <a href=\"http://en.wikipedia.org/wiki/Bytecode\" rel=\"noreferrer\">byte code</a>, which is what the Python interpreter compiles the source to. This code is then executed by Python's virtual machine. </p>\n\n<p>Python's documentation explains the definition like this:</p>\n\n<blockquote>\n <p>Python is an interpreted language, as\n opposed to a compiled one, though the\n distinction can be blurry because of\n the presence of the bytecode compiler.\n This means that source files can be\n run directly without explicitly\n creating an executable which is then\n run.</p>\n</blockquote>\n", | |
| "source": "so", | |
| "questionId": 2998215 | |
| }, | |
| { | |
| "title": "ASCII value of a character in Python", | |
| "body": "<p>From <a href=\"http://mail.python.org/pipermail/python-win32/2005-April/003100.html\" rel=\"noreferrer\">here</a>:</p>\n\n<blockquote>\n <p>function ord() would get the int value\n of the char. And in case you want to\n convert back after playing with the\n number, function chr() does the trick.</p>\n</blockquote>\n\n<pre><code>>>> ord('a')\n97\n>>> chr(97)\n'a'\n>>> chr(ord('a') + 3)\n'd'\n>>>\n</code></pre>\n\n<p>In Python 2, there is also the <code>unichr</code> function, returning the <a href=\"http://en.wikipedia.org/wiki/Unicode\" rel=\"noreferrer\">Unicode</a> character whose ordinal is the <code>unichr</code> argument:</p>\n\n<pre><code>>>> unichr(97)\nu'a'\n>>> unichr(1234)\nu'\\u04d2'\n</code></pre>\n\n<p>In Python 3 you can use <code>chr</code> instead of <code>unichr</code>.</p>\n\n<hr>\n\n<p><a href=\"https://docs.python.org/3/library/functions.html#ord\" rel=\"noreferrer\">ord() - Python 3.6.5rc1 documentation</a></p>\n\n<p><a href=\"https://docs.python.org/2/library/functions.html#ord\" rel=\"noreferrer\">ord() - Python 2.7.14 documentation</a></p>\n", | |
| "source": "so", | |
| "questionId": 227459 | |
| }, | |
| { | |
| "title": "null object in Python?", | |
| "body": "<p>In Python, the 'null' object is the singleton <code>None</code>.</p>\n\n<p>The best way to check things for \"Noneness\" is to use the identity operator, <code>is</code>:</p>\n\n<pre><code>if foo is None:\n ...\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 3289601 | |
| }, | |
| { | |
| "title": "Use different Python version with virtualenv", | |
| "body": "<p>Just use the <code>--python</code> (or short <code>-p</code>) option when creating your virtualenv instance to specify the Python executable you want to use, e.g.:</p>\n\n<pre><code>virtualenv --python=/usr/bin/python2.6 <path/to/new/virtualenv/>\n</code></pre>\n\n<p>N.B. For <strong>Python 3.3</strong> or later, refer to The Aelfinn's <a href=\"https://stackoverflow.com/a/39713544/1450294\">answer</a> below. <em>[Editor's note: I know this should normally be a comment, not an edit, but a new comment would be hidden, and I just spent 45 minutes untangling errors because this important answer was buried under three or four parrot answers. I'm just trying to save everyone time here.]</em></p>\n", | |
| "source": "so", | |
| "questionId": 1534210 | |
| }, | |
| { | |
| "title": "How do you return multiple values in Python?", | |
| "body": "<p><a href=\"http://docs.python.org/library/collections.html#namedtuple-factory-function-for-tuples-with-named-fields\" rel=\"noreferrer\">Named tuples</a> were added in 2.6 for this purpose. Also see <a href=\"http://docs.python.org/library/os.html#os.stat\" rel=\"noreferrer\">os.stat</a> for a similar builtin example.</p>\n\n<pre><code>>>> import collections\n>>> Point = collections.namedtuple('Point', ['x', 'y'])\n>>> p = Point(1, y=2)\n>>> p.x, p.y\n1 2\n>>> p[0], p[1]\n1 2\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 354883 | |
| }, | |
| { | |
| "title": "Correct way to write line to file?", | |
| "body": "<p>This should be as simple as:</p>\n\n<pre><code>with open('somefile.txt', 'a') as the_file:\n the_file.write('Hello\\n')\n</code></pre>\n\n<p>From The Documentation:</p>\n\n<blockquote>\n <p>Do not use <code>os.linesep</code> as a line terminator when writing files opened in text mode (the default); use a single '\\n' instead, on all platforms.</p>\n</blockquote>\n\n<p>Some useful reading:</p>\n\n<ul>\n<li><a href=\"http://docs.python.org/reference/compound_stmts.html#the-with-statement\" rel=\"noreferrer\">The <code>with</code> statement</a></li>\n<li><a href=\"http://docs.python.org/library/functions.html?highlight=open#open\" rel=\"noreferrer\"><code>open()</code></a>\n\n<ul>\n<li>'a' is for append, or use</li>\n<li>'w' to write with truncation</li>\n</ul></li>\n<li><a href=\"http://docs.python.org/library/os.html\" rel=\"noreferrer\"><code>os</code></a> (particularly <a href=\"http://docs.python.org/library/os.html?highlight=os.linesep#os.linesep\" rel=\"noreferrer\"><code>os.linesep</code></a>)</li>\n</ul>\n", | |
| "source": "so", | |
| "questionId": 6159900 | |
| }, | |
| { | |
| "title": "How to import a module given the full path?", | |
| "body": "<p>For Python 3.5+ use:</p>\n\n<pre><code>import importlib.util\nspec = importlib.util.spec_from_file_location(\"module.name\", \"/path/to/file.py\")\nfoo = importlib.util.module_from_spec(spec)\nspec.loader.exec_module(foo)\nfoo.MyClass()\n</code></pre>\n\n<p>For Python 3.3 and 3.4 use:</p>\n\n<pre><code>from importlib.machinery import SourceFileLoader\n\nfoo = SourceFileLoader(\"module.name\", \"/path/to/file.py\").load_module()\nfoo.MyClass()\n</code></pre>\n\n<p>(Although this has been deprecated in Python 3.4.)</p>\n\n<p>Python 2 use:</p>\n\n<pre><code>import imp\n\nfoo = imp.load_source('module.name', '/path/to/file.py')\nfoo.MyClass()\n</code></pre>\n\n<p>There are equivalent convenience functions for compiled Python files and DLLs.</p>\n\n<p>See also. <a href=\"http://bugs.python.org/issue21436\" rel=\"noreferrer\">http://bugs.python.org/issue21436</a>.</p>\n", | |
| "source": "so", | |
| "questionId": 67631 | |
| }, | |
| { | |
| "title": "How do I parse XML in Python?", | |
| "body": "<p>I suggest <a href=\"http://docs.python.org/library/xml.etree.elementtree.html\" rel=\"noreferrer\"><code>ElementTree</code></a>. There are other compatible implementations of the same API, such as <a href=\"http://lxml.de/\" rel=\"noreferrer\"><code>lxml</code></a>, and <code>cElementTree</code> in the Python standard library itself; but, in this context, what they chiefly add is even more speed -- the ease of programming part depends on the API, which <code>ElementTree</code> defines.</p>\n\n<p>After building an Element instance <code>e</code> from the XML, e.g. with the <a href=\"http://docs.python.org/library/xml.etree.elementtree.html#xml.etree.ElementTree.XML\" rel=\"noreferrer\">XML</a> function, or by parsing a file with something like</p>\n\n<pre><code>import xml.etree.ElementTree\ne = xml.etree.ElementTree.parse('thefile.xml').getroot()\n</code></pre>\n\n<p>or any of the many other ways shown at <a href=\"http://docs.python.org/library/xml.etree.elementtree.html\" rel=\"noreferrer\"><code>ElementTree</code></a>, you just do something like:</p>\n\n<pre><code>for atype in e.findall('type'):\n print(atype.get('foobar'))\n</code></pre>\n\n<p>and similar, usually pretty simple, code patterns.</p>\n", | |
| "source": "so", | |
| "questionId": 1912434 | |
| }, | |
| { | |
| "title": "Peak detection in a 2D array", | |
| "body": "<p>I detected the peaks using a <strong>local maximum filter</strong>. Here is the result on your first dataset of 4 paws:\n<img src=\"https://i.stack.imgur.com/Kgt4H.png\" alt=\"Peaks detection result\"></p>\n\n<p>I also ran it on the second dataset of 9 paws and <a href=\"https://i.stack.imgur.com/4CKCh.png\" rel=\"noreferrer\">it worked as well</a>.</p>\n\n<p>Here is how you do it:</p>\n\n<pre><code>import numpy as np\nfrom scipy.ndimage.filters import maximum_filter\nfrom scipy.ndimage.morphology import generate_binary_structure, binary_erosion\nimport matplotlib.pyplot as pp\n\n#for some reason I had to reshape. Numpy ignored the shape header.\npaws_data = np.loadtxt(\"paws.txt\").reshape(4,11,14)\n\n#getting a list of images\npaws = [p.squeeze() for p in np.vsplit(paws_data,4)]\n\n\ndef detect_peaks(image):\n \"\"\"\n Takes an image and detect the peaks usingthe local maximum filter.\n Returns a boolean mask of the peaks (i.e. 1 when\n the pixel's value is the neighborhood maximum, 0 otherwise)\n \"\"\"\n\n # define an 8-connected neighborhood\n neighborhood = generate_binary_structure(2,2)\n\n #apply the local maximum filter; all pixel of maximal value \n #in their neighborhood are set to 1\n local_max = maximum_filter(image, footprint=neighborhood)==image\n #local_max is a mask that contains the peaks we are \n #looking for, but also the background.\n #In order to isolate the peaks we must remove the background from the mask.\n\n #we create the mask of the background\n background = (image==0)\n\n #a little technicality: we must erode the background in order to \n #successfully subtract it form local_max, otherwise a line will \n #appear along the background border (artifact of the local maximum filter)\n eroded_background = binary_erosion(background, structure=neighborhood, border_value=1)\n\n #we obtain the final mask, containing only peaks, \n #by removing the background from the local_max mask (xor operation)\n detected_peaks = local_max ^ eroded_background\n\n return detected_peaks\n\n\n#applying the detection and plotting results\nfor i, paw in enumerate(paws):\n detected_peaks = detect_peaks(paw)\n pp.subplot(4,2,(2*i+1))\n pp.imshow(paw)\n pp.subplot(4,2,(2*i+2) )\n pp.imshow(detected_peaks)\n\npp.show()\n</code></pre>\n\n<p>All you need to do after is use <em>scipy.ndimage.measurements.label</em> on the mask to label all distinct objects. Then you'll be able to play with them individually.</p>\n\n<p><strong>Note</strong> that the method works well because the background is not noisy. If it were, you would detect a bunch of other unwanted peaks in the background. Another important factor is the size of the <em>neighborhood</em>. You will need to adjust it if the peak size changes (the should remain roughly proportional).</p>\n", | |
| "source": "so", | |
| "questionId": 3684484 | |
| }, | |
| { | |
| "title": "Single quotes vs. double quotes in Python", | |
| "body": "<p>I like to use double quotes around strings that are used for interpolation or that are natural language messages, and single quotes for small symbol-like strings, but will break the rules if the strings contain quotes, or if I forget. I use triple double quotes for docstrings and raw string literals for regular expressions even if they aren't needed.</p>\n\n<p>For example:</p>\n\n<pre><code>LIGHT_MESSAGES = {\n 'English': \"There are %(number_of_lights)s lights.\",\n 'Pirate': \"Arr! Thar be %(number_of_lights)s lights.\"\n}\n\ndef lights_message(language, number_of_lights):\n \"\"\"Return a language-appropriate string reporting the light count.\"\"\"\n return LIGHT_MESSAGES[language] % locals()\n\ndef is_pirate(message):\n \"\"\"Return True if the given message sounds piratical.\"\"\"\n return re.search(r\"(?i)(arr|avast|yohoho)!\", message) is not None\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 56011 | |
| }, | |
| { | |
| "title": "How can I get a list of locally installed Python modules?", | |
| "body": "<h2>Solution</h2>\n\n<p>My 50 cents for getting a <code>pip freeze</code>-like list from a Python script:</p>\n\n<pre class=\"lang-python prettyprint-override\"><code>import pip\ninstalled_packages = pip.get_installed_distributions()\ninstalled_packages_list = sorted([\"%s==%s\" % (i.key, i.version)\n for i in installed_packages])\nprint(installed_packages_list)\n</code></pre>\n\n<p>As a (too long) one liner:</p>\n\n<pre class=\"lang-python prettyprint-override\"><code>sorted([\"%s==%s\" % (i.key, i.version) for i in pip.get_installed_distributions()])\n</code></pre>\n\n<p>Giving:</p>\n\n<pre class=\"lang-js prettyprint-override\"><code>['behave==1.2.4', 'enum34==1.0', 'flask==0.10.1', 'itsdangerous==0.24', \n 'jinja2==2.7.2', 'jsonschema==2.3.0', 'markupsafe==0.23', 'nose==1.3.3', \n 'parse-type==0.3.4', 'parse==1.6.4', 'prettytable==0.7.2', 'requests==2.3.0',\n 'six==1.6.1', 'vioozer-metadata==0.1', 'vioozer-users-server==0.1', \n 'werkzeug==0.9.4']\n</code></pre>\n\n<h2>Scope</h2>\n\n<p>This solution applies to the system scope or to a virtual environment scope, and covers packages installed by <code>setuptools</code>, <code>pip</code> and (<a href=\"https://stackoverflow.com/questions/3220404/why-use-pip-over-easy-install\">god forbid</a>) <code>easy_install</code>.</p>\n\n<h2>My use case</h2>\n\n<p>I added the result of this call to my flask server, so when I call it with <code>http://example.com/exampleServer/environment</code> I get the list of packages installed on the server's virtualenv. It makes debugging a whole lot easier.</p>\n\n<h2>Caveats</h2>\n\n<p>I have noticed a strange behaviour of this technique - when the Python interpreter is invoked in the same directory as a <code>setup.py</code> file, it does not list the package installed by <code>setup.py</code>.</p>\n\n<h3>Steps to reproduce:</h3>\n\nCreate a virtual environment\n\n<pre><code>$ cd /tmp\n$ virtualenv test_env\nNew python executable in test_env/bin/python\nInstalling setuptools, pip...done.\n$ source test_env/bin/activate\n(test_env) $ \n</code></pre>\n\nClone a git repo with <code>setup.py</code>\n\n<pre><code>(test_env) $ git clone https://github.com/behave/behave.git\nCloning into 'behave'...\nremote: Reusing existing pack: 4350, done.\nremote: Total 4350 (delta 0), reused 0 (delta 0)\nReceiving objects: 100% (4350/4350), 1.85 MiB | 418.00 KiB/s, done.\nResolving deltas: 100% (2388/2388), done.\nChecking connectivity... done.\n</code></pre>\n\n<p>We have behave's <code>setup.py</code> in <code>/tmp/behave</code>:</p>\n\n<pre><code>(test_env) $ ls /tmp/behave/setup.py\n/tmp/behave/setup.py\n</code></pre>\n\nInstall the python package from the git repo\n\n<pre><code>(test_env) $ cd /tmp/behave && python setup.py install\nrunning install\n...\nInstalled /private/tmp/test_env/lib/python2.7/site-packages/enum34-1.0-py2.7.egg\nFinished processing dependencies for behave==1.2.5a1\n</code></pre>\n\n<h3>If we run the aforementioned solution from <code>/tmp</code></h3>\n\n<pre><code>>>> import pip\n>>> sorted([\"%s==%s\" % (i.key, i.version) for i in pip.get_installed_distributions()])\n['behave==1.2.5a1', 'enum34==1.0', 'parse-type==0.3.4', 'parse==1.6.4', 'six==1.6.1']\n>>> import os\n>>> os.getcwd()\n'/private/tmp'\n</code></pre>\n\n<h3>If we run the aforementioned solution from <code>/tmp/behave</code></h3>\n\n<pre><code>>>> import pip\n>>> sorted([\"%s==%s\" % (i.key, i.version) for i in pip.get_installed_distributions()])\n['enum34==1.0', 'parse-type==0.3.4', 'parse==1.6.4', 'six==1.6.1']\n>>> import os\n>>> os.getcwd()\n'/private/tmp/behave'\n</code></pre>\n\n<p><code>behave==1.2.5a1</code> is missing from the second example, because the working directory contains <code>behave</code>'s <code>setup.py</code> file.</p>\n\n<p>I could not find any reference to this issue in the documentation. Perhaps I shall open a bug for it.</p>\n", | |
| "source": "so", | |
| "questionId": 739993 | |
| }, | |
| { | |
| "title": "Why does Python code run faster in a function?", | |
| "body": "<p>You might ask <em>why</em> it is faster to store local variables than globals. This is a CPython implementation detail.</p>\n\n<p>Remember that CPython is compiled to bytecode, which the interpreter runs. When a function is compiled, the local variables are stored in a fixed-size array (<em>not</em> a <code>dict</code>) and variable names are assigned to indexes. This is possible because you can't dynamically add local variables to a function. Then retrieving a local variable is literally a pointer lookup into the list and a refcount increase on the <code>PyObject</code> which is trivial.</p>\n\n<p>Contrast this to a global lookup (<code>LOAD_GLOBAL</code>), which is a true <code>dict</code> search involving a hash and so on. Incidentally, this is why you need to specify <code>global i</code> if you want it to be global: if you ever assign to a variable inside a scope, the compiler will issue <code>STORE_FAST</code>s for its access unless you tell it not to.</p>\n\n<p>By the way, global lookups are still pretty optimised. Attribute lookups <code>foo.bar</code> are the <em>really</em> slow ones!</p>\n\n<p>Here is small <a href=\"https://wiki.python.org/moin/PythonSpeed/PerformanceTips#Local_Variables\" rel=\"noreferrer\">illustration</a> on local variable efficiency.</p>\n", | |
| "source": "so", | |
| "questionId": 11241523 | |
| }, | |
| { | |
| "title": "Terminating a Python script", | |
| "body": "<pre><code>import sys\nsys.exit()\n</code></pre>\n\n<p>details from the <a href=\"https://docs.python.org/2/library/constants.html#exit\" rel=\"noreferrer\" title=\"exit\"><code>sys</code> module documentation</a>:</p>\n\n<blockquote>\n <p><code>sys.<strong>exit</strong>([<em>arg</em>])</code></p>\n \n <blockquote>\n <p>Exit from Python. This is implemented by raising the\n <a href=\"https://docs.python.org/2/library/exceptions.html#SystemExit\" rel=\"noreferrer\" title=\"SystemExit\"><code>SystemExit</code></a> exception, so cleanup actions specified by finally clauses\n of <a href=\"https://docs.python.org/2/reference/compound_stmts.html#try\" rel=\"noreferrer\" title=\"try\"><code>try</code></a> statements are honored, and it is possible to intercept the\n exit attempt at an outer level.</p>\n \n <p>The optional argument <em>arg</em> can be an integer giving the exit status\n (defaulting to zero), or another type of object. If it is an integer,\n zero is considered “successful termination” and any nonzero value is\n considered “abnormal termination” by shells and the like. Most systems\n require it to be in the range 0-127, and produce undefined results\n otherwise. Some systems have a convention for assigning specific\n meanings to specific exit codes, but these are generally\n underdeveloped; Unix programs generally use 2 for command line syntax\n errors and 1 for all other kind of errors. If another type of object\n is passed, None is equivalent to passing zero, and any other object is\n printed to <a href=\"https://docs.python.org/2/library/sys.html#sys.stderr\" rel=\"noreferrer\" title=\"sys.stderr\"><code>stderr</code></a> and results in an exit code of 1. In particular,\n <code>sys.exit(\"some error message\")</code> is a quick way to exit a program when\n an error occurs.</p>\n \n <p>Since <a href=\"https://docs.python.org/2/library/constants.html#exit\" rel=\"noreferrer\" title=\"exit\"><code>exit()</code></a> ultimately “only” raises an exception, it will only exit\n the process when called from the main thread, and the exception is not\n intercepted.</p>\n </blockquote>\n</blockquote>\n\n<p>Note that this is the 'nice' way to exit. @<a href=\"https://stackoverflow.com/questions/73663/terminating-a-python-script#76374\">glyphtwistedmatrix</a> below points out that if you want a 'hard exit', you can use os._exit(<em>errorcode</em>), though it's likely os-specific to some extent (it might not take an errorcode under windows, for example), and it definitely is less friendly since it doesn't let the interpreter do any cleanup before the process dies.</p>\n", | |
| "source": "so", | |
| "questionId": 73663 | |
| }, | |
| { | |
| "title": "How to find if directory exists in Python", | |
| "body": "<p>You're looking for <a href=\"http://docs.python.org/dev/library/os.path.html#os.path.isdir\"><code>os.path.isdir</code></a>, or <a href=\"http://docs.python.org/dev/library/os.path.html#os.path.exists\"><code>os.path.exists</code></a> if you don't care whether it's a file or a directory.</p>\n\n<p>Example:</p>\n\n<pre><code>import os\nprint(os.path.isdir(\"/home/el\"))\nprint(os.path.exists(\"/home/el/myfile.txt\"))\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 8933237 | |
| }, | |
| { | |
| "title": "What is a mixin, and why are they useful?", | |
| "body": "<p>A mixin is a special kind of multiple inheritance. There are two main situations where mixins are used:</p>\n\n<ol>\n<li>You want to provide a lot of optional features for a class.</li>\n<li>You want to use one particular feature in a lot of different classes.</li>\n</ol>\n\n<p>For an example of number one, consider <a href=\"http://werkzeug.pocoo.org/docs/wrappers/\" rel=\"noreferrer\">werkzeug's request and response system</a>. I can make a plain old request object by saying:</p>\n\n<pre><code>from werkzeug import BaseRequest\n\nclass Request(BaseRequest):\n pass\n</code></pre>\n\n<p>If I want to add accept header support, I would make that</p>\n\n<pre><code>from werkzeug import BaseRequest, AcceptMixin\n\nclass Request(BaseRequest, AcceptMixin):\n pass\n</code></pre>\n\n<p>If I wanted to make a request object that supports accept headers, etags, authentication, and user agent support, I could do this:</p>\n\n<pre><code>from werkzeug import BaseRequest, AcceptMixin, ETagRequestMixin, UserAgentMixin, AuthenticationMixin\n\nclass Request(BaseRequest, AcceptMixin, ETagRequestMixin, UserAgentMixin, AuthenticationMixin):\n pass\n</code></pre>\n\n<p>The difference is subtle, but in the above examples, the mixin classes weren't made to stand on their own. In more traditional multiple inheritance, the <code>AuthenticationMixin</code> (for example) would probably be something more like <code>Authenticator</code>. That is, the class would probably be designed to stand on its own.</p>\n", | |
| "source": "so", | |
| "questionId": 533631 | |
| }, | |
| { | |
| "title": "What is the Python 3 equivalent of "python -m SimpleHTTPServer"", | |
| "body": "<p>From <a href=\"https://docs.python.org/2/library/simplehttpserver.html\">the docs</a>:</p>\n\n<blockquote>\n <p>The <code>SimpleHTTPServer</code> module has been merged into <code>http.server</code> in Python 3.0. The 2to3 tool will automatically adapt imports when converting your sources to 3.0.</p>\n</blockquote>\n\n<p>So, your command is <code>python3 -m http.server</code>.</p>\n", | |
| "source": "so", | |
| "questionId": 7943751 | |
| }, | |
| { | |
| "title": "Map two lists into a dictionary in Python", | |
| "body": "<p>Like this:</p>\n\n<pre><code>>>> keys = ['a', 'b', 'c']\n>>> values = [1, 2, 3]\n>>> dictionary = dict(zip(keys, values))\n>>> print(dictionary)\n{'a': 1, 'b': 2, 'c': 3}\n</code></pre>\n\n<p>Voila :-) The pairwise <code>dict</code> constructor and <code>zip</code> function are awesomely useful: <a href=\"https://docs.python.org/3/library/functions.html#func-dict\" rel=\"noreferrer\">https://docs.python.org/3/library/functions.html#func-dict</a></p>\n", | |
| "source": "so", | |
| "questionId": 209840 | |
| }, | |
| { | |
| "title": "Import a module from a relative path", | |
| "body": "<p>Assuming that both your directories are real Python packages (do have the <code>__init__.py</code> file inside them), here is a safe solution for inclusion of modules relatively to the location of the script.</p>\n\n<p>I assume that you want to do this, because you need to include a set of modules with your script. I use this in production in several products and works in many special scenarios like: scripts called from another directory or executed with python execute instead of opening a new interpreter.</p>\n\n<pre><code> import os, sys, inspect\n # realpath() will make your script run, even if you symlink it :)\n cmd_folder = os.path.realpath(os.path.abspath(os.path.split(inspect.getfile( inspect.currentframe() ))[0]))\n if cmd_folder not in sys.path:\n sys.path.insert(0, cmd_folder)\n\n # Use this if you want to include modules from a subfolder\n cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],\"subfolder\")))\n if cmd_subfolder not in sys.path:\n sys.path.insert(0, cmd_subfolder)\n\n # Info:\n # cmd_folder = os.path.dirname(os.path.abspath(__file__)) # DO NOT USE __file__ !!!\n # __file__ fails if the script is called in different ways on Windows.\n # __file__ fails if someone does os.chdir() before.\n # sys.argv[0] also fails, because it doesn't not always contains the path.\n</code></pre>\n\n<p>As a bonus, this approach does let you force Python to use your module instead of the ones installed on the system.</p>\n\n<p>Warning! I don't really know what is happening when current module is inside an <code>egg</code> file. It probably fails too.</p>\n", | |
| "source": "so", | |
| "questionId": 279237 | |
| }, | |
| { | |
| "title": "Pythonic way to create a long multi-line string", | |
| "body": "<p>Are you talking about multi-line strings? Easy, use triple quotes to start and end them.</p>\n\n<pre><code>s = \"\"\" this is a very\n long string if I had the\n energy to type more and more ...\"\"\"\n</code></pre>\n\n<p>You can use single quotes too (3 of them of course at start and end) and treat the resulting string <code>s</code> just like any other string.</p>\n\n<p><strong>NOTE</strong>: Just as with any string, anything between the starting and ending quotes becomes part of the string, so this example has a leading blank (as pointed out by @root45). This string will also contain both blanks and newlines.</p>\n\n<p>I.e.,:</p>\n\n<pre><code>' this is a very\\n long string if I had the\\n energy to type more and more ...'\n</code></pre>\n\n<p>Finally, one can also construct long lines in Python like this:</p>\n\n<pre><code> s = (\"this is a very\"\n \"long string too\"\n \"for sure ...\"\n )\n</code></pre>\n\n<p>which will <strong>not</strong> include any extra blanks or newlines (this is a deliberate example showing what the effect of skipping blanks will result in):</p>\n\n<pre><code>'this is a verylong string toofor sure ...'\n</code></pre>\n\n<p>No commas required, simply place the strings to be joined together into a pair of parenthesis and be sure to account for any needed blanks and newlines.</p>\n", | |
| "source": "so", | |
| "questionId": 10660435 | |
| }, | |
| { | |
| "title": "Save plot to image file instead of displaying it using Matplotlib", | |
| "body": "<p>While the question has been answered, I'd like to add some useful tips when using <a href=\"http://matplotlib.sourceforge.net/api/pyplot_api.html#matplotlib.pyplot.savefig\" rel=\"noreferrer\">savefig</a>. The file format can be specified by the extension:</p>\n\n<pre><code>savefig('foo.png')\nsavefig('foo.pdf')\n</code></pre>\n\n<p>Will give a rasterized or vectorized output respectively, both which could be useful. In addition, you'll find that <code>pylab</code> leaves a generous, often undesirable, whitespace around the image. Remove it with:</p>\n\n<pre><code>savefig('foo.png', bbox_inches='tight')\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 9622163 | |
| }, | |
| { | |
| "title": ""Large data" work flows using pandas", | |
| "body": "<p>I routinely use tens of gigabytes of data in just this fashion\ne.g. I have tables on disk that I read via queries, create data and append back.</p>\n\n<p>It's worth reading <a href=\"http://pandas-docs.github.io/pandas-docs-travis/io.html#hdf5-pytables\">the docs</a> and <a href=\"https://groups.google.com/forum/m/?fromgroups#!topic/pydata/cmw1F3OFJSc\">late in this thread</a> for several suggestions for how to store your data.</p>\n\n<p>Details which will affect how you store your data, like:<br>\n<em>Give as much detail as you can; and I can help you develop a structure.</em></p>\n\n<ol>\n<li>Size of data, # of rows, columns, types of columns; are you appending\nrows, or just columns? </li>\n<li>What will typical operations look like. E.g. do a query on columns to select a bunch of rows and specific columns, then do an operation (in-memory), create new columns, save these.<br>\n(Giving a toy example could enable us to offer more specific recommendations.)</li>\n<li>After that processing, then what do you do? Is step 2 ad hoc, or repeatable?</li>\n<li>Input flat files: how many, rough total size in Gb. How are these organized e.g. by records? Does each one contains different fields, or do they have some records per file with all of the fields in each file?</li>\n<li>Do you ever select subsets of rows (records) based on criteria (e.g. select the rows with field A > 5)? and then do something, or do you just select fields A, B, C with all of the records (and then do something)?</li>\n<li>Do you 'work on' all of your columns (in groups), or are there a good proportion that you may only use for reports (e.g. you want to keep the data around, but don't need to pull in that column explicity until final results time)?</li>\n</ol>\n\n<h2>Solution</h2>\n\n<p><em>Ensure you have <a href=\"http://pandas.pydata.org/getpandas.html\">pandas at least <code>0.10.1</code></a> installed.</em></p>\n\n<p>Read <a href=\"http://pandas-docs.github.io/pandas-docs-travis/io.html#iterating-through-files-chunk-by-chunk\">iterating files chunk-by-chunk</a> and <a href=\"http://pandas-docs.github.io/pandas-docs-travis/io.html#multiple-table-queries\">multiple table queries</a>.</p>\n\n<p>Since pytables is optimized to operate on row-wise (which is what you query on), we will create a table for each group of fields. This way it's easy to select a small group of fields (which will work with a big table, but it's more efficient to do it this way... I think I may be able to fix this limitation in the future... this is more intuitive anyhow):<br>\n(The following is pseudocode.)</p>\n\n<pre><code>import numpy as np\nimport pandas as pd\n\n# create a store\nstore = pd.HDFStore('mystore.h5')\n\n# this is the key to your storage:\n# this maps your fields to a specific group, and defines \n# what you want to have as data_columns.\n# you might want to create a nice class wrapping this\n# (as you will want to have this map and its inversion) \ngroup_map = dict(\n A = dict(fields = ['field_1','field_2',.....], dc = ['field_1',....,'field_5']),\n B = dict(fields = ['field_10',...... ], dc = ['field_10']),\n .....\n REPORTING_ONLY = dict(fields = ['field_1000','field_1001',...], dc = []),\n\n)\n\ngroup_map_inverted = dict()\nfor g, v in group_map.items():\n group_map_inverted.update(dict([ (f,g) for f in v['fields'] ]))\n</code></pre>\n\n<p>Reading in the files and creating the storage (essentially doing what <code>append_to_multiple</code> does):</p>\n\n<pre><code>for f in files:\n # read in the file, additional options hmay be necessary here\n # the chunksize is not strictly necessary, you may be able to slurp each \n # file into memory in which case just eliminate this part of the loop \n # (you can also change chunksize if necessary)\n for chunk in pd.read_table(f, chunksize=50000):\n # we are going to append to each table by group\n # we are not going to create indexes at this time\n # but we *ARE* going to create (some) data_columns\n\n # figure out the field groupings\n for g, v in group_map.items():\n # create the frame for this group\n frame = chunk.reindex(columns = v['fields'], copy = False) \n\n # append it\n store.append(g, frame, index=False, data_columns = v['dc'])\n</code></pre>\n\n<p>Now you have all of the tables in the file (actually you could store them in separate files if you wish, you would prob have to add the filename to the group_map, but probably this isn't necessary).</p>\n\n<p>This is how you get columns and create new ones:</p>\n\n<pre><code>frame = store.select(group_that_I_want)\n# you can optionally specify:\n# columns = a list of the columns IN THAT GROUP (if you wanted to\n# select only say 3 out of the 20 columns in this sub-table)\n# and a where clause if you want a subset of the rows\n\n# do calculations on this frame\nnew_frame = cool_function_on_frame(frame)\n\n# to 'add columns', create a new group (you probably want to\n# limit the columns in this new_group to be only NEW ones\n# (e.g. so you don't overlap from the other tables)\n# add this info to the group_map\nstore.append(new_group, new_frame.reindex(columns = new_columns_created, copy = False), data_columns = new_columns_created)\n</code></pre>\n\n<p>When you are ready for post_processing:</p>\n\n<pre><code># This may be a bit tricky; and depends what you are actually doing.\n# I may need to modify this function to be a bit more general:\nreport_data = store.select_as_multiple([groups_1,groups_2,.....], where =['field_1>0', 'field_1000=foo'], selector = group_1)\n</code></pre>\n\n<p>About data_columns, you don't actually need to define <strong>ANY</strong> data_columns; they allow you to sub-select rows based on the column. E.g. something like:</p>\n\n<pre><code>store.select(group, where = ['field_1000=foo', 'field_1001>0'])\n</code></pre>\n\n<p>They may be most interesting to you in the final report generation stage (essentially a data column is segregated from other columns, which might impact efficiency somewhat if you define a lot).</p>\n\n<p>You also might want to:</p>\n\n<ul>\n<li>create a function which takes a list of fields, looks up the groups in the groups_map, then selects these and concatenates the results so you get the resulting frame (this is essentially what select_as_multiple does). <em>This way the structure would be pretty transparent to you.</em></li>\n<li>indexes on certain data columns (makes row-subsetting much faster).</li>\n<li>enable compression.</li>\n</ul>\n\n<p>Let me know when you have questions!</p>\n", | |
| "source": "so", | |
| "questionId": 14262433 | |
| }, | |
| { | |
| "title": "Python class inherits object", | |
| "body": "<blockquote>\n <h3>Is there any reason for a class declaration to inherit from <code>object</code>?</h3>\n</blockquote>\n\n<p>tl;dr: In Python 3, apart from compatibility between Python 2 and 3, <em>no reason</em>. In Python 2, <em>many reasons</em>. </p>\n\n<hr>\n\n<h3>Python 2.x story:</h3>\n\n<p>In Python 2.x (from 2.2 onwards) there's two styles of classes depending on the presence or absence of <code>object</code> as a base-class:</p>\n\n<ol>\n<li><p><strong>\"classic\" style</strong> classes: they don't have <code>object</code> as a base class:</p>\n\n<pre><code>>>> class ClassicSpam: # no base class\n... pass\n>>> ClassicSpam.__bases__\n()\n</code></pre></li>\n<li><p><strong>\"new\" style</strong> classes: they have, directly <em>or indirectly</em> (e.g inherit from a <a href=\"https://docs.python.org/3/library/stdtypes.html\" rel=\"noreferrer\">built-in type</a>), <code>object</code> as a base class:</p>\n\n<pre><code>>>> class NewSpam(object): # directly inherit from object\n... pass\n>>> NewSpam.__bases__\n(<type 'object'>,)\n>>> class IntSpam(int): # indirectly inherit from object...\n... pass\n>>> IntSpam.__bases__\n(<type 'int'>,) \n>>> IntSpam.__bases__[0].__bases__ # ... because int inherits from object \n(<type 'object'>,)\n</code></pre></li>\n</ol>\n\n<p>Without a doubt, when writing a class you'll <em>always</em> want to go for new-style classes. The perks of doing so are numerous, to list some of them:</p>\n\n<ul>\n<li><p><a href=\"https://docs.python.org/3/howto/descriptor.html\" rel=\"noreferrer\">Support for descriptors</a>. Specifically, the following constructs are made possible with descriptors: </p>\n\n<ol>\n<li><a href=\"https://docs.python.org/3/library/functions.html#classmethod\" rel=\"noreferrer\"><code>classmethod</code></a>: A method that receives the class as an implicit argument instead of the instance.</li>\n<li><a href=\"https://docs.python.org/3/library/functions.html#staticmethod\" rel=\"noreferrer\"><code>staticmethod</code></a>: A method that does not receive the implicit argument <code>self</code> as a first argument.</li>\n<li>properties with <a href=\"https://docs.python.org/3/library/functions.html#property\" rel=\"noreferrer\"><code>property</code></a>: Create functions for managing the getting, setting and deleting of an attribute. </li>\n<li><a href=\"https://docs.python.org/3/reference/datamodel.html#slots\" rel=\"noreferrer\"><code>__slots__</code></a>: Saves memory consumptions of a class and also results in faster attribute access. Of course, it does <a href=\"https://docs.python.org/3/reference/datamodel.html#notes-on-using-slots\" rel=\"noreferrer\">impose limitations</a>.</li>\n</ol></li>\n<li><p>The <a href=\"https://docs.python.org/3/reference/datamodel.html#object.__new__\" rel=\"noreferrer\"><code>__new__</code></a> static method: lets you customize how new class instances are created. </p></li>\n<li><p><a href=\"https://www.python.org/download/releases/2.3/mro/\" rel=\"noreferrer\">Method resolution order (MRO)</a>: in what order the base classes of a class will be searched when trying to resolve which method to call. </p></li>\n<li><p>Related to MRO, <a href=\"https://docs.python.org/3/library/functions.html#super\" rel=\"noreferrer\"><code>super</code> calls</a>. Also see, <a href=\"https://rhettinger.wordpress.com/2011/05/26/super-considered-super/\" rel=\"noreferrer\"><code>super()</code> considered super.</a></p></li>\n</ul>\n\n<p>If you don't inherit from <code>object</code>, forget these. A more exhaustive description of the previous bullet points along with other perks of \"new\" style classes can be found <a href=\"https://www.python.org/download/releases/2.2.3/descrintro/\" rel=\"noreferrer\">here</a>.</p>\n\n<p>One of the downsides of new-style classes is that the class itself is more memory demanding. Unless you're creating many class objects, though, I doubt this would be an issue and it's a negative sinking in a sea of positives.</p>\n\n<hr>\n\n<h3>Python 3.x story:</h3>\n\n<p>In Python 3, things are simplified. Only new-style classes exist (referred to plainly as classes) so, the only difference in adding <code>object</code> is requiring you to type in 8 more characters. This:</p>\n\n<pre><code>class ClassicSpam:\n pass\n</code></pre>\n\n<p>is completely equivalent (apart from their name :-) to this:</p>\n\n<pre><code>class NewSpam(object):\n pass\n</code></pre>\n\n<p>and to this:</p>\n\n<pre><code>class Spam():\n pass\n</code></pre>\n\n<p>all have <code>object</code> in their <code>__bases__</code>.</p>\n\n<pre><code>>>> [object in cls.__bases__ for cls in {Spam, NewSpam, ClassicSpam}]\n[True, True, True]\n</code></pre>\n\n<hr>\n\n<h2>So, what should you do?</h2>\n\n<p><strong>In Python 2:</strong> <em>always inherit from <code>object</code> explicitly</em>. Get the perks.</p>\n\n<p><strong>In Python 3:</strong> inherit from <code>object</code> if you are writing code that tries to be Python agnostic, that is, it needs to work both in Python 2 and in Python 3. Otherwise don't, it really makes no difference since Python inserts it for you behind the scenes.</p>\n", | |
| "source": "so", | |
| "questionId": 4015417 | |
| }, | |
| { | |
| "title": "How can I reverse a list in python?", | |
| "body": "<p>You can make use of the <a href=\"https://www.python.org/dev/peps/pep-0322/\" rel=\"noreferrer\"><code>reversed</code></a> function for this as:</p>\n\n<pre><code>>>> array=[0,10,20,40]\n>>> for i in reversed(array):\n... print(i)\n</code></pre>\n\n<p>Note that <code>reversed(...)</code> does not return a list. You can get a reversed list using <code>list(reversed(array))</code>.</p>\n", | |
| "source": "so", | |
| "questionId": 3940128 | |
| }, | |
| { | |
| "title": "How to get line count cheaply in Python?", | |
| "body": "<p>You can't get any better than that.</p>\n\n<p>After all, any solution will have to read the entire file, figure out how many <code>\\n</code> you have, and return that result.</p>\n\n<p>Do you have a better way of doing that without reading the entire file? Not sure... The best solution will always be I/O-bound, best you can do is make sure you don't use unnecessary memory, but it looks like you have that covered.</p>\n", | |
| "source": "so", | |
| "questionId": 845058 | |
| }, | |
| { | |
| "title": "Count the number occurrences of a character in a string", | |
| "body": "<blockquote>\n <p><a href=\"https://docs.python.org/2/library/stdtypes.html#str.count\" rel=\"noreferrer\">str.count(sub[, start[, end]])</a></p>\n \n <p>Return the number of non-overlapping occurrences of substring <code>sub</code> in the range <code>[start, end]</code>. Optional arguments <code>start</code> and <code>end</code> are interpreted as in slice notation.</p>\n</blockquote>\n\n<pre><code>>>> sentence = 'Mary had a little lamb'\n>>> sentence.count('a')\n4\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1155617 | |
| }, | |
| { | |
| "title": "How to remove items from a list while iterating?", | |
| "body": "<p>You can use a list comprehension to create a new list containing only the elements you don't want to remove:</p>\n\n<pre><code>somelist = [x for x in somelist if not determine(x)]\n</code></pre>\n\n<p>Or, by assigning to the slice <code>somelist[:]</code>, you can mutate the existing list to contain only the items you want:</p>\n\n<pre><code>somelist[:] = [x for x in somelist if not determine(x)]\n</code></pre>\n\n<p>This approach could be useful if there are other references to <code>somelist</code> that need to reflect the changes.</p>\n\n<p>Instead of a comprehension, you could also use <code>itertools</code>. In Python 2:</p>\n\n<pre><code>from itertools import ifilterfalse\nsomelist[:] = ifilterfalse(determine, somelist)\n</code></pre>\n\n<p>Or in Python 3:</p>\n\n<pre><code>from itertools import filterfalse\nsomelist[:] = filterfalse(determine, somelist)\n</code></pre>\n", | |
| "source": "so", | |
| "questionId": 1207406 | |
| }, | |
| { | |
| "title": "How do I create a constant in Python?", | |
| "body": "<p>No there is not. You cannot declare a variable or value as constant in Python. Just don't change it.</p>\n\n<p>If you are in a class, the equivalent would be:</p>\n\n<pre><code>class Foo(object):\n CONST_NAME = \"Name\"\n</code></pre>\n\n<p>if not, it is just</p>\n\n<pre><code>CONST_NAME = \"Name\"\n</code></pre>\n\n<p>But you might want to have a look at the code snippet <a href=\"http://code.activestate.com/recipes/65207-constants-in-python/?in=user-97991\" rel=\"noreferrer\">Constants in Python</a> by Alex Martelli.</p>\n", | |
| "source": "so", | |
| "questionId": 2682745 | |
| } | |
| ] | |
| } | |
| ] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment