Python

From 太極
Jump to navigation Jump to search

Basic

Resources

Install, setup

Multiple python versions

How to use pyenv to run multiple versions of Python on a Mac

Mac

How to set up virtual environments for Python on MacOS

IDE

Cloud Jupyter

nbdev

Emacs

Emacs Shell mode: how to send region to shell?

Cheat sheet

The Most Frequently Asked Questions About Python Programming

https://www.makeuseof.com/tag/python-programming-faq/

Running

Interactively

Use Ctrl+d to quit.

How to run a python script file

python mypython.py

Run python statements from a command line

Use -c (command) option. 

python -c "import psutil"

run python source code line by line

run python source code line by line 

python -m pdb <script.py>

Install a new module

See an example of installing HTSeq.

pip

The Python Package Index (PyPI) is the definitive list of packages (or modules)

For example, motionEye can be installed by pip install or pip2 install; see its wiki and source code on Github. 

sudo apt-get install python-pip
pip --version
pip install SomePackage
pip show --files SomePackage
pip install --upgrade SomePackage
pip install --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip 
pip install ‐‐upgrade pip  # Upgrade itself
pip uninstall SomePackage

sudo apt install python3-pip
pip3 --version

How to install pip to manage PyPI packages easily

The danger of upgrading pip

Don't use sudo + pip

https://askubuntu.com/questions/802544/is-sudo-pip-install-still-a-broken-practice

"--user" option in pip

https://github.com/pypa/pip/issues/4186 

$ pip install Pygments
...
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/Pygments-2.2.0.dist-info'
/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
$ pip install --user Pygments
Collecting Pygments
  Using cached Pygments-2.2.0-py2.py3-none-any.whl
Installing collected packages: Pygments
Successfully installed Pygments-2.2.0

virtualenv

Python “Virtual Environments” allows us to install a Python package in an isolated location, rather than installing it globally. 

# Python 2
$ sudo pip install virtualenv
$ virtualenv <DIR_NAME>
$ source <DIR_NAME>/bin/activate
(<DIR_NAME>) ~$ which python
....
$ deactivate

# Python 3, https://docs.python.org/3/tutorial/venv.html
$ python3 -m venv <DIR_NAME>
$ source <DIR_NAME>/bin/activate
(<DIR_NAME>) ~$ which python
....
$ deactivate

pipx

Pipx – Install And Run Python Applications In Isolated Environments

python setup.py

If a package has been bundled by its creator using the standard approach to bundling modules (with Python’s distutils tool), all you need to do is download the package, uncompress it and type: 

python setup.py build
sudo python setup.py install

For Python 2, the packages are installed under /usr/local/lib/python2.7/dist-packages/. 

$ ls -l /usr/local/lib/python2.7/dist-packages/
total 12
-rw-r--r-- 1 root staff  273 Jan 12 13:45 easy-install.pth
drwxr-sr-x 4 root staff 4096 Jan 12 13:45 HTSeq-0.6.1p1-py2.7-linux-x86_64.egg
drwxr-sr-x 4 root staff 4096 Jan 12 13:42 pysam-0.9.1.4-py2.7-linux-x86_64.egg

Get a list of installed modules

http://stackoverflow.com/questions/739993/how-can-i-get-a-list-of-locally-installed-python-modules 

pydoc modules

Not helpful. See the pip list command.

Check installed packages' versions

If you install packages through pip, use 

$ pip list
...
pyOpenSSL (0.13.1)
pyparsing (2.0.1)
pysam (0.10.0)
python-dateutil (1.5)
pytz (2013.7)
rudix (2016.12.13)
scipy (0.13.0b1)
setuptools (1.1.6)
singledispatch (3.4.0.3)
six (1.4.1)
tornado (4.4.2)
vboxapi (1.0)
xattr (0.6.4)
zope.interface (4.1.1)

And more information about a package by using pip show PACKAGE. 

$ pip show pysam
Name: pysam
Version: 0.10.0
Summary: pysam
Home-page: https://github.com/pysam-developers/pysam
Author: Andreas Heger
Author-email: [email protected]
License: MIT
Location: /Users/XXX/Library/Python/2.7/lib/python/site-packages
Requires:

The following method works whether the package is installed by source or binary package 

>>> import pysam
>>> print(pysam.__version__)
0.10.0
>>> print pysam.__version__
0.10.0

See http://hammelllab.labsites.cshl.edu/tetoolkit-faq/

Install a specific version of package through pip

https://stackoverflow.com/questions/5226311/installing-specific-package-versions-with-pip

For example, pysam package was actively released. But the new release (0.11.2.2) may introduce some bugs. So I have to install an older version (0.10.0 works for me on Mac El Capitan and Sierra). 

$ sudo -H pip uninstall pysam
Uninstalling pysam-0.11.2.2:
......
$ sudo -H pip install pysam==0.10.0
Collecting pysam==0.10.0
  Downloading pysam-0.10.0.tar.gz (2.3MB)
    100% |████████████████████████████████| 2.3MB 418kB/s 
Installing collected packages: pysam
  Running setup.py install for pysam ... done
Successfully installed pysam-0.10.0

warning: Please check the permissions and owner of that directory

I got this message when I use root to run the 'sudo pip install PACKAGE' command.

See

python3-pip installed but pip3 command not found?

sudo apt-get remove python3-pip; sudo apt-get install python3-pip

DeepSurv example

https://github.com/jaredleekatzman/DeepSurv 

git clone https://github.com/jaredleekatzman/DeepSurv.git
sudo cp /usr/bin/pip /usr/bin/pip.bak
sudo nano /usr/bin/pip # See https://stackoverflow.com/a/50187211 more detail

# Method 1 for Theano
sudo pip install theano
# Method 2 for Theano
pip install --user --upgrade https://github.com/Theano/Theano/archive/master.zip 

pip install --user --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip
cd DeepSurv/
pip install . --user
sudo apt install python-pytest
pip install h5py --user
sudo pip uninstall protobuf # https://stackoverflow.com/a/33623372
pip install protobuf --user
sudo apt install python-tk
py.test
============ test session starts ===========
platform linux2 -- Python 2.7.12, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /home/brb/github/DeepSurv, inifile:
collected 7 items

tests/test_deepsurv.py .......

========== 7 passed in 5.77 seconds ========

How to list all installed modules

help('modules')

Comment

  1. Use the comment symbol # for a single line
  2. Use a delimiter “”” on each end of the comment. Attention: Don't use triple-quotes

Python Comments from zentut.com.

Docstring

Try / Except

Try and Except in Python 

try:
    number = int(input("Enter a number: "))
    print(number)
except:
    print("Invalid Input")

How to find the location of installed modules

There are different ways

  1. python -v
  2. import MODULENAME
  3. help('MODULENAME')

Using this way, I find the 'RPi' module is installed under /usr/lib/python2.7/dist-packages.

if __name__ == "__main__":

How to Get the Current Directory in Python

How to Get the Current Directory in Python

Import a compiled C module

string and string operators

Reference:

  1. Python for Genomic Data Science from coursera.
  2. Python Hello World and String Manipulation
dna[0]='g' 
dna[-1]='c' (start counting from the right)
dna[-2]='g'
dna[0:3]='gat' (the end always excluded)
dna[:3]='gat'
dna[2:]='tgc'
len(dna)=6
type(dna)
print(dna)
dna.count('c')
dna.upper()
dna.find('ag')=3  (only the first occurrence of 'ag' is reported)
dna.find('17', 2) (start looking from pos 17)
dna.rfind('ag')   ( search backwards in string)
dna.islower()    (True)
dna.isupper()    (False)
dna.replace('a', 'A')
print(dna.upper().isupper())

User's input

dna=raw_input("Enter a DNA sequence: ")  # python 2
dna=input("Enter a DNA sequence: ")      # python 3

To convert a user's input (a string) to others 

int(x, [, base])
flaot(x)
str(x) #converts x to a string
str(65) # '65'

chr(x)  # converts an integer to a character
chr(65) # 'A'

Print

Why is parenthesis in print voluntary in Python 2.7?

Fancy Output

print("THE DNA's GC content is ", gc, "%") # gives too many digits following the dot
print("THE DNA's GC content is %5.3f %%" % " % gc) 
# the percent operator separating the formatting string and the value to
# replace the format placeholder
print("%d" % 10.6)  # 10
print("%e" % 10.6)  # 10.060000e+01
print("%s" % dna)   # gatagc

List

A list is an ordered set of values 

gene_expr=['gene', 5.16e-08, 0.001385, 7.33e-08]
print(gene_expr[2]
gene_expr[0]='Lif'

Slice a list (it will create a new list) 

gene_expr[-3:]  # [5.16e-08, 0.001385, 7.33e-08]
gene_expr[1:3] = [6.09e-07]

Clear the list 

gene_expr[]=[]

List functions

Size of the list 

len(gene_expr)

Delete an element 

del gene_expr[1]

Extend/append to a list 

gene_expr).extend([5.16e-08, 0.00123])

Count the number of times an element appears in a list 

print(gene_expr.count('Lif'), gene_expr.count('gene'))

Reverse all elements in a list 

gene_expr.reverse()
print(gene_expr)
help(list)

Lists as Stacks 

stack=['a', 'b', 'c', 'd']
stack.append('e')

Sorting lists 

mylist=[3, 31, 123, 1, 5]
sorted(mylist)
mylist  # not changed
mylist.sort()

mylist=['c', 'g', 'T', 'a', 'A']
mylist.sort()

Don't change an element in a string! 


motif = 'nacggggtc'
motif[0] = 'a'    # ERROR

Tuples

A tuple consists of a number of values separated by commas, and is another standard sequence data type, like strings and lists. 

t=1,2,3
t
t=(1,2,3)  # we may input tuples with or without surrounding parentheses

Sets

A set is an unordered collection with no duplicate elements. 

brca1={'DNA repair', 'zine ion binding'}
brca2={protein binding', 'H4 histone'}
brca1 | brca2
brca1 & brca2
brca1 - brca2

Dictionaries

A dictionary is an unordered set of key and value pairs, with the requirement that the keys are unique (within on dictionary). 

TF_motif = {'SP1' : 'gggcgg', 
            'C/EBP' : 'attgcgcaat',
            'ATF' : 'tgacgtca',
            'c-Myc' : 'cacgtg',
            'Oct-1' : 'atgcaaat'}
# Access
print("The recognition sequence for the ATF transcription is %s." % TF_motif['ATF']) 
# Update
TF_motif['AP-1'] = 'tgagtca'
# Delete
del TF_motif['SP1']
# Size of a list
len(TF_motif)
# Get a list of all the 'keys' in a dictionary
list(TF_motif.keys())
# Get a list of all the 'values'
list(TF_motif.values())
# sort
sorted(TF_motif.keys())
sorted(TF_motif.values())

We can retrieve data from dictionaries using the items() method. 

for name,seq in seqs.item():
    print(name, seq)

In summary, strings, lists and dictionaries are most useful data types for bioinformatics.

if statement

dna=input('Enter DNA sequence: ')
if 'n' in dna :
  nbases=dna.count('n')
  print("dna sequence has %d undefined bases " % nbases)

if condtion 1:
  do action 1
elif condition 2:
  do action 2
else:
  do action 3

Logical operators

Use and, or, not. 

dna=input('Enter DNA sequence: ')
if 'n' in dna or 'N' in dna:
    nbases=dna.count('n')+dna.count('N')
    print("dna sequence has %d undefined bases " % nbases)
else:
    print("dna sequence has no undefined bases)

Loops

while 

dna=input('Enter DNA sequence:')
pos=dna.find('gt', 0)

while pos>-1 :
    print("Donar splice site candidate at position %d" %pos)
    pos=dna.find('gt', pos+1)

for 

motifs=["attccgt", "aggggggttttttcg", "gtagc"]
for m in motifs:
    print(m, len(m))

range 

for i in range(4):
    print(i)
for i in range(1,10,2):
    print(i)

Problem: find all characters in a given protein sequence are valid amino acids. 

protein='SDVIHRYKUUPAKSHGWYVCJRSRFTWMVWWRFRSCRA'
for i in range(len(protein)):
    if protein[i] not in 'ABCDEFGHIKLMNPQRSTVWXYZ': 
        print("this is not a valid protein sequence!")
        break

continue 

protein='SDVIHRYKUUPAKSHGWYVCJRSRFTWMVWWRFRSCRA'
corrected_protein=''
for i in range(len(protein)):
    if protein[i] not in 'ABCDEFGHIKLMNPQRSTVWXYZ': 
        continue
    corrected_protein=corrected_protein+protein[i]
print("COrrected protein seq is %s" % corrected_protein)

else Statement used with loops

  • If used with a for loop, the else statement is executed when the loop has exhausted iterating the list
  • If used with a while loop, the else statement is executed when the condition becomes false
# Find all prime numbers smaller than a given integer
N=10
for y in range(2, N):
    for x in range(2, y):
        if y %x == 0:
            print(y, 'equals', x, '*', y//x)
            break
        else:
            // loop fell through without finding a factor
            print(y, 'is a prime number')

The pass statement is a placeholder 

if motif not in dna:
    pass
else:
    some_function_here()

Functions

Get modular with Python function 

def function_name(arguments) :
    function_code_block
    return output

For example, 

def gc(dna) :
    "this function computes the gc perc of a dna seq"
    nbases=dna.count('n')+dna.count('n')
    gcpercent=float(dna.count('c')+dna.count('C')+dna.count('g)
+dna.count('G'))*100.0/(len(dna)-nbases)
    return gcpercent
gc('AAAAGTNNAGTCC')
help(gc)

SyntaxError: invalid syntax

https://stackoverflow.com/a/11890194

On the Python shell add an empty line at the end of function definition. Eg 

>>> def fun(a):
...     return a+1
... 
>>> fun(9)
10
>>> exit()

On a python script 

def fun(a):
    return a+1
print fun(9)

Debug functions

https://stackoverflow.com/a/4929267

You can launch a Python program through pdb by using pdb myscript.py or python -m pdb myscript.py 

$ cat debug.py
def fun(a):
    a= a*2
    a= a*3
    return a+1
print fun(5)

$ python -m pdb debug.py
> /home/pi/Downloads/debug.py(1)<module>()
-> def fun(a):
(Pdb) b fun
Breakpoint 1 at /home/pi/Downloads/debug.py:1
(Pdb) c
> /home/pi/Downloads/debug.py(2)fun()
-> a= a*2
(Pdb) n
> /home/pi/Downloads/debug.py(3)fun()
-> a= a*3
(Pdb) 
> /home/pi/Downloads/debug.py(4)fun()
-> return a+1
(Pdb) p a
30
(Pdb) n
--Return--
> /home/pi/Downloads/debug.py(4)fun()->31
-> return a+1
(Pdb) exit

Boolean functions

Problem: checks if a given dna seq contains an in-frame stop condon 

dna=input("Enter a dna seq: ")
if (has_stop_codon(dna)) :
    print("input seq has an in frame stop codon.")
else :
    print("input seq has no in frame stop codon.")

def has_stop_codon(dna) :
    "This function checks if given dna seq has in frame stop codons."
    stop_codon_found=False
    stop_codons=['tga', 'tag', 'taa']
    for i in range(0, len(dna), 3) :
        codon=dna[i:i+3].lower()
        if codon in stop_codons : 
            stop_codon_found=True
            break
    return stop_codon_found

Function default parameter values

Suppose the has_stop_codon function also accepts a frame argument (equal to 0, 1, or 2) which specifies in what frame we want to look for stop codons. 

def has_stop_codon(dna, frame=0) :
    "This function checks if given dna seq has in frame stop codons."
    stop_codon_found=False
    stop_codons=['tga', 'tag', 'taa']
    for i in range(frame, len(dna), 3) :
        codon=dna[i:i+3].lower()
        if codon in stop_codons : 
            stop_codon_found=True
            break
    return stop_codon_found

dna="atgagcggccggct"
has_stop_codon(dna)    # False
has_stop_codon(dna, 0) # False
has_stop_codon(dna, 1) # True
has_stop_codon(frame=0, dna=dna)

More examples

Reverse complement of a dna sequence 

def reversecomplement(seq):
    """Return the reverse complement of the dna string."""
    seq = reverse_string(seq)
    seq = complement(seq)
    return seq

reversecomplement('CCGGAAGAGCTTACTTAG')

To reverse a string 

def reverse_string(seq):
    return seq[::-1]

reverse_string(dna)

Complement a DNA Sequence 

def complement(dna):
    """Return the complementary sequence string."""
    basecomplement = {'A':'T', 'C':'G', 'G':'C', 'T':'A', 
                      'N':'N', 'a':t', 'c':'g', 'g':'c', 't':'a', 'n':'n'} # dictionary
    letters = list(dna) # list comprehensions
    letters = [basecomplement[base] for base in letters]
    return ''.join(letters)

Split and Join functions

sentence="enzymes and other proteins come in many shapes"
sentence.split()  # split on all whitespaces
sentence.split('and') # use 'and' as the separator

'-'.join(['enzymes', 'and', 'other', 'proteins', 'come', 'in', 'many', 'shapes'])

Variable number of function arguments

def newfunction(fi, se, th, *rest):
  print("First: %s" % fi)
  print("Second: %s" % se)
  print("Third: %s" % th)
  print("Rest... %s" % rest)
  return

Modules and packages

Packages group multiple modules under on name, by using "dotted module names". For example, the module name A.B designates a submodule named B in a package named A. See What's the difference between a Python module and a Python package?

<dnautil.py> 

#!/usr/bin/python
"""
dnautil module contains a few useful functions for dna seq
"""
def gc(dna) :
    blah
    blah
    return gcpercent

When a module is imported, Python first searches for a built-in module with that name.

If built-in module is not found, Python then searches for a file obtained by adding the extension .py to the name of the module that it's imported:

  • in your current directory,
  • the directory where Python has been installed
  • in a path, i.e., a colon(':') separated list of file paths, stored in the environment variable PYTHONPATH.

You can use the sys.path variable from the sys built-in module to check the list of all directories where Python look for files 

import sys
sys.path

If the sys.path variable does not contains the directory where you put your module you can extend it: 

sys.path.append("/home/$USER/python")
sys.path

Using modules (from PACKAGE/DIRNAME/FILENAME import CLASS)

from math import *
print(floor(3.7))

import dnautil
dna="atgagggctaggt"
gc(dna)          # gc is not defined
dnautil.gc(dna)  # Good

Import Names from a Module 

from dnautil import *
gc(dna)          # OK

from dnautil import gc, has_stop_codon

Get modular with Python functions & Learn object-oriented programming with Python from opensource.com.

Packages & __init__.py

Each package in Python is a directory which MUST contain a special file __init__.py. This file can be empty and it indicates that the directory it contains is a Python package, so it can be imported the same way a module can be imported. https://docs.python.org/2/tutorial/modules.html

Example: suppose you have several modules dnautil.py, rnautil.py , and proteinutil.py. You want to group them in a package called "bioseq" which processes all types of biological sequences. The structure of the package: 

bioseq/
  __init__.py
  dnautil.py
  rnautil.py
  proteinutil.py
  fasta/
    __init__.py
    fastautil.py
  fastq/
    __init__.py
    fastqutil.py

Loading from packages: 

import bioseq.dnautil
bioseq.dnautil.gc(dna)

from bioseq import dnautil
dnautil.gc(dna)

from bioseq.fasta.fastautil import fastqseqread

Example

Building a Multiple Choice Quiz by freeCodeCamp.org

QuestionFile.py 

class Question:
    def __init__(self, prompt, answer):
        self.prompt = prompt
        self.answer = answer

app.py 

from QuestionFile import Question

question_prompts = [
    "What color are apples?\n(a) Red/Green\n(b) Purple\n(c) Orange\n\n",
    "What color are Bananas?\n(a) Teal\n(b) Magenta\n(c) Yellow\n\n",
    "What color are strawberries?\n(a) Yellow\n(b) Red\n(c) Blue\n\n"
]

questions = [
    Question(question_prompts[0], "a"),
    Question(question_prompts[1], "c"),
    Question(question_prompts[2], "b")
]

def run_test(question):
    score = 0
    for question in questions:
        answer = input(question.prompt)
        if answer == question.answer:
            score += 1
    print("You got " + str(score) + " /" + str(len(questions))+ " correct")

run_test(questions)

Run the program by python3 app.py

Files - Communicate with the outside

f=open('myfile', 'r') # read
f=open('myfile')
f=open('myfile', 'w') # write
f=open('myfile', 'a') # append

Take care if a file does not exists

try:
    f = open('myfile')
except IOError:
    print("the file myfile does not exist!!")

Reading

for line in f:
    print(line)

Change positions within a file object 

f.seek(0)  # go to the beginning of the file
f.read()

Read a single line 

f.seek(0)
f.readline()

Write into a file 

f=open("/home/$USER/myfile, 'a)
f.write("this is a new line")
f.close()

Read text file from a URL

import urllib.request

url = "http://textfiles.com/adventure/aencounter.txt"
file = urllib.request.urlopen(url)

for line in file:
   print(line.decode('utf-8'))

Command line arguments

Suppose we run 'python processfasta.py myfile.fa' in the command line, then 

import sys
print(sys.argv)  #  ['processfasta.py', 'myfile.fa']

More completely 

#!/usr/bin/python
"""
processfasta.py builds a dictionary with all sequences from a FASTA file.
"""

import sys
filename=sys.argv[1]

try:
  f = open(filename)
except IOError:
    print("File %s does not exist!" % filename)

Parsing command line arguments with getopt. Suppose we want to store in the dictionary the sequences bigger than a given length provided in the command line: 'processfasta.py -l 250 myfile.fa' 

#!/usr/bin/python
import sys
import getopt

def usage():
    print """
processfasta.py: reads a FASTA file and builds a 
dictionary with all sequence bigger than a given length

processfasta.py [-h] [-l <length>] <filename>

 -h           print this message
 -l <length>  filter all sequences with a length
              smaller than <length>
              (default <length>=0)
 <filename>   the file has to be in FASTA format

o, a = getopt.getopt(sys.argv[1:], '1:h')
opts = {} # empty dictionary
seqlen=0;

for k,v in o:
    opts[k] = v
if 'h' in opts.keys():  # he means the user wants help
    usage(); sys.exit()
if len(a) < 1:
    usage(); sys.exit("input fasta file is missing")
if 'l' in opts.keys():
    if opts['l'] <0 :
        print("length of seq should be positive!"); sys.exit(0);
    seqlen=opts['l']

stdin and stdout

sys.stdin.read()

sys.stdout.write("Some useful ouput.\n")

sys.stderr.write("Warning: input file was not found\n")

Call external programs

import subprocess
subprocess.call('["ls", "-l"]) # return code indicates the success or failure of the execution

subprocess.call('["tophat", "genome_mouse_idx", "PE_reads_1.fq.gz", "PE_reads_2.fq.gz"])

Exceptions

5 Python Examples to Handle Exceptions using try, except and finally

Biopython & Pubmed

  • Parsers for various bioinformatics file formats (FASTA, Genbank)
  • Access to online services like NCBI Entrez or Pubmed databases
  • Interfaces to common bioinformatics programs such as BLAST, Clustalw and others.
import Bio
print(Bio.__version__)

Running BLAST over the internet 

from Bio.Blast import NCBIWWW
fasta_string = open("myseq.fa").read()
result_handle = NCBIWWW.qblast("blastn":, "nt", fasta_string)
# blastn is the program to use
# nt is the database to search against
# default output is xml
help(NCBIWWW.qblast)

The BLAST record 

from Bio.Blast import NCBIXML
blast_record = NCBIXML.read(result_handle)

Parse BLAST output 

len(blast_record.alignments)

E_VALUE_THRESH = 0.01
for alignment in blas_record.alignments:
  for hsp in alignment.hsps:
    if hsp.expect < E_VALUE_THRESH:
      print('***Alignment***')
      print('sequence:', alignment.title)
      print('length:', alignment.length)
      print('e value:', hsp.expect)
      print(hsp.query)
      print(hsp.match)
      print(hsp.sbjct)

More help with Biopython

pubmed_parser

Parser for Pubmed Open-Access XML Subset and MEDLINE XML Dataset

pyTest

pyc file

What is the difference between .py and .pyc files? [duplicate]. I observe it can cause a problem when I want to modify a python file but it keeps using the old pyc file so my change is not used (Raspbery Pi e-ink example).

Shutdown or restart OS

Below is tested on Raspbian 

import os
os.system('sudo shutdown -h now')

Popular python libraries

20 Python libraries you can’t live without

psutil

# pip install psutil --user
for x in range(10):
    psutil.cpu_percent(interval=1)

numpy

scipy

Hypergeometric test

seaborn

matplotlib

https://matplotlib.org/users/installing.html

Installation. 

python -m pip install -U pip
python -m pip install -U matplotlib

# https://stackoverflow.com/a/50328517
sudo apt-get install python3.5-tk

Example. 

from sklearn import datasets
iris = datasets.load_iris()
import matplotlib.pyplot as plt
iris = iris.data

# Scatterplot
plt.scatter(iris[:,1], iris[:,2])
plt.show()

# Boxplot
plot.boxplot(iris[:,1])
plt.show()

# Histogram
plt.hist(iris[:,1])
plt.show()

scikit-learn

scikit-learn: Machine Learning in Python

Installation. 

pip install -U scikit-learn

Example. 

$ python
>>> from sklearn import datasets
>>> iris = datasets.load_iris()
>>> digits = datasets.load_digits()

feedparser

Never miss a Magazine article — build your own RSS notification system

Boto

A Python interface to Amazon Web Services

PIL, Pillow

plotnine

Python and R – Part 2: Visualizing Data with Plotnine

Trouble shooting

ImportError: cannot import name main when running pip

https://stackoverflow.com/a/50187211

No matching distribution found for XXX

Got an error No matching distribution found for lasagne==0.2.dev1 when I ran 'pip install .' on DeepSurv.

https://github.com/imatge-upc/saliency-salgan-2017/issues/29

Python AttributeError: 'module' object has no attribute 'SSL_ST_INIT'

See https://stackoverflow.com/a/52398193. I got this message after I ran sudo pip install --upgrade cryptography and pip show cryptography. The reason I try to upgrade cryptography is the following message 

$ pip show protobuf
/home/brb/.local/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: 
  RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
Name: protobuf
...

And OpenSSL & pyOpenSSL-0.15.1.egg-inf are under /usr/lib/python2.7/dist-packages directory on my Ubuntu 16.04.

Note the following solutions do not work 

$ sudo pip uninstall pyopenssl
$ sudo pip install pyOpenSSL==16.2.0

I always get an error message 

...
  File "/usr/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 118, in <module>
    SSL_ST_INIT = _lib.SSL_ST_INIT
AttributeError: 'module' object has no attribute 'SSL_ST_INIT'

And a quick solution is to do sudo rm -r /usr/local/lib/python2.7/dist-packages/OpenSSL. I also did sudo pip install pyopenssl but I did not follow this answer (sudo apt install --reinstall python-openssl).

/usr/bin/env: ‘python’: No such file or directory

On Ubuntu 20.04, 

sudo apt-get install python-is-python3

This solved an error when I used youtube-dl.

Projects based on python

  • pithos Pandora on linux
  • Many Raspberry Pi GPIO projects
  • GeneScissors It also requires pip and scikit-learn packages.
  • KeepNote It depends on Python 2.X, sqlite and PyGTK.
  • Zim It depends on Python, Gtk and the python-gtk bindings.
  • Cherrytree It depends on Python2, Python-gtk2, Python-gtksourceview2, p7zip-full, python-enchant and python-dbus.

GUI programming

New book: Create Graphical User Interfaces with Python

Qt for GUI development

Python 3

pip3

Use pip3 instead of pip for Python 3. For example, 

pip3 install --upgrade pip

pip3 install -U scikit-learn

pip3 install -U matplotlib

http.server

Edit Files With Workspaces. The 'http.server' module is contained in python3. 

cd ~/website
python3 -m http.server

R and Python: reticulate package

See R's reticulate interface to Python.

Conda, Anaconda, miniconda

conda --version

# Manage environment
conda info --envs  # see a list of environments. 
                   # The active environment is the one with an asterisk (*)
# create a new environment
conda create --name myenv
# remove an environment
conda remove --name myenv --all

# Manage Python
conda create --name snakes python=3.5
conda activate snowflakes # activate
conda info --envs
python --version
conda activate  # Change your current environment back to the default (base)
conda deactivate # exit any python virtualenv

# Managing packages
conda search beautifulsoup4
conda install beautifulsoup4
conda list

# Updating Anaconda or Miniconda
conda update conda

Run R with Jupyter notebook

Example 1: GEO2RNAseq

GEO2RNAseq

Web framework

Flask

Django

Games

Simulate gravity in your Python game

Retrieved from "https://wiki.taichimd.us/index.php?title=Python&oldid=27111"