Back Home

Setting up Local Environments

pip install virtualenv

to create a virtualenv envrionment

virtualenv -p /c/Python27/python <folder>

to activate it

source <folder>/bin/activate

Try/Catch

try:
    open('/path/to/does/not/exist', 'rb')
except (SystemExit, KeyboardInterrupt):
    raise
except Exception, e:
    logger.error('Failed to open file', exc_info=True)

Logging

python supports a standard logging framework:

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

Setting the logging configuraiton:

import logging
 
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
 
# create a file handler
 
handler = logging.FileHandler('hello.log')
handler.setLevel(logging.INFO)
 
# create a logging format
 
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
 
# add the handlers to the logger
 
logger.addHandler(handler)
 
logger.info('Hello baby')

Logging Config via ini File

Logging code to import config:

import logging
 
# load my module
 
import my_module
 
# load the logging configuration
 
logging.config.fileConfig('logging.ini')
 
my_module.foo()
bar = my_module.Bar()
bar.bar()

The logging.ini file can look like:

[loggers]
keys=root
 
[handlers]
keys=consoleHandler
 
[formatters]
keys=simpleFormatter
 
[logger_root]
level=DEBUG
handlers=consoleHandler
 
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)
 
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

Logging with - Loguru

Xpath with Parcel

- Xpath is supported with libxml2 which is fast but has binary dependencies that can be a problem on windows - A functional alternative is Parsel

from parsel import Selector
 
sel = Selector(text=u"""<html>
        <body>
            <h1>Hello, Parsel!</h1>
            <ul>
                <li><a href="http://example.com">Link 1</a></li>
                <li><a href="http://scrapy.org">Link 2</a></li>
            </ul
        </body>
        </html>""")
for item in sel.xpath('.//a/@href'):
    print (item.get())

will print

 
http://example.com
http://scrapy.org

Static Typing - MyPy

http://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy/

def sum_and_stringify(nums: List[int]) -> str:  
    """Adds up the numbers in a list and returns the result as a string."""
    return str(sum(nums))

Grabbing Data - a better curl - Requests

The Requests library

import requests
r = requests.get('https://github.com/timeline.json')
r.json()

a posting example:

import requests, json
url = 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere'
payload = open("request.json")
headers = {'content-type': 'application/json', 'Accept-Charset': 'UTF-8'}
r = requests.post(url, data=payload, headers=headers)4

Python Testing

Python Transpiler

  • RapydScript is a pre-compiler for JavaScript. The syntax is very similar to Python, but allows JavaScript as well. This project was written as a cleaner alternative to CoffeeScript.

Python Logging Libraries

Python Visualization

  • MATLAB like plotting framework
  • Seaborn Built on top of matplotlib

Creates more sophisticated graphs that look more professional

  • Basemap library for plotting 2D data on maps in Python

Had lots of problems with installation

  • Folium Visualize data on a Leaflet map Built-in tilesets from: OpenStreetMap, MapQuest Open, MapQuest Open Aerial, Mapbox, and Stamen, and supports custom tilesets with Mapbox or Cloudmade API keys.
  • Bokeh A Python interactive visualization library that targets modern web browsers for presentations.
  • Plot.ly Make interactive charts online from Excel or CSV data.

* https://altair-viz.github.io/ has advanced, straight forward vizualisation also sample of recopies https://github.com/piccolbo/altair_recipes

 
python.txt · Last modified: 2019/01/29 02:03 by root
 
RSS - 200 © CrosswireDigitialMedia Ltd