1、www.it-ebooks.infoExpert Python ProgrammingSecond EditionBecome an ace Python programmer by learning best coding practices and advance-level concepts with Python 3.5Micha JaworskiTarek ZiadBIRMINGHAM - MUMBAIwww.it-ebooks.infoExpert Python ProgrammingSecond EditionCopyright 2016 Packt PublishingAll
2、rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.Every effort has been made in the
3、preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or
4、 alleged to be caused directly or indirectly by this book.Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.Fi
5、rst published: September 2008Second edition: May 2016Production reference: 1160516Published by Packt Publishing Ltd.Livery Place35 Livery StreetBirmingham B3 2PB, UK.ISBN 978-1-78588-685-www.it-ebooks.infoCreditsAuthorsMicha JaworskiTarek ZiadReviewerFacundo BatistaCommissioning EditorKunal ParikhAc
6、quisition EditorMeeta RajaniTechnical EditorPankaj KadamCopy EditorLaxmi SubramanianProofreaderSafis EditingIndexerRekha NairGraphicsJason MonteiroProduction CoordinatorAparna BhagatCover WorkAparna Bhagatwww.it-ebooks.infoAbout the AuthorsMicha Jaworski has 7 years of experience in Python. He is al
7、so the creator of graceful, which is a REST framework built on top of falcon. He has been in various roles at different companies: from an ordinary full-stack developer through software architect to VP of engineering in a fast-paced start-up company. He is currently a lead backend engineer in TV Sto
8、re team at Opera Software. He is highly experienced in designing high-performance distributed services. He is also an active contributor to some of the popular Python open source projects.Tarek Ziad is an engineering manager at Mozilla, working with a team specialized in building web services in Pyt
9、hon at scale for Firefox. Hes contributed to the Python packaging effort and has worked with a lot of different Python web frameworks since Zope in the early days.Tarek has also created Afpy, the French Python User Group, and has written two books on Python in French. He has delivered numerous talks
10、 and tutorials in French at international events such as Solutions Linux, PyCon, OSCON, and EuroPython.www.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！About the ReviewerFacundo Batista is a specialist in the Python programming language, with more than 15 years of expe
11、rience with it. He is a core developer of the language, and a member by merit of the Python Software Foundation. He also received the 2009 Community Service Award for organizing PyCon Argentina and the Argentinian Python community as well as contributions to the standard library and work in translat
12、ing the Python documentation.He delivers talks in the main Python conferences in Argentina and other countries (The United States and Europe). In general, he has strong distributed collaborative experience from being involved in FLOSS development and working with people around the globe for more tha
13、n 10 years.He worked as a telecommunication engineer at Telefnica Mviles and Ericsson, and as a Python expert at Cyclelogic (developer in chief) and Canonical (senior software developer, his current position).He also loves playing tennis, and is a father of two wonderful children.www.it-ebooks.info欢
14、迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！www.PacktPeBooks, discount offers, and moreDid you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktP and as a print book customer, you are e
15、ntitled to a discount on the eBook copy. Get in touch with us at for more details.At www.PacktP, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.TMhttps:/ you need instant solution
16、s to your IT questions? PacktLib is Packts online digital book library. Here, you can search, access, and read Packts entire library of books.Why subscribe? Fully searchable across every book published by Packt Copy and paste, print, and bookmark content On demand and accessible via a web browserwww
17、.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！ i Table of ContentsPreface xiChapter 1: Current Status of Python 1Where are we now and where we are going? 2Why and how does Python change? 2Getting up to date with changes PEP documents 3Python 3 adoption at the time of w
18、riting this book 4The main differences between Python 3 and Python 2 5Why should I care? 5The main syntax differences and common pitfalls 5Syntax changes 6Changes in the standard library 7Changes in datatypes and collections 8The popular tools and techniques used for maintaining cross-version compat
19、ibility 8Not only CPython 12Why should I care? 13Stackless Python 13Jython 14IronPython 14PyPy 15Modern approaches to Python development 16Application-level isolation of Python environments 17Why isolation? 19Popular solutions 21virtualenv 21venv 23buildout 24Which one to choose? 24www.it-ebooks.inf
20、o欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents ii System-level environment isolation 25Virtual development environments using Vagrant 26Containerization versus virtualization 27Popular productivity tools 28Custom Python shells IPython, bpython, ptpython, and so on 2
21、9Setting up the PYTHONSTARTUP environment variable 30IPython 30bpython 30ptpython 31Interactive debuggers 31Useful resources 32Summary 33Chapter 2: Syntax Best Practices below the Class Level 35Pythons built-in types 36Strings and bytes 36Implementation details 38String concatenation 39Collections 4
22、0Lists and tuples 40Dictionaries 45Sets 49Beyond basic collections the collections module 50Advanced syntax 51Iterators 51The yield statement 52Decorators 56General syntax and possible implementations 57Usage and useful examples 61Context managers the with statement 68General syntax and possible imp
23、lementations 69Other syntax elements you may not know yet 72The for else statement 73Function annotations 73The general syntax 74The possible uses 74Summary 75Chapter 3: Syntax Best Practices above the Class Level 77Subclassing built-in types 78Accessing methods from superclasses 80Old-style classes
24、 and super in Python 2 82Understanding Pythons Method Resolution Order 83www.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents iii super pitfalls 87Mixing super and explicit class calls 87Heterogeneous arguments 89Best practices 90Advanced attribute access
25、 patterns 91Descriptors 92Real-life example lazily evaluated attributes 95Properties 98Slots 101Metaprogramming 102Decorators a method of metaprogramming 103Class decorators 103Using the _new_() method to override instance creation process 105Metaclasses 108The general syntax 109New Python 3 syntax
26、for metaclasses 112Metaclass usage 115Metaclass pitfalls 115Some tips on code generation 116exec, eval, and compile 117Abstract Syntax Tree 118Projects using code generation patterns 120Summary 123Chapter 4: Choosing Good Names 125PEP 8 and naming best practices 125Why and when to follow PEP 8? 126B
27、eyond PEP 8 team-specific style guidelines 126Naming styles 127Variables 127Constants 128Naming and usage 129Public and private variables 130Functions and methods 131The private controversy 132Special methods 134Arguments 134Properties 134Classes 135Modules and packages 135The naming guide 136Using
28、the has or is prefix for Boolean elements 136Using plurals for variables that are collections 136Using explicit names for dictionaries 136www.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents iv Avoiding generic names 136Avoiding existing names 138Best pra
29、ctices for arguments 138Building arguments by iterative design 139Trust the arguments and your tests 139Using *args and *kwargs magic arguments carefully 141Class names 143Module and package names 143Useful tools 144Pylint 144pep8 and flake8 146Summary 147Chapter 5: Writing a Package 149Creating a p
30、ackage 149The confusing state of Python packaging tools 150The current landscape of Python packaging thanks to PyPA 150Tool recommendations 151Project configuration 152setup.py 152setup.cfg 153MANIFEST.in 154Most important metadata 154Trove classifiers 155Common patterns 156The custom setup command
31、161Working with packages during development 161setup.py install 162Uninstalling packages 162setup.py develop or pip -e 162Namespace packages 163Why is it useful? 163PEP 420 implicit namespace packages 166Namespace packages in previous Python versions 167Uploading a package 168PyPI Python Package Ind
32、ex 169Uploading to PyPI or other package index 169.pypirc 170Source packages versus built packages 171sdist 171bdist and wheels 172Standalone executables 174When are standalone executables useful? 175www.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents v
33、Popular tools 176PyInstaller 177cx_Freeze 181py2exe and py2app 183Security of Python code in executable packages 184Making decompilation harder 185Summary 186Chapter 6: Deploying Code 187The Twelve-Factor App 188Deployment automation using Fabric 189Your own package index or index mirror 195PyPI mir
34、roring 196Deployment using a package 197Common conventions and practices 207The filesystem hierarchy 207Isolation 207Using process supervision tools 208Application code should be run in user space 210Using reverse HTTP proxies 210Reloading processes gracefully 211Code instrumentation and monitoring
35、212Logging errors sentry/raven 213Monitoring system and application metrics 215Dealing with application logs 218Basic low-level log practices 218Tools for log processing 220Summary 223Chapter 7: Python Extensions in Other Languages 225Different language means C or C+ 226How do extensions in C or C+
36、work 226Why you might want to use extensions 228Improving performance in critical code sections 228Integrating existing code written in different languages 229Integrating third-party dynamic libraries 230Creating custom datatypes 230Writing extensions 230Pure C extensions 231A closer look at Python/
37、C API 235Calling and binding conventions 240Exception handling 242Releasing GIL 244Reference counting 246www.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents vi Cython 248Cython as a source to source compiler 248Cython as a language 250Challenges 253Addit
38、ional complexity 253Debugging 254Interfacing with dynamic libraries without extensions 255ctypes 255Loading libraries 255Calling C functions using ctypes 257Passing Python functions as C callbacks 258CFFI 262Summary 263Chapter 8: Managing Code 265Version control systems 265Centralized systems 266Dis
39、tributed systems 268Distributed strategies 270Centralized or distributed? 271Use Git if you can 271Git flow and GitHub flow 272Continuous development processes 276Continuous integration 277Testing every commit 278Merge testing through CI 279Matrix testing 280Continuous delivery 280Continuous deploym
40、ent 281Popular tools for continuous integration 282Jenkins 282Buildbot 286Travis CI 288GitLab CI 290Choosing the right tool and common pitfalls 290Problem 1 too complex build strategies 291Problem 2 too long building time 291Problem 3 external job definitions 292Problem 4 lack of isolation 293Summar
41、y 294Chapter 9: Documenting Your Project 295The seven rules of technical writing 295Write in two steps 296Target the readership 297www.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents vii Use a simple style 298Limit the scope of information 299Use realist
42、ic code examples 299Use a light but sufficient approach 300Use templates 301A reStructuredText primer 301Section structure 303Lists 305Inline markup 306Literal block 306Links 307Building the documentation 308Building the portfolio 308Design 309Usage 310Operations 315Making your own portfolio 315Buil
43、ding the landscape 316Producers layout 317Consumers layout 318Documentation building and continuous integration 322Summary 323Chapter 10: Test-Driven Development 325I dont test 325Test-driven development principles 326Preventing software regression 328Improving code quality 329Providing the best dev
44、eloper documentation 329Producing robust code faster 330What kind of tests? 330Acceptance tests 330Unit tests 331Functional tests 331Integration tests 332Load and performance testing 332Code quality testing 333Python standard test tools 333unittest 333doctest 337I do test 339unittest pitfalls 339uni
45、ttest alternatives 340nose 340py.test 344www.it-ebooks.info欢迎加入非盈利Py t h o n 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents viii Testing coverage 348Fakes and mocks 351Building a fake 351Using mocks 356Testing environment and dependency compatibility 358Dependency matrix testing 358Do
46、cument-driven development 362Writing a story 362Summary 364Chapter 11: Optimization General Principles and Profiling Techniques 365The three rules of optimization 365Make it work first 366Work from the users point of view 367Keep the code readable and maintainable 367Optimization strategy 368Find an
47、other culprit 368Scale the hardware 368Writing a speed test 369Finding bottlenecks 370Profiling CPU usage 370Macro-profiling 371Micro-profiling 375Measuring Pystones 378Profiling memory usage 379How Python deals with memory 379Profiling memory 382C code memory leaks 390Profiling network usage 391Sum
48、mary 392Chapter 12: Optimization Some Powerful Techniques 393Reducing the complexity 394Cyclomatic complexity 396The big O notation 396Simplifying 399Searching in a list 399Using a set instead of a list 400Using collections 401deque 401defaultdict 403namedtuple 404www.it-ebooks.info欢迎加入非盈利Py t h o n
49、 编程学习交流QQ群783462347，群里免费提供500+本Py t h o n 书籍！Table of Contents ix Using architectural trade-offs 405Using heuristics and approximation algorithms 405Using task queues and delayed processing 406Using probabilistic data structures 410Caching 411Deterministic caching 412Nondeterministic caching 415Cache services 416Memcached 416Summary 419Chapter 13: Concurrency 421Why concurrency? 422Multithreading 423What is multithreading? 424How Python deals with threads 425When should threading be used? 426Building responsive interfaces 426Delegating work 426Multiuser applications 427An example of a threade
