Your rooms Logout
Authors Sign In/Up Select UK
FormatsDate PublishedPricePages
Paperback 2018-12-31 $34.95 240
Kindle Edition 2018-12-31 225

Have you read this book?
Join the discussion!

Serious Python

By
Published by No Starch Press on 2018-12-31
Paperback: $34.95
COMPUTERS / Programming Languages, COMPUTERS / Programming / Functional


An indispensable collection of practical tips and real-world advice for tackling common Python problems and taking your code to the next level. Features interviews with high-profile Python developers who share their tips, tricks, best practices, and real-world advice gleaned from years of experience.

Sharpen your Python skills as you dive deep into the Python programming language with Serious Python. You’ll cover a range of advanced topics like multithreading and memorization, get advice from experts on things like designing APIs and dealing with databases, and learn Python internals to help you gain a deeper understanding of the language itself. Written for developers and experienced programmers, Serious Python brings together over 15 years of Python experience to teach you how to avoid common mistakes, write code more efficiently, and build better programs in less time.

As you make your way through the book’s extensive tutorials, you’ll learn how to start a project and tackle topics like versioning, layouts, coding style, and automated checks. You’ll learn how to package your software for distribution, optimize performance, use the right data structures, define functions efficiently, pick the right libraries, build future-proof programs, and optimize your programs down to the bytecode. You’ll also learn how to:

– Make and use effective decorators and methods, including abstract, static, and class methods
– Employ Python for functional programming using generators, pure functions, and functional functions
– Extend flake8 to work with the abstract syntax tree (AST) to introduce more sophisticated automatic checks into your programs
– Apply dynamic performance analysis to identify bottlenecks in your code
– Work with relational databases and effectively manage and stream data with PostgreSQL

If you’ve been looking for a way to take your Python skills from good to great, Serious Python will help you get there. Learn from the experts and get seriously good at Python with Serious Python!


(Paperback, 2018-12-31)
Embed ⇩


ASIN: 1593278780
ISBN: 9781593278786
EAN: 9781593278786

SEEN A REVIEW OR FEATURE FOR THIS BOOK? Tell us!

HAVE YOU READ SERIOUS PYTHON? WHAT DID YOU THINK OF IT?

Book cover For novels: minor spoilers are fine, and kind of necessary in order to discuss the book; but do avoid huge spoilers like giving away the ending!
Authors are warmly invited to dive into the conversation.

Read a preview from Serious Python

3D preview available at the top of this page...

Serious Python

- B e l t A d v i c e o n D e p l o y m e n t , S c a l a b i l i t y , T e s t i n g , a n d M o r e by Julien Danjou San Francisco

Serious Python. Copyright ? 2019 by Julien Danjou. All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-10: 1-59327-878-0 ISBN-13: 978-1-59327-878-6 Publisher: William Pollock Production Editor: Laurel Chun Cover Illustration: Josh Ellingson Interior Design: Octopod Studios Developmental Editors: Liz Chadwick with Ellie Bru Technical Reviewer: Mike Driscoll Copyeditor: Paula L. Fleming Compositor: Laurel Chun Proofreader: James Fraleigh For information on distribution, translations, or bulk sales, please contact No Starch Press, Inc. directly: No Starch Press, Inc. 245 8th Street, San Francisco, CA 94103 phone: 1.415.863.9900; info@nostarch.com www.nostarch.com Library of Congress Cataloging-in-Publication Data Names: Danjou, Julien, author. Title: Serious Python : black-belt advice on deployment, scalability, testing, and more / Julien Danjou. Description: San Francisco, CA : No Starch Press, Inc., [2019]. Identifiers: LCCN 2018042631 (print) | LCCN 2018050473 (ebook) | ISBN 9781593278793 (epub) | ISBN 1593278799 (epub) | ISBN 9781593278786 (print) | ISBN 1593278780 (print) | ISBN 9781593278793 (ebook) | ISBN 1593278799 (ebook) Subjects: LCSH: Python (Computer program language) Classification: LCC QA76.73.P98 (ebook) | LCC QA76.73.P98 D36 2019 (print) | DDC 005.13/3--dc23 LC record available at https://lccn.loc.gov/2018042631 No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The information in this book is distributed on an 'As Is? basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it.

About the Author Julien Danjou has been a free software hacker for close to twenty years and has been developing software with Python for twelve years. He currently works as Project Team Leader for the distributed cloud platform OpenStack, which has the largest existing open-source Python codebase at? 2.5 million lines of Python. Before building clouds, Julien created the awesome window manager and contributed to various software such as Debian and GNU Emacs. About the Technical Reviewer Mike Driscoll has been programming with Python for more than a decade. He has been writing about Python on his blog, The Mouse vs. The Python, for many years. Mike is the author of several Python books including Python 101, Python Interviews, and ReportLab: PDF Processing with Python. You can find Mike on Twitter or GitHub via his handle: @driscollis.

Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1: Starting Your Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Chapter 2: Modules, Libraries, and Frameworks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Chapter 3: Documentation and Good API Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Chapter 4: Handling Timestamps and Time Zones. . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Chapter 5: Distributing Your Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Chapter 6: Unit Testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Chapter 7: Methods and Decorators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Chapter 8: Functional Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Chapter 9: The Abstract Syntax Tree, Hy, and Lisp-like Attributes . . . . . . . . . . . . . . . . 135 Chapter 10: Performances and Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Chapter 11: Scaling and Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Chapter 12: Managing Relational Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Chapter 13: Write Less, Code More. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Acknowledgments xv Introduction 1 Who Should Read This Book and Why. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 About This Book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 Starting Your Project 5 Versions of Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Laying Out Your Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 What to Do. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 What Not to Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Version Numbering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Coding Style and Automated Checks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Tools to Catch Style Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Tools to Catch Coding Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Joshua Harlow on Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2 Modules, Libraries, and Frameworks 15 The Import System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 The sys Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Import Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Custom Importers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Meta Path Finders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Useful Standard Libraries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 External Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 The External Libraries Safety Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Protecting Your Code with an API Wrapper . . . . . . . . . . . . . . . . . . . . . . . . 23 Package Installation: Getting More from pip. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Using and Choosing Frameworks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Doug Hellmann, Python Core Developer, on Python Libraries. . . . . . . . . . . . . . . . . . . . 27 3 Documentation and Good API Practice 33 Documenting with Sphinx. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Getting Started with Sphinx and reST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Sphinx Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Writing a Sphinx Extension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Managing Changes to Your APIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Numbering API Versions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

xii''? Contents in Detail Documenting Your API Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Marking Deprecated Functions with the warnings Module. . . . . . . . . . . . . . . 43 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Christophe de Vienne on Developing APIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4 Handling Timestamps and Time Zones 49 The Problem of Missing Time Zones. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Building Default datetime Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Time Zone'Aware Timestamps with dateutil. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Serializing Time Zone'Aware datetime Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Solving Ambiguous Times. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5 Distributing Your Software 57 A Bit of setup.py History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Packaging with setup.cfg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 The Wheel Format Distribution Standard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Sharing Your Work with the World. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Entry Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Visualizing Entry Points. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Using Console Scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Using Plugins and Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Nick Coghlan on Packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 6 Unit Testing 75 The Basics of Testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Some Simple Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Skipping Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Running Particular Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Running Tests in Parallel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Creating Objects Used in Tests with Fixtures. . . . . . . . . . . . . . . . . . . . . . . . 81 Running Test Scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 Controlled Tests Using Mocking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Revealing Untested Code with coverage. . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Virtual Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Setting Up a Virtual Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Using virtualenv with tox. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Re-creating an Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 Using Different Python Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Integrating Other Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Testing Policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Robert Collins on Testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Contents in Detail''? xiii 7 Methods and Decorators 99 Decorators and When to Use Them. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Creating Decorators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Writing Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Stacking Decorators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Writing Class Decorators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 How Methods Work in Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Static Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Class Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Abstract Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Mixing Static, Class, and Abstract Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Putting Implementations in Abstract Methods. . . . . . . . . . . . . . . . . . . . . . . 114 The Truth About super. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 8 Functional Programming 119 Creating Pure Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Generators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Creating a Generator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Returning and Passing Values with yield. . . . . . . . . . . . . . . . . . . . . . . . . . 123 Inspecting Generators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 List Comprehensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Functional Functions Functioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Applying Functions to Items with map() . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Filtering Lists with filter() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Getting Indexes with enumerate(). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Sorting a List with sorted(). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Finding Items That Satisfy Conditions with any() and all() . . . . . . . . . . . . . . 128 Combining Lists with zip(). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 A Common Problem Solved. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Useful itertools Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 9 The Abstract Syntax Tree, Hy, and Lisp-like Attributes 135 Looking at the AST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Writing a Program Using the AST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 The AST Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Walking Through an AST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Extending flake8 with AST Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Writing the Class. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Ignoring Irrelevant Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Checking for the Correct Decorator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Looking for self . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 A Quick Introduction to Hy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Paul Tagliamonte on the AST and Hy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

10 Performances and Optimizations 151 Data Structures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Understanding Behavior Through Profiling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 cProfile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Disassembling with the dis Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Defining Functions Efficiently . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Ordered Lists and bisect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 namedtuple and Slots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 Memoization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Faster Python with PyPy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Achieving Zero Copy with the Buffer Protocol. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Victor Stinner on Optimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 11 Scaling and Architecture 177 Multithreading in Python and Its Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 Multiprocessing vs. Multithreading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Event-Driven Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Other Options and asyncio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Service-Oriented Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Interprocess Communication with ZeroMQ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 12 Managing Relational Databases 187 RDBMSs, ORMs, and When to Use Them. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Database Backends. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Streaming Data with Flask and PostgreSQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Writing the Data-Streaming Application. . . . . . . . . . . . . . . . . . . . . . . . . . 191 Building the Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Dimitri Fontaine on Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 13 Write Less, Code More 201 Using six for Python 2 and 3 Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Strings and Unicode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Handling Python Modules Moves. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 The modernize Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Using Python Like Lisp to Make a Single Dispatcher. . . . . . . . . . . . . . . . . . . . . . . . . . 203 Creating Generic Methods in Lisp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Generic Methods with Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 Context Managers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Less Boilerplate with attr. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Index 215

Acknowledgments Writing this first book has been a tremendous effort. Looking back, I had no clue how crazy this journey would be but also no idea how fulfilling it would turn out to be. They say that if you want to go fast you should go alone, but that if you want to go far you should go together. This is the fourth edition of the original book I wrote, and I would not have made it here without the people who helped along the way. This is a team effort and I would like to thank everyone who participated. Most of the interviewees gave me their time and trust without a second? thought, and I owe a lot of what we teach in this book to them: Doug Hellmann for his great advice about building libraries, Joshua Harlow? for his good humor and knowledge about distributed systems, Christophe de? Vienne for his experience in building frameworks, Victor Stinner for his incredible CPython knowledge, Dimitri Fontaine for his database wisdom, Robert Collins for messing up with testing, Nick Coghlan for his work in getting Python into better shape, and Paul Tagliamonte for his amazing hacker spirit.

xvi''? Acknowledgments Thanks to the No Starch crew for working with me on bringing this book to a brand new level ? especially to Liz Chadwick for her editing skills, Laurel Chun for keeping me on track, and Mike Driscoll for his technical insight. My gratitude also goes to the free software communities who shared their knowledge and helped me grow, especially to the Python community which always has been welcoming and enthusiastic.

Introduc t ion If you're reading this, the odds are good you've been working with Python for some time already. Maybe you learned it using some tutorials, delved into some existing programs, or started from scratch. Whatever the case, you've hacked your way into learning it. That's exactly how I got familiar with Python up until I started working on big open source projects 10 years ago. It is easy to think that you know and understand Python once you've written your first program. The language is that simple to grasp. However, it takes years to master it and to develop a deep comprehension of its advantages and shortcomings. When I started Python, I built my own Python libraries and applications on a 'garage project? scale. Things changed once I started working with hundreds of developers on software that thousands of users rely on. For example, the OpenStack platform'a project I contribute to'represents over 9 million lines of Python code, which collectively needs to be concise,

2''? Introduction efficient, and scalable to the needs of whatever cloud computing application its users require. When you have a project of this size, things like testing and documentation absolutely require automation, or else they won't get done at all. I thought I knew a lot about Python before working on projects of this scale'a scale I could hardly imagine when I started out'but I've learned a lot more. I've also had the opportunity to meet some of the best Python hackers in the industry and learn from them. They've taught me everything from general architecture and design principles to various helpful tips and tricks. Through this book, I hope to share the most important things I've learned so that you can build better Python programs'and build them more efficiently, too! The first version of this book, The Hacker's Guide to Python, came out in 2014. Now Serious Python is the fourth edition, with updated and entirely new contents. I hope you enjoy it! Who Should Read This Book and Why This book is intended for Python coders and developers who want to take their Python skills to the next level. In it, you'll find methods and advice that will help you get the most out of Python and build future-proof programs. If you're already working on a project, you'll be able to apply the techniques discussed right away to improve your current code. If you're starting your first project, you'll be able to create a blueprint with the best practice. I'll introduce you to some Python internals to give you a better understanding of how to write efficient code. You will gain a greater insight into the inner workings of the language that will help you understand problems or inefficiencies. The book also provides applicable battle-tested solutions to problems such as testing, porting, and scaling Python code, applications, and libraries. This will help you avoid making the mistakes that others have made and discover strategies that will help you maintain your software in the long run. About This Book This book is not necessarily designed to be read from front to back. You should feel free to skip to sections that interest you or are relevant to your work. Throughout the book, you'll find a wide range of advice and practical tips. Here's a quick breakdown of what each chapter contains. Chapter 1 provides guidance about what to consider before you undertake a project, with advice on structuring your project, numbering versions, setting up automated error checking, and more. At the end there's an interview with Joshua Harlow. Chapter 2 introduces Python modules, libraries, and frameworks and talks a little about how they work under the hood. You'll find guidance on

In this first chapter, we'll look at a few aspects of starting a project and what you should think about before you begin, such as which Python version to use, how to structure your modules, how to effectively number software versions, and how to ensure best coding practices with automatic error checking. Versions of Python Before beginning a project, you'll need to decide what version(s) of Python it will support. This is not as simple a decision as it may seem. It's no secret that Python supports several versions at the same time. Each minor version of the interpreter gets bug-fix support for 18 months and security support for 5 years. For example, Python? 3.7, released on

Starting Your Project''? 7 Laying Out Your Project Starting a new project is always a bit of a puzzle. You can't be sure how your project will be structured, so you might not know how to organize your files. However, once you have a proper understanding of best practices, you'll understand which basic structure to start with. Here I'll give some tips on dos and don'ts for laying out your project. What to Do First, consider your project structure, which should be fairly simple. Use packages and hierarchy wisely: a deep hierarchy can be a nightmare to navigate, while a flat hierarchy tends to become bloated. Then, avoid making the common mistake of storing unit tests outside the package directory. These tests should definitely be included in a subpackage of your software so that they aren't automatically installed as a tests top-level module by setuptools (or some other packaging library) by accident. By placing them in a subpackage, you ensure they can be installed and eventually used by other packages so users can build their own unit tests. Figure 1-2 illustrates what a standard file hierarchy should look like. foobar foobar setup.py setup.cfg README.rst image.png docs conf.py quickstart.rst index.rst __init__.py cli.py storage.py tests data __init__.py test_storage.py test_cli.py Figure 1-2: Standard package directory The standard name for a Python installation script is setup.py. It comes with its companion setup.cfg, which should contain the installation script configuration details. When run, setup.py will install your package using the Python distribution utilities. You can also provide important information to users in README.rst (or README.txt, or whatever filename suits your fancy). Finally, the docs directory should contain the package's documentation in reStructuredText format, which will be consumed by Sphinx (see Chapter 3).

8''? Chapter 1 Packages will often have to provide extra data for the software to use, such as images, shell scripts, and so forth. Unfortunately, there's no universally accepted standard for where these files should be stored, so you should just put them wherever makes the most sense for your project depending on their functions. For example, web application templates could go in a templates directory in your package root directory. The following top-level directories also frequently appear: ? etc for sample configuration files ? tools for shell scripts or related tools ? bin for binary scripts you've written that will be installed by setup.py What Not to Do There is a particular design issue that I often encounter in project structures that have not been fully thought out: some developers will create files or modules based on the type of code they will store. For example, they might create functions.py or exceptions.py files. This is a terrible approach and doesn't help any developer when navigating the code. When reading a codebase, the developer expects a functional area of a program to be confined in a particular file. The code organization doesn't benefit from this approach, which forces readers to jump between files for no good reason. Organize your code based on features, not on types. It is also a bad idea to create a module directory that contains only an __init__.py file, because it's unnecessary nesting. For example, you shouldn't create a directory named hooks with a single file named hooks/__init__.py in it, where hooks.py would have been enough. If you create a directory, it should contain several other Python files that belong to the category the directory represents. Building a deep hierarchy unnecessarily is confusing. You should also be very careful about the code that you put in the __init__.py file. This file will be called and executed the first time that a module contained in the directory is loaded. Placing the wrong things in your __init__.py can have unwanted side effects. In fact, __init__.py files should be empty most of the time, unless you know what you're doing. Don't try to remove __init__.py files altogether though, or you won't be able to import your Python module at all: Python requires an __init__.py file to be present for the directory to be considered a submodule. Version Numbering Software versions need to be stamped so users know which is the more recent version. For every project, users must be able to organize the timeline of the evolving code. There is an infinite number of ways to organize your version numbers. However, PEP 440 introduces a version format that every Python package, and ideally every application, should follow so that other programs and packages can easily and reliably identify which versions of your package they require.

10''? Chapter 1 Many distributed version control system (DVCS) platforms, such as Git and Mercurial, are able to generate version numbers using an identifying hash (for Git, refer to git describe). Unfortunately, this system isn't compatible with the scheme defined by PEP? 440: for one thing, identifying hashes aren't orderable. Coding Style and Automated Checks Coding style is a touchy subject, but one we should talk about before we dive further into Python. Unlike many programming languages, Python uses indentation to define blocks. While this offers a simple solution to the age-old question 'Where should I put my braces'? it introduces a new question: 'How should I indent'? That was one of the first questions raised in the community, so the Python folks, in their vast wisdom, came up with the PEP? 8: Style Guide for Python Code (https://www.python.org/dev/peps/pep-0008/). This document defines the standard style for writing Python code. The list of guidelines boils down to: ? Use four spaces per indentation level. ? Limit all lines to a maximum of 79 characters. ? Separate top-level function and class definitions with two blank lines. ? Encode files using ASCII or UTF-8. ? Use one module import per import statement and per line. Place import statements at the top of the file, after comments and docstrings, grouped first by standard, then by third party, and finally by local library imports. ? Do not use extraneous whitespaces between parentheses, square brackets, or braces or before commas. ? Write class names in camel case (e.g., CamelCase), suffix exceptions with Error (if applicable), and name functions in lowercase with words and underscores (e.g., separated_by_underscores). Use a leading underscore for _private attributes or methods. These guidelines really aren't hard to follow, and they make a lot of sense. Most Python programmers have no trouble sticking to them as they write code. However, errare humanum est, and it's still a pain to look through your code to make sure it fits the PEP? 8 guidelines. Luckily, there's a pep8 tool (found at https://pypi.org/project/pep8/) that can automatically check any Python file you send its way. Install pep8 with pip, and then you can use it on? a file like so: $ pep8 hello.py hello.py:4:1: E302 expected 2 blank lines, found 1

Sign in with your Inkflash login details:
Welcome! You’re just one step away from a personalised 3D book exploring experience:
Your name
Email address
Choose a password: Forgot your password?
What’s 5 added to 8?

Fingerpress.co.uk - book publisher
Inkflash is owned and operated by Fingerpress (UK). Copyright ©, all rights reserved.

Site design and development by Matt Stephens, Dino Fancellu and William Narmontas.
Follow Inkflash on Twitter (@InkflashVR) and LinkedIn for the latest site developments.

Acknowledgments, image attributions, shout-outs etc

This website uses cookies to count visitors. Use at your own peril!!!!