python-cookbook

Code samples from the "Python Cookbook, 3rd Edition", published by O'Reilly & Associates, May, 2013.

Table of contents

Chapter 1 Data Structures and Algorithms

• Unpacking a Sequence into Separate Variables
• Unpacking Elements from Iterables of Arbitrary Length
• Keeping the Last N Items
• Finding the Largest or Smallest N Items
• Implementing a Priority Queue
• Mapping Keys to Multiple Values in a Dictionary
• Keeping Dictionaries in Order
• Calculating with Dictionaries
• Finding Commonalities in Two Dictionaries
• Removing Duplicates from a Sequence while Maintaining Order
• Naming a Slice
• Determining the Most Frequently Occurring Items in a Sequence
• Sorting a List of Dictionaries by a Common Key
• Sorting Objects Without Native Comparison Support
• Grouping Records Together Based on a Field
• Filtering Sequence Elements
• Extracting a Subset of a Dictionary
• Mapping Names to Sequence Elements
• Transforming and Reducing Data at the Same Time
• Combining Multiple Mappings into a Single Mapping

Chapter 2 Strings and Text

• Splitting Strings on Any of Multiple Delimiters
• Matching Text at the Start or End of a String
• Matching Strings Using Shell Wildcard Patterns
• Matching and Searching for Text Patterns
• Searching and Replacing Text
• Searching and Replacing Case-Insensitive Text
• Specifying a Regular Expression for the Shortest Match
• Writing a Regular Expression for Multiline Patterns
• Normalizing Unicode Text to a Standard Representation
• Working with Unicode Characters in Regular Expressions
• Stripping Unwanted Characters from Strings
• Sanitizing and Cleaning Up Text
• Aligning Text Strings
• Combining and Concatenating Strings
• Interpolating Variables in Strings
• Reformatting Text to a Fixed Number of Columns
• Handling HTML and XML Entities in Text
• Tokenizing Text
• Writing a Simple Recursive Descent Parser
• Performing Text Operations on Byte Strings

Chapter 3 Numbers, Dates, and Times

• Rounding Numerical Values
• Performing Accurate Decimal Calculations
• Formatting Numbers for Output
• Working with Binary, Octal, and Hexadecimal Integers
• Packing and Unpacking Large Integers from Bytes
• Performing Complex-Valued Math
• Working with Infinity and NaNs
• Calculating with Fractions
• Calculating with Large Numerical Arrays
• Performing Matrix and Linear Algebra Calculations
• Picking Things at Random
• Converting Days to Seconds, and Other Basic Time Conversions
• Determining Last Friday’s Date
• Finding the Date Range for the Current Month
• Converting Strings into Datetimes
• Manipulating Dates Involving Time Zones

Chapter 4 Iterators and Generators

• Manually Consuming an Iterator
• Delegating Iteration
• Creating New Iteration Patterns with Generators
• Implementing the Iterator Protocol
• Iterating in Reverse
• Defining Generator Functions with Extra State
• Taking a Slice of an Iterator
• Skipping the First Part of an Iterable
• Iterating Over All Possible Combinations or Permutations
• Iterating Over the Index-Value Pairs of a Sequence
• Iterating Over Multiple Sequences Simultaneously
• Iterating on Items in Separate Containers
• Creating Data Processing Pipelines
• Flattening a Nested Sequence
• Iterating in Sorted Order Over Merged Sorted Iterables
• Replacing Infinite while Loops with an Iterator

Chapter 5 Files and I/O

• Reading and Writing Text Data
• Printing to a File
• Printing with a Different Separator or Line Ending
• Reading and Writing Binary Data
• Writing to a File That Doesn’t Already Exist
• Performing I/O Operations on a String
• Reading and Writing Compressed Datafiles
• Iterating Over Fixed-Sized Records
• Reading Binary Data into a Mutable Buffer
• Memory Mapping Binary Files
• Manipulating Pathnames
• Testing for the Existence of a File
• Getting a Directory Listing
• Bypassing Filename Encoding
• Printing Bad Filenames
• Adding or Changing the Encoding of an Already Open File
• Writing Bytes to a Text File
• Wrapping an Existing File Descriptor As a File Object
• Making Temporary Files and Directories
• Communicating with Serial Ports
• Serializing Python Objects

Chapter 6 Data Encoding and Processing

• Reading and Writing CSV Data
• Reading and Writing JSON Data
• Parsing Simple XML Data
• Parsing Huge XML Files Incrementally
• Turning a Dictionary into XML
• Parsing, Modifying, and Rewriting XML
• Parsing XML Documents with Namespaces
• Interacting with a Relational Database
• Decoding and Encoding Hexadecimal Digits
• Decoding and Encoding Base64
• Reading and Writing Binary Arrays of Structures
• Reading Nested and Variable-Sized Binary Structures
• Summarizing Data and Performing Statistics

Chapter 7 Functions

• Writing Functions That Accept Any Number of Arguments
• Writing Functions That Only Accept Keyword Arguments
• Attaching Informational Metadata to Function Arguments
• Returning Multiple Values from a Function
• Defining Functions with Default Arguments
• Defining Anonymous or Inline Functions
• Capturing Variables in Anonymous Functions
• Making an N-Argument Callable Work As a Callable with Fewer Arguments
• Replacing Single Method Classes with Functions
• Carrying Extra State with Callback Functions
• Inlining Callback Functions
• Accessing Variables Defined Inside a Closure

Chapter 8 Classes and Objects

• Changing the String Representation of Instances
• Customizing String Formatting
• Making Objects Support the Context-Management Protocol
• Saving Memory When Creating a Large Number of Instances
• Encapsulating Names in a Class
• Creating Managed Attributes
• Calling a Method on a Parent Class
• Extending a Property in a Subclass
• Creating a New Kind of Class or Instance Attribute
• Using Lazily Computed Properties
• Simplifying the Initialization of Data Structures
• Defining an Interface or Abstract Base Class
• Implementing a Data Model or Type System
• Implementing Custom Containers
• Delegating Attribute Access
• Defining More Than One Constructor in a Class
• Creating an Instance Without Invoking init
• Extending Classes with Mixins
• Implementing Stateful Objects or State Machines
• Calling a Method on an Object Given the Name As a String
• Implementing the Visitor Pattern
• Implementing the Visitor Pattern Without Recursion
• Managing Memory in Cyclic Data Structures
• Making Classes Support Comparison Operations
• Creating Cached Instances

Chapter 9 Metaprogramming

• Putting a Wrapper Around a Function
• Preserving Function Metadata When Writing Decorators
• Unwrapping a Decorator
• Defining a Decorator That Takes Arguments
• Defining a Decorator with User Adjustable Attributes
• Defining a Decorator That Takes an Optional Argument
• Enforcing Type Checking on a Function Using a Decorator
• Defining Decorators As Part of a Class
• Defining Decorators As Classes
• Applying Decorators to Class and Static Methods
• Writing Decorators That Add Arguments to Wrapped Functions
• Using Decorators to Patch Class Definitions
• Using a Metaclass to Control Instance Creation
• Capturing Class Attribute Definition Order
• Defining a Metaclass That Takes Optional Arguments
• Enforcing an Argument Signature on *args and **kwargs
• Enforcing Coding Conventions in Classes
• Defining Classes Programmatically
• Initializing Class Members at Definition Time
• Implementing Multiple Dispatch with Function Annotations
• Avoiding Repetitive Property Methods
• Defining Context Managers the Easy Way
• Executing Code with Local Side Effects
• Parsing and Analyzing Python Source
• Disassembling Python Byte Code

Chapter 10 Modules and Packages

• Making a Hierarchical Package of Modules
• Controlling the Import of Everything
• Importing Package Submodules Using Relative Names
• Splitting a Module into Multiple Files
• Making Separate Directories of Code Import Under a Common Namespace
• Reloading Modules
• Making a Directory or Zip File Runnable As a Main Script
• Reading Datafiles Within a Package
• Adding Directories to sys.path
• Importing Modules Using a Name Given in a String
• Loading Modules from a Remote Machine Using Import Hooks
• Patching Modules on Import
• Installing Packages Just for Yourself
• Creating a New Python Environment
• Distributing Packages

Chapter 11 Network and Web Programming

• Interacting with HTTP Services As a Client
• Creating a TCP Server
• Creating a UDP Server
• Generating a Range of IP Addresses from a CIDR Address
• Creating a Simple REST-Based Interface
• Implementing a Simple Remote Procedure Call with XML-RPC
• Communicating Simply Between Interpreters
• Implementing Remote Procedure Calls
• Authenticating Clients Simply
• Adding SSL to Network Services
• Passing a Socket File Descriptor Between Processes
• Understanding Event-Driven I/O
• Sending and Receiving Large Arrays

Chapter 12 Concurrency

• Starting and Stopping Threads
• Determining If a Thread Has Started
• Communicating Between Threads
• Locking Critical Sections
• Locking with Deadlock Avoidance
• Storing Thread-Specific State
• Creating a Thread Pool
• Performing Simple Parallel Programming
• Dealing with the GIL (and How to Stop Worrying About It)
• Defining an Actor Task
• Implementing Publish/Subscribe Messaging
• Using Generators As an Alternative to Threads
• Polling Multiple Thread Queues
• Launching a Daemon Process on Unix

Chapter 13 Utility Scripting and System Administration

• Accepting Script Input via Redirection, Pipes, or Input Files
• Terminating a Program with an Error Message
• Parsing Command-Line Options
• Prompting for a Password at Runtime
• Getting the Terminal Size
• Executing an External Command and Getting Its Output
• Copying or Moving Files and Directories
• Creating and Unpacking Archives
• Finding Files by Name
• Reading Configuration Files
• Adding Logging to Simple Scripts
• Adding Logging to Libraries
• Making a Stopwatch Timer
• Putting Limits on Memory and CPU Usage
• Launching a Web Browser

Chapter 14 Testing, Debugging, and Exceptions

• Testing Output Sent to stdout
• Patching Objects in Unit Tests
• Testing for Exceptional Conditions in Unit Tests
• Logging Test Output to a File
• Skipping or Anticipating Test Failures
• Handling Multiple Exceptions
• Catching All Exceptions
• Creating Custom Exceptions
• Raising an Exception in Response to Another Exception
• Reraising the Last Exception
• Issuing Warning Messages
• Debugging Basic Program Crashes
• Profiling and Timing Your Program
• Making Your Programs Run Faster

Chapter 15 C Extensions

• Accessing C Code Using ctypes
• Writing a Simple C Extension Module
• Writing an Extension Function That Operates on Arrays
• Managing Opaque Pointers in C Extension Modules
• Defining and Exporting C APIs from Extension Modules
• Calling Python from C
• Releasing the GIL in C Extensions
• Mixing Threads from C and Python
• Wrapping C Code with Swig
• Wrapping Existing C Code with Cython
• Using Cython to Write High-Performance Array Operations
• Turning a Function Pointer into a Callable
• Passing NULL-Terminated Strings to C Libraries
• Passing Unicode Strings to C Libraries
• Converting C Strings to Python
• Working with C Strings of Dubious Encoding
• Passing Filenames to C Extensions
• Passing Open Files to C Extensions
• Reading File-Like Objects from C
• Consuming an Iterable from C
• Diagnosing Segmentation Faults

Appendix Further Reading

• Online Resources
• Books for Learning Python
• Advanced Books

Index

Colophon