Table of Contents

Preface 1. Text 1.1 Processing a String One Character at a Time; 1.2 Converting Between Characters and Numeric Codes 1.3 Testing Whether an Object Is String-like 1.4 Aligning Strings 1.5 Trimming Space from the Ends of a String 1.6 Combining Strings; 1.7 Reversing a String by Words or Characters 1.8 Checking Whether a String Contains a Set of Characters 1.9 Simplifying Usage of Strings' translate Method 1.10 Filtering a String for a Set of Characters 1.11 Checking Whether a String Is Text or Binary; 1.12 Controlling Case 1.13 Accessing Substrings 1.14 Changing the Indentation of a Multiline String 1.15 Expanding and Compressing Tabs 1.16 Interpolating Variables in a String; 1.17 Interpolating Variables in a String in Python 2.4 1.18 Replacing Multiple Patterns in a Single Pass 1.19 Checking a String for Any of Multiple Endings 1.20 Handling International Text with Unicode 1.21 Converting Between Unicode and Plain Strings 1.22 Printing Unicode Characters to Standard Output; 1.23 Encoding Unicode Data for XML and HTML 1.24 Making Some Strings Case-Insensitive 1.25 Converting HTML Documents to Text on a Unix Terminal 2. Files 2.1 Reading from a File 2.2 Writing to a File 2.3 Searching and Replacing Text in a File 2.4 Reading a Specific Line from a File 2.5 Counting Lines in a File 2.6 Processing Every Word in a File 2.7 Using Random-Access Input/Output 2.8 Updating a Random-Access File 2.9 Reading Data from zip Files 2.10 Handling a zip File Inside a String 2.11 Archiving a Tree of Files into a Compressed tar File 2.12 Sending Binary Data to Standard Output Under Windows 2.13 Using a Clike iostream Syntax 2.14 Rewinding an Input File to the Beginning; 2.15 Adapting a File-like Object to a True File Object 2.16 Walking Directory Trees 2.17 Swapping One File Extension for AnotherThroughout a Directory Tree 2.18 Finding a File Given a Search Path 2.19 Finding Files Given a Search Path and a Pattern; 2.20 Finding a File on the Python Search Path 2.21 Dynamically Changing the Python Search Path 2.22 Computing the Relative Path from One Directory to Another 2.23 Reading an Unbuffered Character in a Cross-Platform Way 2.24 Counting Pages of PDF Documents on Mac OS X 2.25 Changing File Attributes on Windows 2.26 Extracting Text from OpenOffice.org Documents 2.27 Extracting Text from Microsoft Word Documents 2.28 File Locking Using a Cross-Platform API 2.29 Versioning Filenames 2.30 Calculating CRC-64 Cyclic Redundancy Checks 3. Time and Money 3.1 Calculating Yesterday and Tomorrow 3.2 Finding Last Friday 3.3 Calculating Time Periods in a Date Range 3.4 Summing Durations of Songs 3.5 Calculating the Number of Weekdays Between Two Dates 3.6 Looking up Holidays Automatically 3.7 Fuzzy Parsing of Dates 3.8 Checking Whether Daylight Saving Time Is Currently in Effect 3.9 Converting Time Zones 3.10 Running a Command Repeatedly 3.11 Scheduling Commands 3.12 Doing Decimal Arithmetic 3.13 Formatting Decimals as Currency 3.14 Using Python as a Simple Adding Machine 3.15 Checking a Credit Card Checksum 3.16 Watching Foreign Exchange Rates 4. Python Shortcuts 4.1 Copying an Object 4.2 Constructing Lists with List Comprehensions 4.3 Returning an Element of a List If It Exists 4.4 Looping over Items and Their Indices in a Sequence 4.5 Creating Lists of Lists Without Sharing References 4.6 Flattening a Nested Sequence 4.7 Removing or Reordering Columns in a List of Rows 4.8 Transposing Two-Dimensional Arrays 4.9 Getting a Value from a Dictionary; 4.10 Adding an Entry to a Dictionary 4.11 Building a Dictionary Without Excessive Quoting 4.12 Building a Dict from a List of Alternating Keys and Values 4.13 Extracting a Subset of a Dictionary 4.14 Inverting a Dictionary 4.15 Associating Multiple Values with Each Key in a Dictionary 4.16 Using a Dictionary to Dispatch Methods or Functions 4.17 Finding Unions and Intersections of Dictionaries 4.18 Collecting a Bunch of Named Items 4.19 Assigning and Testing with One Statement 4.20 Using printf in Python 4.21 Randomly Picking Items with Given Probabilities 4.22 Handling Exceptions Within an Expression; 4.23 Ensuring a Name Is Defined in a Given Module 5. Searching and Sorting 5.1 Sorting a Dictionary 5.2 Sorting a List of Strings Case-Insensitively 5.3 Sorting a List of Objects by an Attribute of the Objects 5.4 Sorting Keys or Indices Based on the Corresponding Values 5.5 Sorting Strings with Embedded Numbers; 5.6 Processing All of a List's Items in Random Order 5.7 Keeping a Sequence Ordered as Items Are Added 5.8 Getting the First Few Smallest Items of a Sequence 5.9 Looking for Items in a Sorted Sequence 5.10 Selecting the nth Smallest Element of a Sequence; 5.11 Showing off quicksort in Three Lines 5.12 Performing Frequent Membership Tests on a Sequence 5.13 Finding Subsequences 5.14 Enriching the Dictionary Type with Ratings Functionality 5.15 Sorting Names and Separating Them by Initials 6. Object-Oriented Programming 6.1 Converting Among Temperature Scales; 6.2 Defining Constants 6.3 Restricting Attribute Setting; 6.4 Chaining Dictionary Lookups 6.5 Delegating Automatically as an Alternative to Inheritance 6.6 Delegating Special Methods in Proxies 6.7 Implementing Tuples with Named Items 6.8 Avoiding Boilerplate Accessors for Properties 6.9 Making a Fast Copy of an Object 6.10 Keeping References to Bound MethodsWithout Inhibiting Garbage Collection 6.11 Implementing a Ring Buffer 6.12 Checking an Instance for Any State Changes 6.13 Checking Whether an Object Has Necessary Attributes 6.14 Implementing the State Design Pattern 6.15 Implementing the "Singleton" Design Pattern; 6.16 Avoiding the "Singleton" Design Pattern with the Borg Idiom; 6.17 Implementing the Null Object Design Pattern 6.18 Automatically Initializing Instance Variablesfrom _ _init_ _ Arguments 6.19 Calling a Superclass _ _init_ _ Method If It Exists 6.20 Using Cooperative Supercalls Concisely and Safely 7. Persistence and Databases 7.1 Serializing Data Using the marshal Module 7.2 Serializing Data Using the pickle and cPickle Modules 7.3 Using Compression with Pickling 7.4 Using the cPickle Module on Classes and Instances 7.5 Holding Bound Methods in a Picklable Way; 7.6 Pickling Code Objects 7.7 Mutating Objects with shelve; 7.8 Using the Berkeley DB Database 7.9 Accesssing a MySQL Database; 7.10 Storing a BLOB in a MySQL Database 7.11 Storing a BLOB in a PostgreSQL Database 7.12 Storing a BLOB in a SQLite Database 7.13 Generating a Dictionary Mapping Field Names to Column Numbers; 7.14 Using dtuple for Flexible Accessto Query Results 7.15 Pretty-Printing the Contents of Database Cursors 7.16 Using a Single Parameter-Passing StyleAcross Various DB API Modules; 7.17 Using Microsoft Jet via ADO 7.18 Accessing a JDBC Database from a Jython Servlet 7.19 Using ODBC to Get Excel Data with Jython; 8. Debugging and Testing 8.1 Disabling Execution of Some Conditionals and Loops 8.2 Measuring Memory Usage on Linux; 8.3 Debugging the Garbage-Collection Process 8.4 Trapping and Recording Exceptions 8.5 Tracing Expressions and Comments in Debug Mode 8.6 Getting More Information from Tracebacks 8.7 Starting the Debugger Automatically After an Uncaught Exception 8.8 Running Unit Tests Most Simply 8.9 Running Unit Tests Automatically 8.10 Using doctest with unittest in Python 2.4 8.11 Checking Values Against Intervals in Unit Testing 9. Processes, Threads, and Synchronization 9.1 Synchronizing All Methods in an Object; 9.2 Terminating a Thread 9.3 Using a Queue.Queue as a Priority Queue 9.4 Working with a Thread Pool 9.5 Executing a Function in Parallel on Multiple Argument Sets 9.6 Coordinating Threads by Simple Message Passing 9.7 Storing Per-Thread Information; 9.8 Multitasking Cooperatively Without Threads 9.9 Determining Whether Another Instance of a ScriptIs Already Running in Windows; 9.10 Processing Windows Messages Using MsgWaitForMultipleObjects; 9.11 Driving an External Process with popen 9.12 Capturing the Output and Error Streamsfrom a Unix Shell Command 9.13 Forking a Daemon Process on Unix 10. System Administration 10.1 Generating Random Passwords 10.2 Generating Easily Remembered Somewhat-Random Passwords 10.3 Authenticating Users by Means of a POP Server 10.4 Calculating Apache Hits per IP Address 10.5 Calculating the Rate of Client Cache Hits on Apache 10.6 Spawning an Editor from a Script 10.7 Backing Up Files 10.8 Selectively Copying a Mailbox File 10.9 Building a Whitelist of Email Addresses From a Mailbox 10.10 Blocking Duplicate Mails; 10.11 Checking Your Windows Sound System 10.12 Registering or Unregistering a DLL on Windows 10.13 Checking and Modifying the Set of Tasks WindowsAutomatically Runs at Login 10.14 Creating a Share on Windows 10.15 Connecting to an Already Running Instance of Internet Explorer 10.16 Reading Microsoft Outlook Contacts; 10.17 Gathering Detailed System Information on Mac OS X 11. User Interfaces 11.1 Showing a Progress Indicator on a Text Console; 11.2 Avoiding lambda in Writing Callback Functions 11.3 Using Default Values and Bounds with tkSimpleDialog Functions 11.4 Adding Drag and Drop Reordering to a Tkinter Listbox 11.5 Entering Accented Characters in Tkinter Widgets 11.6 Embedding Inline GIFs Using Tkinter 11.7 Converting Among Image Formats 11.8 Implementing a Stopwatch in Tkinter 11.9 Combining GUIs and Asynchronous I/O with Threads 11.10 Using IDLE's Tree Widget in Tkinter 11.11 Supporting Multiple Values per Row in a Tkinter Listbox 11.12 Copying Geometry Methods and Options Between Tkinter Widgets 11.13 Implementing a Tabbed Notebook for Tkinter; 11.14 Using a wxPython Notebook with Panels 11.15 Implementing an ImageJ Plug-in in Jython 11.16 Viewing an Image from a URL with Swing and Jython 11.17 Getting User Input on Mac OS 11.18 Building a Python Cocoa GUI Programmatically 11.19 Implementing Fade-in Windows with IronPython 12. Processing XML 12.1 Checking XML Well-Formedness 12.2 Counting Tags in a Document 12.3 Extracting Text from an XML Document 12.4 Autodetecting XML Encoding 12.5 Converting an XML Document into a Tree of Python Objects 12.6 Removing Whitespace-only Text Nodesfrom an XML DOM Node's Subtree 12.7 Parsing Microsoft Excel's XML 12.8 Validating XML Documents 12.9 Filtering Elements and Attributes Belonging to a Given Namespace 12.10 Merging Continuous Text Events with a SAX Filter 12.11 Using MSHTML to Parse XML or HTML 13. Network Programming 13.1 Passing Messages with Socket Datagrams 13.2 Grabbing a Document from the Web 13.3 Filtering a List of FTP Sites; 13.4 Getting Time from a Server via the SNTP Protocol 13.5 Sending HTML Mail 13.6 Bundling Files in a MIME Message 13.7 Unpacking a Multipart MIME Message 13.8 Removing Attachments from an Email Message 13.9 Fixing Messages Parsed by Python 2.4 email.FeedParser; 13.10 Inspecting a POP3 Mailbox Interactively 13.11 Detecting Inactive Computers 13.12 Monitoring a Network with HTTP; 13.13 Forwarding and Redirecting Network Ports 13.14 Tunneling SSL Through a Proxy 13.15 Implementing the Dynamic IP Protocol; 13.16 Connecting to IRC and Logging Messages to Disk 13.17 Accessing LDAP Servers 14. Web Programming 14.1 Testing Whether CGI Is Working 14.2 Handling URLs Within a CGI Script 14.3 Uploading Files with CGI 14.4 Checking for a Web Page's Existence; 14.5 Checking Content Type via HTTP 14.6 Resuming the HTTP Download of a File 14.7 Handling Cookies While Fetching Web Pages; 14.8 Authenticating with a Proxy for HTTPS Navigation 14.9 Running a Servlet with Jython 14.10 Finding an Internet Explorer Cookie; 14.11 Generating OPML Files 14.12 Aggregating RSS Feeds; 14.13 Turning Data into Web Pages Through Templates 14.14 Rendering Arbitrary Objects with Nevow 15. Distributed Programming; 15.1 Making an XML-RPC Method Call 15.2 Serving XML-RPC Requests; 15.3 Using XML-RPC with Medusa 15.4 Enabling an XML-RPC Server to Be Terminated Remotely 15.5 Implementing SimpleXMLRPCServer Niceties 15.6 Giving an XML-RPC Server a wxPython GUI 15.7 Using Twisted Perspective Broker 15.8 Implementing a CORBA Server and Client 15.9 Performing Remote Logins Using telnetlib 15.10 Performing Remote Logins with SSH 15.11 Authenticating an SSL Client over HTTPS 16. Programs About Programs 16.2 Importing a Dynamically Generated Module 16.3 Importing from a Module Whose Name Is Determined at Runtime 16.4 Associating Parameters with a Function (Currying) 16.5 Composing Functions 16.6 Colorizing Python Source Using the Built-in Tokenizer 16.7 Merging and Splitting Tokens 16.8 Checking Whether a String Has Balanced Parentheses 16.9 Simulating Enumerations in Python 16.10 Referring to a List Comprehension While Building It 16.11 Automating the py2exe Compilation of Scripts into Windows Executables; 16.12 Binding Main Script and Modules into One Executable on Unix; 17. Extending and Embedding 17.1 Implementing a Simple Extension Type 17.2 Implementing a Simple Extension Type with Pyrex 17.3 Exposing a C Library to Python 17.4 Calling Functions from a Windows DLL 17.5 Using SWIG-Generated Modules in a Multithreaded Environment 17.6 Translating a Python Sequence into a C Arraywith the PySequence_Fast Protocol 17.7 Accessing a Python Sequence Item-by-Item with the Iterator Protocol 17.8 Returning None from a Python-Callable C Function 17.9 Debugging Dynamically Loaded C Extensions with gdb 17.10 Debugging Memory Problems 18. Algorithms 18.1 Removing Duplicates from a Sequence 18.2 Removing Duplicates from a SequenceWhile Maintaining Sequence Order; 18.3 Generating Random Samples with Replacement 18.4 Generating Random Samples Without Replacement 18.5 Memoizing (Caching) the Return Values of Functions 18.6 Implementing a FIFO Container; 18.7 Caching Objects with a FIFO Pruning Strategy 18.8 Implementing a Bag (Multiset) Collection Type 18.9 Simulating the Ternary Operator in Python 18.10 Computing Prime Numbers; 18.11 Formatting Integers as Binary Strings 18.12 Formatting Integers as Strings in Arbitrary Bases 18.13 Converting Numbers to Rationals via Farey Fractions 18.14 Doing Arithmetic with Error Propagation 18.15 Summing Numbers with Maximal Accuracy; 18.16 Simulating Floating Point 18.17 Computing the Convex Hulls and Diameters of 2D Point Sets 19. Iterators and Generators; 19.1 Writing a range-like Function with Float Increments 19.2 Building a List from Any Iterable 19.3 Generating the Fibonacci Sequence 19.4 Unpacking a Few Items in a Multiple Assignment; 19.5 Automatically Unpacking the Needed Number of Items 19.6 Dividing an Iterable into n Slices of Stride n 19.7 Looping on a Sequence by Overlapping Windows 19.8 Looping Through Multiple Iterables in Parallel 19.9 Looping Through the Cross-Product of Multiple Iterables 19.10 Reading a Text File by Paragraphs; 19.11 Reading Lines with Continuation Characters 19.12 Iterating on a Stream of Data Blocks as a Stream of Lines 19.13 Fetching Large Record Sets from a Database with a Generator 19.14 Merging Sorted Sequences 19.15 Generating Permutations, Combinations, and Selections 19.16 Generating the Partitions of an Integer; 19.17 Duplicating an Iterator 19.18 Looking Ahead into an Iterator; 19.19 Simplifying Queue-Consumer Threads 19.20 Running an Iterator in Another Thread 19.21 Computing a Summary Report with itertools.groupby 20. Descriptors, Decorators, and Metaclasses; 20.1 Getting Fresh Default Values at Each Function Call 20.2 Coding Properties as Nested Functions 20.3 Aliasing Attribute Values; 20.4 Caching Attribute Values 20.5 Using One Method as Accessor for Multiple Attributes 20.6 Adding Functionality to a Class by Wrapping a Method 20.7 Adding Functionality to a Class by Enriching All Methods 20.8 Adding a Method to a Class Instance at Runtime 20.9 Checking Whether Interfaces Are Implemented; 20.10 Using _ _new_ _ and _ _init_ _ Appropriately in Custom Metaclasses 20.11 Allowing Chaining of Mutating List Methods; 20.12 Using Cooperative Supercalls with Terser Syntax 20.13 Initializing Instance Attributes Without Using _ _init_ _; 20.14 Automatic Initialization of Instance Attributes 20.15 Upgrading Class Instances Automatically on reload 20.16 Binding Constants at Compile Time 20.17 Solving Metaclass Conflicts; Index

About the Author

Alex Martelli spent 8 years with IBM Research, winning three Outstanding Technical Achievement Awards. He then spent 13 as a Senior Software Consultant at think3 inc, developing libraries, network protocols, GUI engines, event frameworks, and web access frontends. He has also taught programming languages, development methods, and numerical computing at Ferrara University and other venues. He's a C++ MVP for Brainbench, and a member of the Python Software Foundation. He currently works for AB Strakt, a Python-centered software house in G?teborg, Sweden, mostly by telecommuting from his home in Bologna, Italy. Alex's proudest achievement is the articles that appeared in Bridge World (January/February 2000), which were hailed as giant steps towards solving issues that had haunted contract bridge theoreticians for decades. Anna Ravenscroft has a background in training and mentoring, particularly in office technologies. She brings a fresh perspective to Python with a focus on practical, real-world problem solving. David Ascher is the lead for Python projects at ActiveState, including Komodo, ActiveState's integrated development environment written mostly in Python. David has taught courses about Python to corporations, in universities, and at conferences. He also organized the Python track at the 1999 and 2000 O'Reilly Open Source Conventions, and was the program chair for the 10th International Python Conference. In addition, he co-wrote Learning Python (both editions) and serves as a director of the Python Software Foundation. David holds a B.S. in physics and a Ph.D. in cognitive science, both from Brown University.