Skip To Page Navigation|Skip To Page Content

Addison-Wesley / Prentice Hall

Computer Science

My Instructor Resource Center : Log in or request access

Text Processing in Python
David Mertz

ISBN-10: 0321112547
ISBN-13: 9780321112545

Publisher: Addison-Wesley Professional
Copyright: 2003
Format: Paper; 544 pp
Published: 06/02/2003
Status: Instock

Net price: $28.04 (What's this?)

Customers outside the U.S., click here.

Request a printed exam copy

Email this page to a colleague
Prepare for the First Day of Class
Learn about customization options

Text Processing in Python describes techniques for manipulation of text using the

Python programming language. At the broadest level, text processing is simply

taking textual information and doing something with it. This might be

restructuring or reformatting it, extracting smaller bits of information from it,

or performing calculations that depend on the text. Text processing is arguably

what most programmers spend most of their time doing. Because Python is

clear, expressive, and object-oriented it is a perfect language for doing text

processing, even better than Perl. As the amount of data everywhere continues

to increase, this is more and more of a challenge for programmers. This book is

not a tutorial on Python. It has two other goals: helping the programmer get

the job done pragmatically and efficiently; and giving the reader an

understanding - both theoretically and conceptually - of why what works works

and what doesn't work doesn't work. Mertz provides practical pointers and tips

that emphasize efficent, flexible, and maintainable approaches to the textprocessing

tasks that working programmers face daily.

A clear, practical guide to using Python for text processing, arguably what most programmers spend most of their time doing.

° Demonstrates how Python is the perfect language for text-processing functions.

° Provides practical pointers and tips that emphasize efficient, flexible, and maintainable approaches to text-processing challenges.

° Helps programmers develop solutions for dealing with the increasing amounts of data with which we are all inundated.

Preface.

What Is Text Processing?

The Philosophy of Text Processing.

What You'll Need to Use This Book.

Conventions Used in This Book.

A Word on Source Code Examples.

External Resources.

1. Python Basics.

Techniques and Patterns.

Utilizing Higher-Order Functions in Text Processing.

Exercise: More on combinatorial functions.

Specializing Python Datatypes.

Base Classes for Datatypes.

Exercise: Filling out the forms (or deciding not to)

Problem: Working with lines from a large file.

Standard Modules.

Working with the Python Interpreter.

Working with the Local Filesystem.

Running External Commands and Accessing OS Features.

Special Data Values and Formats.

Other Modules in the Standard Library.

Serializing and Storing Python Objects.

Platform Specific Operations.

Working with Multi-Media Formats.

Miscellaneous Other Modules.

2. Basic String Operations.

Some Common Tasks.

Problem: Quickly sorting lines on custom criteria.

Problem: Reformatting paragraphs of text.

Problem: Column statistics for delimited or flat-record files.

Problem: Counting characters, words, lines and paragraphs.

Problem: Transmit binary data as ASCII.

Problem: Creating word or letter histograms.

Problem: Reading a file backwards by record, line, or paragraph.

Standard Modules.

Basic String Transformations.

Strings as Files, and Files as Strings.

Converting Between Binary and ASCII.

Cryptography.

Compression.

Unicode.

Solving Problems.

Exercise: Many ways to take out the garbage.

Exercise: Making sure things are what they should be

Exercise: Finding needles in haystacks (full-text indexing).

3. Regular Expressions.

A Regular Expression Tutorial.

Just What is a Regular Expression Anyway?

Matching Patterns In Text: The Basics.

Matching Patterns In Text: Intermediate.

Advanced Regular Expression Extensions.

Some Common Tasks.

Problem: Making a text block flush left.

Problem: Summarizing command-line option documentation.

Problem: Detecting duplicate words.

Problem: Checking for server errors.

Problem: Reading lines with continuation characters

Problem: Identifying URLs and email addresses in texts.

Problem: Pretty printing numbers.

Standard Modules.

Versions and optimizations.

Simple Pattern Matching.

Regular Expression Modules.

4. Parsers and State Machines.

An Introduction to Parsers.

When data becomes deep and texts become stateful.

What is a grammar?

An EBNF grammar for IF/THEN/END structures.

Pencil-and-Paper Parsing.

Exercise: Some variations on the language.

An Introduction to State Machines.

Understanding State Machines.

Text Processing State Machines.

When Not To Use A State Machine.

When to Use a State Machine.

An Abstract State Machine Class.

Processing a Report with a Concrete State Machine.

Subgraphs and State Reuse.

Exercise: Finding other solutions.

Parser Libraries for Python.

Specialized Parsers in the Standard Library.

Low-Level State Machine Parsing.

High-Level EBNF Parsing.

High-Level Programmatic Parsing.

5. Internet Tools and Techniques.

Working With Email and Newsgroups.

Manipulating and Creating Message Texts.

Communicating with Mail Servers.

Message Collections and Message Parts.

World Wide Web Applications.

Common Gateway Interface.

Parsing, Creating, and Manipulating HTML Documents.

Accessing Internet Resources.

Synopses of Other Internet Modules.

Standard Internet-Related Tools.

Third Party Internet-Related Tools.

Understanding XML.

Python Standard Library XML Modules.

Third Party XML-Related Tools

A. A Selective and Impressionistic Short Review of Python.

What Kind of Language is Python?

Namespaces and Bindings.

Assignment and Dereferencing.

Function and Class Definitions.

import Statements.

for Statements.

except Statements.

Datatypes.

Simple Types.

String Interpolation.

Printing.

Container Types.

Compound Types.

Flow Control.

if / then / else Statements.

Boolean Shortcutting.

for / continue / break Statements.

map( ), filter( ), reduce( ), and List Comprehensions.

while / else / continue / break Statements.

Functions, Simple Generators and the yield Statement.

Raising and Catching Exceptions.

Data as Code.

Functional Programming.

Emphasizing Expressions using lambda.

Special List Functions.

List-Application Functions as Flow Control.

Extended Call Syntax and apply( ).

B. A Data Compression Primer.

Introduction.

Lossless and Lossy Compression.

A Data Set Example.

Whitespace Compression.

Run-Length Encoding.

Hu_man Encoding.

Lempel-Ziv Compression.

Solving the Right Problem.

A Custom Text Compressor.

References.

C. Understanding Unicode.

Some Background on Characters.

What is Unicode?

Encodings.

Declarations.

Finding Codepoints.

Resources.

D. A State-Machine for Adding Markup to Text.

E. Glossary.

Index. 0321112547T01302003

Text Processing [PTG: AW PROFESSIONAL] (Computer Science)

View a Sample Chapter PDF:/samplechapter/0321112547.pdf

David Mertz came to writing about programming via the unlikely route of first being a humanities professor. Along the way, he was a senior software developer, and now runs his own development company, Gnosis Software ("We know stuff!"). David writes regular columns and articles for IBM developerWorks, Intel Developer Network, O'Reilly ONLamp, and other publications.

0321112547AB05022003

Text Processing in Python is an example-driven, hands-on tutorial that carefully teaches programmers how to accomplish numerous text processing tasks using the Python language. Filled with concrete examples, this book provides efficient and effective solutions to specific text processing problems and practical strategies for dealing with all types of text processing challenges.

Text Processing in Python begins with an introduction to text processing and contains a quick Python tutorial to get you up to speed. It then delves into essential text processing subject areas, including string operations, regular expressions, parsers and state machines, and Internet tools and techniques. Appendixes cover such important topics as data compression and Unicode. A comprehensive index and plentiful cross-referencing offer easy access to available information. In addition, exercises throughout the book provide readers with further opportunity to hone their skills either on their own or in the classroom. A companion Web site (http://gnosis.cx/TPiP) contains source code and examples from the book.

Here is some of what you will find in thie book:

When do I use formal parsers to process structured and semi-structured data? Page 257
How do I work with full text indexing? Page 199
What patterns in text can be expressed using regular expressions? Page 204
How do I find a URL or an email address in text? Page 228
How do I process a report with a concrete state machine? Page 274
How do I parse, create, and manipulate internet formats? Page 345
How do I handle lossless and lossy compression? Page 454
How do I find codepoints in Unicode? Page 465

0321112547B05022003

Pearson Higher Education offers special pricing when you choose to package your text with other student resources. If you're interested in creating a cost-saving package for your students contact your Pearson Higher Education representative.