Teaching Computers Threefold

How do you teach a computer to learn?

According to Jim Young, a mathematics professor at the College of Charleston, you approach the task from three directions:

1. Deductive inference, or working from the general to the specific. Tropical storms cause big waves. There's a tropical storm off the coast, so, therefore, there should be good surfing at Folly Beach. Examples are logic, proof theory (Young: "What proofs require a spark of genius from a human and which proofs can be effectively discovered by a computer?

2. Inductive inference, or working from examples to general principles. Big waves at the beach are typically followed by a big storm, so, therefore, storms must push waves out ahead of them. Examples: Statistics ("How can we extract and measure useful information from an entire population using random subsamples of that population and prior knowledge?"); Learning Theory (which studies ways humans and computers learn from the past to predict the future); Information Theory ("How can images, text and audio files be encoded effectively?"); and Artificial Intelligence ("How do we recognize if and when a machine is displaying intelligence? When and how can we trust intelligent machines?").

3. Dynamic Inference (a term coined by Young). Its components: Computational complexity (Is it possible to create processes that solve problems fast enough to be useful?); Complex Systems Studies (which look at how seemingly simple rules can display exceedingly complicated behavior over time); and Evolutionary Algorithms (algorithm = "a predetermined set of instructions for solving a specific problem in a limited number of steps"), the idea that problem-solving formulas could be written with a tendency to mutate, perhaps evolving better formulas over time.

Some computer systems already are using these principles, sometimes with stunning success.

A learning-theory application at the Postal Service now reads handwritten addresses with a lower error rate than human mail handlers. "How do you teach a machine to recognize the number 9, whether you write it or I write it? Well, one answer is, you show it a lot of nines," said Young. "These neural nets can now recognize a nine they've never seen before."

That concept, called neural network computing, models a machine's problem-solving processes on the structure of the human brain. Created to make predictions, neural networks function by looking for patterns in examples from the past. The more examples they see, the more accurate they become.

Robert Chapman, a state research scientist who spends most of his time working at the Hollings Marine Lab, uses the technique to predict shrimp harvests. For the past three years, Chapman has been feeding the computer all sorts of historical data: water temperatures, salinity records, etc.

Today, Chapman can take just a few bits of simple data months in advance and make extremely accurate predictions about the size of the local shrimp harvest. "What this does is, it extracts the (predictive) mechanisms from the data," said Chapman. "We don't know what the mechanisms are ... but it learns from the past."

As exciting as neural networks are, the area with the most science-fiction potential might well be evolutionary algorithms: mathematical formulas with built-in imperfections that mutate as they reproduce themselves.

Like the biological theory of evolution, this mathematical mutation produces plenty of flops (which are discarded), but when random chance produces an improvement, success is passed along to future generations.

Given the right starting instructions and sufficient processing speed, evolutionary algorithms conceivably could discover things their creators never even imagined. In a sense, they already have: Young and Starr cite evolutionary algorithms that, when applied to the study of electronic circuits and control systems, independently re-created 21 existing patents...and discovered two new ones.

This article appeared in The Post and Courier and updated online at Charleston.net on Monday, February 28, 2005.