Intro To Computational Linguistics

Intro To Computational Linguistics


Natural language processing comes in many varieties. The most robust natural language systems are tailored to the most limited applications. The simplest approach to natural language processing is to program the computer to look for a limited set of key words or phrases. When the computer finds these words it produces a programmed response. The ELIZA program offers a particularly compelling example of the keyword approach to natural language processing. ELIZA was written at MIT in the mid-1960s to mimic the role of a psychoanalyst interviewing a patient. Examples of ELIZA and related programs are now widely available on the web and personal computers.

ELIZA was never intended to be a model of natural language understanding, yet it is still one of the most popular artificial intelligence programs in the public domain. As long as the user accepts the premise that the program is conducting an open-ended interview, ELIZA can produce a convincing imitation of a talking computer. ELIZA works by searching for a list of keywords in the input. If the program finds one of these words, it asks a preprogrammed question that centers around the keyword. If the program does not find a word on its list, it chooses from a set of open-ended responses, such as Tell me more or Go on. Continue reading “Intro To Computational Linguistics”

Intro To Computational Linguistics

Intro To Computational Linguistics

Machine Translation

At the end of the 1950s, researchers in the United States, Russia, and Western Europe were confident that high-quality machine translation (MT) of scientific and technical documents would be possible within a very few years. After the promise had remained unrealized for a decade, the National Academy of Sciences of the United States published the much cited but little read report of its Automatic Language Processing Advisory Committee. The ALPAC Report recommended

that the resources that were being expended on MT as a solution to immediate practical problems should be redirected towards more fundamental questions of language processing that would have to be answered before any translation machine could be built. The number of laboratories working in the field was sharply reduced all over the world, and few of them were able to obtain funding for more long-range research programs in what then came to be known as computational linguistics.

There was a resurgence of interest in machine translation in the 1980s and, although the approaches adopted differed little from those of the 1960s, many of the efforts, notably in Japan, were rapidly deemed successful. This seems to have had less to do with advances in linguistics and software technology or with the greater size and speed of computers than with a better appreciation of special situations where ingenuity might make a limited success of rudimentary MT. The most conspicuous example was the METEO system, developed at the University of Montreal, which has long provided the French translations of the weather reports used by airlines, shipping companies, and others. Some manufacturers of machinery have found it possible to translate maintenance manuals used within their organizations (not by their customers) largely automatically by having the technical writers use only certain words and only in carefully prescribed ways.

Why Machine Translation Is Hard

Many factors contribute to the difficulty of machine translation, including words with multiple meanings, sentences with multiple grammatical structures, uncertainty about what a pronoun refers to, and other problems of grammar. But two common misunderstandings make translation seem altogether simpler than it is. First, translation is not primarily a linguistic operation, and second, translation is not an operation that preserves meaning.

There is a famous example that makes the first point well. Consider the sentence:

The police refused the students a permit because they feared violence.

Suppose that it is to be translated into a language like French in which the word for ‘police’ is feminine. Presumably the pronoun that translates ‘they’ will also have to be feminine. Now replace the word ‘feared’ with ‘advocated’. Now, suddenly, it seems that ‘they’ refers to the students and not to the police and, if the word for students is masculine, it will therefore require a different translation. The knowledge required to reach these conclusions has nothing linguistic about it. It has to do with everyday facts about students, police, violence, and the kinds of relationships we have seen these things enter into.

The second point is, of course, closely related. Consider the following question, stated in French: Ou voulez-vous que je me mette? It means literally, “Where do you want me to put myself?” but it is a very natural translation for a whole family of English questions of the form “Where do you want me to sit/stand/sign my name/park/tie up my boat?” In most situations, the English “Where do you want me?” would be acceptable, but it is natural and routine to add or delete information in order to produce a fluent translation. Sometimes it cannot be avoided because there are languages like French in which pronouns must show number and gender, Japanese where pronouns are often omitted altogether, Russian where there are no articles, Chinese where nouns do not differentiate singular and plural nor verbs present and past, and German where flexibility of the word order can leave uncertainties about what is the subject and what is the object.

The Structure of Machine Translation Systems

While there have been many variants, most MT systems, and certainly those that have found practical application, have parts that can be named for the chapters in a linguistic text book. They have lexical, morphological, syntactic, and possibly semantic components, one for each of the two languages, for treating basic words, complex words, sentences and meanings. Each feeds into the next until a very abstract representation of the sentence is produced by the last one in the chain.

There is also a ‘transfer’ component, the only one that is specialized for a particular pair of languages, which converts the most abstract source representation that can be achieved into a corresponding abstract target representation. The target sentence is produced from this essentially by reversing the analysis process. Some systems make use of a so-called ‘interlingua’ or intermediate language, in which case the transfer stage is divided into two steps, one translating a source sentence into the interlingua and the other translating the result of this into an abstract representation in the target language.

One other problem for computers is dealing with metaphor. Metaphors are a common part of language and occur frequently in the computer world:

  • How can I kill the program?
  • How do I get back into dos?
  • My car drinks gasoline

One approach treats metaphor as a failure of regular semantic rules

Compute the normal meaning of get into—dos violates its selection restrictions

dos isn’t an enclosure so the interpreter fails

Next have to search for an unconventional meaning for get into and recompute its meaning

If an unconventional meaning isn’t available, you can try using context, or world knowledge

Statistical procedures aren’t likely to generate interpretations for new metaphors.


Interpretation routines might result in overgeneralizations:

How can I kill dos? —> *How can I give birth to dos?

*How can I slay dos?

Mary caught a cold from John —> *John threw Mary his cold.

Catching a cold in unintentional (as opposed to catching a thief)

Getting Started

The best way to learn about language processing is to write your own computer programs. To do this, users will need access to a computer that can display information on the internet. Anyone with an email account on a personal computer has this type of access. The exercises in this class are written for the Perl programming language. This language is widely available on mainframe computers, and allows users to manipulate strings of text with a modicum of ease. In order to use Perl on a mainframe computer, however, the reader will have to access the computer directly via a terminal emulation program.

The only other item that you will need for Perl programming is a text editor. Text editors provide a means of writing the commands that make up a Perl program. Mainframe computers typically have a program that allows users to write text files. You can also use these programs to write a Perl program. The University of Kansas mainframe uses the Pico and vi editors. Once you have assembled the basic tools for creating Perl programs you are ready to begin language processing.

Intro To Computational Linguistics

Intro To Computational Linguistics

The image of humans conversing with their computers is both a thoroughly accepted cliche of science fiction and the ultimate goal of computer programming, and yet, the year 2001 has come and gone without the appearance of anything like the HAL 9000 talking computer featured in the movie 2001: A Space Odyssey.

Computational linguists attempt to use computers to process human languages. The field of computational linguistics has two general aims:

  • The technological. To enable computers to analyze and process natural language.
  • The psychological. To model human language processing on computers.

From the technological perspective, natural language applications include:

  • Speech recognition. Today, many personal computers include speech recognition software.
  • Natural language interfaces to software. For example, demonstration systems have been built that let a user ask for flight information.


chatterbots, e.g., Alice

natural language understanding, e.g., a perl parser

Document retrieval and information extraction from written text. For example, a computer system could scan newspaper articles, looking for information about events of a particular type and enter the information into a database.


web searches, e.g., google.

course information and enrollment, e.g., KU, Linguistics.

  • Machine translation. Computers offer the promise of quick translations between languages.


machine translation, e.g., SDL International

The rapid growth of the Internet/WWW and the emergence of the information society poses exciting new challenges to computational linguistics. Although the new media combine text, graphics, sound and movies, the whole wealth of multimedia information can only be structured, indexed and navigated through language. For browsing, navigating, filtering and processing the information on the web, we need language technology. The increasing multilingual nature of the web constitutes an additional challenge for language technology. The multilingual web can only be mastered with the help of multilingual tools for indexing and navigating.

Computational linguists adopting the psychological perspective hypothesize that at some abstract level, the brain is a kind of biological computer, and that an adequate answer to how people understand and generate language must be in terms formal and precise enough to be modeled by a computer.

About Us | RIT Press

About Us | RIT Press

RIT Press is a scholarly publishing enterprise at Rochester Institute of Technology. Established in 2001 as RIT Cary Graphic Arts Press, the Press initially focused on publishing titles that documented graphic communication processes, printing history, and bookmaking. As its editorial policies have evolved, the Press has broadened its reach to include content that supports all academic disciplines offered at Rochester Institute of Technology, our host institution. These include — but are not limited to — business, computer science, applied science and technology, engineering, graphic arts, deaf studies, and liberal arts.

RIT Press is dedicated to the innovative use of new publishing technology while upholding high standards in content quality, publication design, and print/digital production.  The Press offers specialized titles for niche academic audiences,  trade editions for mass-market audiences, occasional limited editions with unique aesthetic standards.

What We Talk About When We Talk About Design History: Design Observer

What We Talk About When We Talk About Design History: Design Observer

From the packaging of our belongings to the presentation of our surroundings, most of us recognize that design has, over the course of the past century, become a ubiquitous component in everyday life. Design is signage and graffiti and labels and lace, posters and propaganda and toothbrushes and teapots: objects and artefacts that captivate and delight us, frustrate or provoke us, but why?

This is where design historians come in.

Design history is, after all, social history: it’s an evolutionary (and somewhat cautionary) tale of use and abuse, of innovation and migration, of the inevitable tide of obsolescence that puzzles some of us to such a vexing degree that we simply have no other choice but to become design historians to start making sense of things.

And we begin, like all historians, by doing research.

Stanford Machine Learning

via Machine Learning.

In this course, you’ll learn about some of the most widely used and successful machine learning techniques. You’ll have the opportunity to implement these algorithms yourself, and gain practice with them. You will also learn some of practical hands-on tricks and techniques (rarely discussed in textbooks) that help get learning algorithms to work well. This is an “applied” machine learning class, and we emphasize the intuitions and know-how needed to get learning algorithms to work in practice, rather than the mathematical derivations.

Familiarity with programming, basic linear algebra (matrices, vectors, matrix-vector multiplication), and basic probability (random variables, basic properties of probability) is assumed. Basic calculus (derivatives and partial derivatives) would be helpful and would give you additional intuitions about the algorithms, but isn’t required to fully complete this course.