About the Microsoft Speech SDK Documents


Copyright

Information in this document is subject to change without notice and does not represent a commitment on the part of Microsoft Corporation. Companies, names, and data used in examples herein are fictitious unless otherwise noted. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents or pending patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. The furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property rights.

Copyright © 1995-1997 Microsoft Corporation. All rights reserved.

Macintosh is a registered trademark of Apple Computer, Inc. Sound Blaster is a trademark of Creative Technology, Ltd. Alpha APX is a trademark of Digital Equipment Corporation. Pentium is a registered trademark of Intel Corporation. OS/2 is a registered trademark of International Business Machines Corporation. Media Vision is a trademark of Media Vision, Inc. Microsoft, MS, MS-DOS, Visual Basic, Windows, and Win32 are registered trademarks and Visual C++ and Windows NT are trademarks of Microsoft Corporation. MIPS is a registered trademark of MIPS Computer Systems, Inc. Unicode is a trademark of Unicode, Inc. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd.


About This Book

Welcome to the Microsoft® Speech API Developer's Guide. This book provides a complete reference to the high-level and low-level speech objects that can be used to implement speech recognition and text-to-speech in Microsoft® Win32® applications. These speech objects are OLE component objects that conform to the OLE Component Object Model (COM).

To get the most out of this book, you should be familiar with the following material:

C/C++ programming concepts.

OLE programming concepts, including COM, obtaining pointers to interfaces, and calling functions.

Win32 application programming interface (API).


Document Conventions

The following document conventions are observed in this book.

Convention | Meaning
Bold text | Indicates a word that is a function name, data type, or other fixed element of the Windows, OLE, or Speech API — for example, ISRCentral::Pause.
Italic text | Indicates a word that is a placeholder or variable — for example, pdwPitch. Function parameters in API reference material is italic to indicate that any variable name can be used. In addition, first mentions of important terms are italicized to highlight their definitions.
UPPERCASE TEXT | Indicates a constant or structure — for example, SRGRMFMT_CFG or SRMODEINFO. Uppercase also indicates Microsoft® MS-DOS® filenames and paths.
Monospace text | Indicates source code examples.

Note: The interface syntax in this book follows the variable-naming convention known as Hungarian notation, in which variables are prefixed with lower-case letters to indicate their data type. For example, pszString is a pointer to a zero-terminated string. For more information about Hungarian notation, see Programming Windows by Charles Petzold.


For More Information

For more information about Windows programming, see the documentation included in the Microsoft Win32 Software Development Kit.

For more information about C/C++ programming, see the documentation for the Microsoft® Visual C++™ development system for Windows.

For information about OLE, see the Win32 SDK. Another good source is Inside OLE by Kraig Brockschmidt, which shows by example how to build OLE applications.


Why Should I Use Speech in My Application?

Whenever I mention speech recognition or text-to-speech, people instantly think of the computer on "Star Trek." The computer's synthesized voice sounds exactly like a human voice. The speech recognition is even better. It never listens in on conversations between people, and when given a command, the computer never makes a mistake. It's great. If computers had perfect speech recognition and text-to-speech like that today, everyone would be talking to computers. Unfortunately, it will be decades before speech technology is that good.

Even though speech technology isn't perfect, it's still useful to applications. Unfortunately, because some people expect more from speech recognition and text-to-speech than what the technology can deliver, they dismiss the technology outright. I will describe some reasonable expectations for speech technology and show you what you can accomplish with its current capabilities.


Why Should I Use Speech If It's Not Perfect?

Computer-based videos were introduced in multimedia PCs around 1991. They were a cool idea, but when the videos first appeared, they were really small. A 386/33 megahertz machine could support a video about 80x50 pixels with 15 frames per second (fps) and low-quality sound. Since then computer videos have improved significantly. They're now 320x200 pixels with 24 fps; conveying 24 times as much data per second as in 1991. These improvements in resolution and fps are still small compared to the ultimate video experience: a digital surround-sound movie. Real movies have about 4000x2000-pixel resolution, 24-bit color, and 4 channels of 44 kHz, 16-bit audio. That's several hundred times the data rate and processor speed of the videos you get on today's PCs.


So what does this have to do with speech recognition and text-to-speech?

Even though computer videos are significantly inferior to movies, many applications benefit from computer video technology, especially games and multimedia titles. Videos are so necessary to some applications that they wouldn't be able to function without them--even though computer videos are light years away from a real movie.

Speech recognition and text-to-speech are new technologies, just like computer videos. Although speech technologies aren't nearly as good as those on "Star Trek," many applications can get good use out of them.


Does Speech Replace the Keyboard and Mouse?

You might now be asking: If speech isn't perfect, how can I use it for my application? After all, speech is replacing the keyboard and mouse, and the keyboard never makes a mistake.

Don't worry.

Speech recognition will not replace the keyboard and the mouse, nor will text-to-speech eliminate text. Speech recognition and text-to-speech are just two more user interface "devices" available to an application developer. Speech recognition can be added to an already long list of user-interface "devices" such as the keyboard, mouse, joystick, and pen. Likewise, text-to-speech can be added to text, graphics, animated videos, and sound. Very few applications will use speech as their sole means of communication with users. Most will mix and match according to the strengths of each device.

If you think about it, applications already use several user-interface devices to communicate with users. A game makes use of both a joystick and keyboard. Users manipulate the joystick to "tell" the computer which direction they want their character to move. The keyboard is used to type in commands, such as "Say 'Hello.'" The joystick is a better input device for movement, while the keyboard is better for entering text. The game's choice of output "devices" works the same way. The application uses a combination of pictures, text, and sound to communicate with users. When an enemy aircraft blows up, a game doesn't draw "Boom!" in large letters. Instead, the game plays a recording of an explosion.

Speech is just another "device." As such, it has its place among other widely used devices.


What Can Speech Do?

Now that I've described reasonable expectations of speech technology, I'll tell you a bit about the current capabilities of text-to-speech and speech recognition.

Text-to-speech comes in two flavors, synthesized text-to-speech and concatenated text-to-speech.

Synthesized speech is what people typically think of when I mention text-to-speech. It reads text by analyzing the words and having the computer figure out the phonetic pronunciations for the words. The phonemes are then passed into a complex algorithm that simulates the human vocal tract and emits the sound. This method allows the text-to-speech to speak any word, even made-up ones like "Zamphoon," but it produces a voice that has very little emotion and is distinctly not human. You'd use this if you knew that the application had to speak, but you couldn't predict what it would need to say. Synthesized speech usually requires a 486/33 megahertz machine with 1 megabyte of working-set RAM.

Concatenated text-to-speech does something different. It analyzes the text and pulls recordings, words, and phrases out of a prerecorded library. The digital audio recordings are concatenated. Because the voice is just a recording that you've made, it sounds good. Unfortunately, if the text includes a word or phrase that you didn't record, the text-to-speech can't say it. Concatenated text-to-speech can be viewed as a form of audio compression because words or common phrases have to be recorded only once. For example, many telephone applications will have a recording for, "Press 1 to play new message; press 2 to send a fax," and so on, and another recording for, "Press 1 to fast-forward; press 2 to rewind." A concatenated text-to-speech will have only one recording of "press" rather than four. If concatenated text-to-speech doesn't seem that much different to you from recording your own .WAV files, you're right. However, concatenated text-to-speech will save you development time and bugs, allowing you to add more features to your software. Because concatenated text-to-speech just plays a .WAV file, it takes very little processor power and only a bit of memory, since most of the audio is stored on disk.

Speech recognition is somewhat more complicated to classify than text-to-speech. Each speech recognition engine has three characteristics:

Continuous vs. discrete: If speech recognition is continuous, users can speak to the system naturally. If it's discrete, users need to pause between each word. Obviously, continuous recognition is preferred over discrete recognition, but continuous recognition takes more processing power.

Vocabulary size: Speech recognition can support a small or large vocabulary. Small-vocabulary recognition allows users to give simple commands to their computers. To dictate a document, the system must have large-vocabulary recognition. Large-vocabulary recognition takes a lot more processor power and memory than small-vocabulary recognition.

Speaker dependency: Speaker-independent speech recognition works right out of the box, while speaker-dependent systems require that each user spend about 30 minutes training the system to his or her voice.

Although any combination of the three characteristics is possible, two combinations are popular today.

"Command and Control" speech recognition is continuous, small vocabulary, and speaker independent. This means that users can use several hundred different commands or phrases. If a user says a command that is not in the list, the speech-recognition system will return either "not recognized," or will think it heard a similar-sounding command. Because users of Command and Control can say only specific phrases, the phrases must be either visible on the screen--so intuitive that all users will know what to say--or the users must learn what phrases they can say. Command and Control speech recognition requires a 486/66 megahertz machine with 1 megabyte of working-set RAM.

"Discrete Dictation" speech recognition is discrete, large vocabulary, and speaker dependent. It's used to dictate text into word processors or mail, or for natural-language commands. Although users may say anything they wish, they must leave pauses between words, making the speech unnatural. Discrete dictation requires a Pentium/60 megahertz machine with 8 megabytes of RAM.


Sample Uses for Speech Technologies

It's impossible to say, "Thou shalt use speech to do XXX." Not only is every application different, but speech technology is so new that application writers haven't had much experience with it. As a general rule, use speech when it provides significant advantages to users, not just for the "Wow!" effect. Try to find uses that make the application easier to use, or allow users to access features more quickly than with the keyboard or mouse. In the case of games, use speech to add realism to the interaction with characters, making the game more involving and fun.

Telephony applications benefit most from speech recognition and text-to-speech because their sole means of communicating with users is through the telephone audio. Text-to-speech is not only easier to use than manually recording all of the prompts, but it also enables new features that allow arbitrary text to be read, such as names, addresses, and e-mail. Speech recognition is a great substitute for touch-tone menus not only because it's more natural and flexible, but also because many users don't have touch-tone phones. Several engine vendors have phone numbers you can call for a demonstration of telephony applications.

Games, especially adventure games, are the next beneficiaries of speech. Any game in which the user "talks" with characters will enjoy the fruits of speech. Speech recognition allows users to really talk to the characters, which is a lot more fun than typing in a response. Rather than just printing out what characters "speak," use text-to-speech to really make them speak. Not only does this increase the realism of the game, but it allows users to take their hands off the keyboard, sit back in their chairs, and enjoy the game.

Multimedia titles can use concatenated text-to-speech for recording audio passages, treating it like improved audio compression. They can also incorporate speech recognition so that users can sit back and relax, just as they can with games.

Of course, there are other applications that can use speech. For example, some applications use speech because they require that the user's hands be free for other things, or because the user can't see the screen.


Microsoft Speech API Documentation

This was just a brief overview of what you can do with speech technology. You should continue reading the Microsoft Speech API Documentation for more detail.


Overview of Speech Technologies

Speech recognition is the ability of a computer to understand the spoken word for the purpose of receiving command and data input from the speaker. Text-to-speech is the ability of a computer to convert text information into synthetic speech output.

Speech recognition and text-to-speech use engines, which are the programs that do the actual work of recognizing speech or playing text. Most speech-recognition engines convert incoming audio data to engine-specific phonemes, which are then translated into text that an application can use. (A phoneme is the smallest structural unit of sound that can be used to distinguish one utterance from another in a spoken language.) A text-to-speech engine performs the same process, in reverse. Engines are supplied by vendors that specialize in speech software; they may be bundled with new audio-enabled computers and sound cards, purchased separately, or licensed from the vendor.

The speech-recognition engine transcribes audio data received from an audio source, such as a microphone or a telephone line. The text-to-speech engine converts text to audio data, which is sent to an audio destination, such as a speaker, a headphone, or a telephone line. Under some circumstances, an engine may be able to transcribe audio data to or from a file.

An engine typically provides more than one mode for recognizing speech or playing text. For example, a speech-recognition engine will have a mode for each language or dialect that it can recognize. Likewise, a text-to-speech engine will have a mode for each voice, which plays text in a different speaking style or personality. Other modes may be optimized for a particular audio sampling rate, such as 8 kilohertz (kHz) for use over a telephone line.

Speech recognition can be as simple as a predefined set of voice commands that an application can recognize. More complex speech recognition involves the use of a grammar, which defines a set of words and phrases that can be recognized. A grammar may use rules to predict the most likely words to follow the word just spoken, or it may define a context that identifies the subject of dictation and the expected style of language.

Both speech-recognition and text-to-speech engines may make use of a pronunciation lexicon, which is a database of correct pronunciations for words and phrases to be recognized or played.

An engine's approach to recognizing speech or playing text determines the quality of speech in an application — that is, the accuracy of recognition or clarity of playback — and the amount of effort required from the user to get good accuracy or clarity. The engine's approach also affects the processor speed and memory required by an application; it may also influence the application's features or the design of its user interface.

This section provides an overview of speech technology. An understanding of both speech recognition and text-to-speech will help you decide how to best incorporate speech in your application and how to choose a technology that supports what you want to do.


Speech Recognition

All speech recognition involves detecting and recognizing words. Most speech-recognition engines can be categorized by the way in which they perform these basic tasks:

· Word separation. The degree of isolation between words required for the engine to recognize a word.

· Speaker dependence. The degree to which the engine is restricted to a particular speaker.

· Matching techniques. The method that the engine uses to match a detected word to known words in its vocabulary.

· Vocabulary size. The number of words that the engine searches to match a word.


Word Separation

Speech-recognition engines typically require one of the following types of verbal input to detect words:

· Discrete speech. Every word must be isolated by a pause before and after the word — usually about a quarter of a second — for the engine to recognize it. Discrete speech recognition requires much less processing than word-spotting or continuous speech, but it is less user-friendly.

· Word-spotting. A series of words may be spoken in a continuous utterance, but the engine recognizes only one word or phrase. For example, if a word-spotting engine listens for the word "time" and the user says "Tell me the time" or "Time to go," the engine recognizes only the word "time."

Word-spotting is used when a limited number of commands or answers are expected from the user and the way that the user speaks the commands is either unpredictable or unimportant.

Continuous speech recognizers can also be used to do word-spotting.

· Continuous speech. The engine encounters a continuous utterance with no pauses between words, but it can recognize the words that were spoken.

Continuous speech recognition is the best technology from a usability standpoint, because it is the most natural speaking style for human beings. However, it is the most computationally intensive because identifying the beginning and ending of words is very difficult — much like reading printed text without spaces or punctuation.


Dependence on the Speaker

Speech-recognition engines may require training to recognize speech well for a particular speaker, or they may be able to adapt to a greater or lesser extent. Engines can be grouped into these categories:

· Speaker-dependent. The engine requires the user to train it to recognize his or her voice. Training usually involves speaking a series of pre-selected phrases. Each new speaker must perform the same training.

Speaker-dependent engines work without training, but their accuracy usually starts below 95 percent and does not improve until the user completes the training. This technique takes the least amount of processing, but it is frustrating for most users because the training is tedious, taking anywhere from five minutes to several hours.

· Speaker-adaptive. The engine trains itself to recognize the user's voice while the user performs ordinary tasks. Accuracy usually starts at about 90 percent, but rises to more acceptable levels after a few hours of use.

Two considerations must be taken into account with speaker-adaptive technology. First, the user must somehow inform the engine when it makes a mistake so that it does not learn based on the mistake. Second, even though recognition improves for the individual user, other people who try to use the system will get higher error rates until they have used the system for a while.

· Speaker-independent. The engine starts with an accuracy above 95 percent for most users (those who speak without accents). Almost all speaker-independent engines have training or adaptive abilities that improve their accuracy by a few more percentage points, but they do not require the use of such training. Speaker-independent systems require several times the computational power of speaker-dependent systems.


Matching Techniques

Speech-recognition engines match a detected word to a known word using one of these techniques:

· Whole-word matching. The engine compares the incoming digital-audio signal against a prerecorded template of the word. This technique takes much less processing than subword matching, but it requires that the user (or someone) prerecord every word that will be recognized — sometimes several hundred thousand words. Whole-word templates also require large amounts of storage (between 50 and 512 bytes per word) and are practical only if the recognition vocabulary is known when the application is developed.

· Subword matching. The engine looks for subwords — usually phonemes — and then performs further pattern recognition on those. This technique takes more processing than whole-word matching, but it requires much less storage (between 5 and 20 bytes per word). In addition, the pronunciation of the word can be guessed from English text without requiring the user to speak the word beforehand.


Vocabulary Size

Speech-recognition engines typically support several different sizes of vocabulary. Vocabulary size does not represent the total number of words that a given engine can recognize. Instead, it determines the number of words that the engine can recognize accurately in a given state, which is defined by the word or words that were spoken before the current point in time. For example, if an engine is listening for "Tell me the time," "Tell me the day," and "What year is it?" and the user has already said "tell," "me," and "the," the current state has these two words: "time" and "day."

Speech-recognition engines support these vocabulary sizes:

· Small vocabulary. The engine can accurately recognize about 50 different words in a given state. Although many small-vocabulary engines can exceed this number, larger numbers of words significantly reduce the recognition accuracy and increase the computation load. Small-vocabulary engines are acceptable for command and control, but not for dictation.

· Medium vocabulary. The engine can accurately recognize about 1000 different words in a given state. Medium-vocabulary engines are good for natural language command and control or data entry.

· Large vocabulary. The engine can recognize several thousand words in a given state, perhaps up to 100,000. Large-vocabulary engines require much more computation than small-vocabulary engines, but the large vocabulary is necessary for dictation.


Typical Speech Recognizers on the Market

Three types of speech recognizers are common on the market today:

· Command and Control. The engine allows users to give simple commands, like "Minimize Window" or "Send mail to Fred" to the computer. It also allows for limited data entry such as numbers. Command and Control engines are continuous speech, speaker independent, sub-word modeling, and use a small vocabulary. To find more about what you can do with these recognizers look in the "Voice Commands" section.

· Discrete Dictation. The engine allows users to speak in arbitrary text so long as the user leaves pauses between words, providing text entry rates of up to 50 words per minute. Discrete dictation engines are discrete speech, speaker adaptive, sub-word modeling, and use a large vocabulary. To find more about what you can do with these recognizers look in the "Dictation" section.

· Continuous Dictation. The engine allows users to speak in arbitrary text using continuous speech, providing text entry rates of up to 120 words per minute. Continuous Dictation engines are continuous speech, speaker dependent, sub-word modeling, and use a large vocabulary. To find more about what you can do with these recognizers look in the "Dictation" section.


Text-to-Speech

To render text into speech, a text-to-speech engine must first determine the phonemes required to speak a word and then translate those phonemes into digital-audio data. Most text-to-speech engines can be categorized by the method that they use to translate phonemes into audible sound.

A text-to-speech engine translates text or phonemes into audible sound in one of several ways, either by synthesis or by diphone concatenation.

Concatenated Word. Although Concatenated Word systems are really synthesizers; these are one of the most commonly used text-to-speech systems around. In a concatenated word engine, the application designer provides recordings for phrases and individual words. The engine pastes the recordings together to speak out a sentence or phrase. If you use voice-mail then you've heard one of these engines speaking, "[You have] [three] [new messages]." The engine has recordings for "You have", all of the digits, and "new messages".

Synthesis. A text-to-speech engine that uses synthesis generates sounds like those created by the human vocal cords and applies various filters to simulate throat length, mouth cavity, lip shape, and tongue position. The voice produced by current synthesis technology tends to sound less human than a voice produced by diphone concatenation, but it is possible to obtain different qualities of voice by changing a few parameters.

Diphone Concatenation. A text-to-speech engine that uses diphone concatenation links short digital-audio segments together and performs intersegment smoothing to produce a continuous sound. Each diphone consists of two phonemes, one that leads into the sound and one that finishes the sound. For example, the word "hello" consists of these phonemes: h eh l œ. The corresponding diphones are silence-h h-eh eh-l l-œ œ-silence.

Diphones are acquired by recording many hours of a human voice and painstakingly identifying the beginning and ending of phonemes. Although this technique can produce a more realistic voice, it takes a considerable amount of work to create a new voice and the voice is not localizable because the phonemes are specific to the speaker's language.

To find out more about text-to-speech look in the "Voice Text" section.


Speech Application Programming Interface

The Microsoft Speech Application Programming Interface (API) uses the OLE Component Object Model (COM) architecture under Win32® (Windows® 95 and Windows NT® 3.51). The component objects can be accessed through C/C++ or Visual Basic's ® OLE Automation. The speech architecture is divided into two levels, a high level that is designed for ease and speed of implementation, and a low level that allows applications complete control of the technology.

This article briefly describes how to add simple speech recognition and text-to-speech to an application using C++ and the architecture involved. We begin with the high-level interfaces, known as Voice Commands and Voice Text.


Voice Commands

The voice-command interface is the high-level interface for speech recognition. It is designed to provide command and control speech recognition for applications. With this interface, a user gives the computer simple commands, such as "Open the file," and can answer simple yes/no questions. Command and Control does not allow speech dictation; developers wishing to implement dictation in applications need to use the low-level API.

The voice-command design mimics a Windows menu in behavior, providing a "menu" of commands that users can speak. Basically, to use voice commands, an application designs a Voice menu that corresponds to a window or state within the application. Most programs will have one Voice menu for the main window and one for every dialog box. Contained within every Voice menu is a list of voice commands that users can say. When they say one, the application is notified which command was spoken. "Open a file" and "Send mail to <e-mail name>" are typical voice commands. Each voice command has information in addition to the spoken command, such as a description string and a command ID.


Sample

Adding voice commands is fairly simple. An application using these would only have to make the following modifications:

CoCreateInstance (CLSID_VCmd, NULL, CLSCTX_LOCAL_SERVER, IID_IVoiceCmd, &pIVoiceCommand);

CoCreateInstance() instantiates a local copy of the Voice Commands Object.

pIVCmdNotifySink = new CMyIVCmdNotifySink;

This creates a instance of the CMyIVCmdNotifySink class that receives notifications when the user says a command. The functionality of the class is specific to the application.

pIVoiceCommand->Register ("", pIVCmdNotifySink, IID_IVCmdNotifySink, 0, NULL);

Calling Register informs the voice-commands object of the notification sink and also specifies where the audio is to come from.

sData.pData = a pointer to a list of VCMDCOMMAND structures;

sData.dwSize = size of the memory pointed to by sData.pData;

Next, the application must include a list of VCMDCOMMAND structures. Each structure contains a command ("Open a file"), description ("This opens a file from the disk"), command ID for identification, and a few other parameters. The commands are specific to the application.

strcpy (VCmdName.szApplication, "Application Name");

strcpy (VCmdName.szState, "Main");

pIVoiceCommand->MenuCreate (&VCmdName, NULL, VCMDMC_CREATE_TEMP, &pIVCmdMenu);

pIVCmdMenu->Add (number of commands, sData, NULL);

The code above creates an instance of a Voice menu object using the commands specified in sData. The menu generated in this example is temporary, but menus can be saved to a system storage, similar to the registry, so that an application doesn't have to reconstruct the VCMDCOMMAND structures every time.

pIVCmdMenu->Activate (hWnd, 0);

Calling Activate causes the speech-recognition engine to start "listening" for the specified commands whenever the window, hWnd, is active.

Once the application has called Activate, speech recognition will begin listening. When the user says one of the commands in the menu, the notification sink will be called with pIVCmdNotifySink->CommandRecognize (..., dwID, ...). dwID contains the ID command spoken, just like a Windows menu. Other notifications such as the VU-meter also come through, but most applications will only care about dWID.

When an application is no longer required to get recognitions for the menu anymore, it simply calls:

pIVCmdMenu->Deactivate ();

Finally, when the application shuts down, it releases the object:

pIVCmdMenu->Release();

pIVoiceCommand->Release();

Wait until pIVCmdNotifySink is released by engine.

That's a concise overview of how this high-level interface works.


Objects

Four objects are involved in speech recognition:

Voice-command object. This is the central control point for the voice-command module. Most applications will use only this as a means to create Voice-menu objects or Voice-menu enumerators.

Notification sink. The notification sink code is supplied by the application. The voice-command object calls methods in this object both when commands are spoken and for VU-level updates. It is up to the application to act upon the notifications.

Voice menu. The Voice-menu object provides most of the controls for speech recognition and is similar to a Windows-menu object accessed with an HMENU. The object allows commands to be added or removed from the menu and to have speech recognition activated (and listening) or deactivated.

Voice-menu enumerator. Applications use the enumerator to obtain the list of Voice menus that are stored in the internal database and to determine which of these menus are active. Most applications will not use the Voice-menu enumerator.

For more information look in the "Voice Commands" section.


Voice Text

Voice Text is the high-level interface for text-to-speech.


Sample

Adding voice text is fairly simple. An application has to make only the following modifications:

CoCreateInstance (CLSID_VTxt, NULL, CLSCTX_LOCAL_SERVER, IID_IVoiceText, &pIVoiceText);

Calling CoCreateInstance creates an instance of the voice text object.

pIVoiceText->Register ("", "Demo Application", NULL, IID_IVTxtNotifySink, NULL, NULL);

Applications have to call Register so the voice text knows the name of the application and to what audio device the speech will be played, since some applications will be telephone aware. Also, an application can provide a notification sink so that it's alerted when speaking starts or stops, but this isn't necessary.

The next step is to send out text to be spoken.

pIVoiceText->Speak ("Hello world.", 0, NULL);

Finally, when the application is finished using voice text it releases the object.

pIVoiceText->Release();


Objects

There are two objects involved in doing text-to-speech:

Voice-Text Object. This is the object that handles speech and the only one that the application deals with.

Notification Sink. The notification sink code is optional and is supplied by the application. The voice-text object calls methods in this object when audio starts or stops playing, or with mouth-animation cues.

Each object has a set of interfaces that it supports. These are listed in the sections below.

For more information look in the "Voice Text" section.


How to Implement Using the Low-Level Interfaces

Of course, not everyone is satisfied by the high-level speech interfaces because they don't always supply enough flexibility, sacrificing some flexibility and control for ease of implementation. Therefore, the API also has lower-level interfaces and objects that require a bit more work but which provide a lot more control over the speech recognition and text-to-speech processes. The high-level modules (voice commands and voice text) actually use these same low-level interfaces and objects to do speech recognition.


Low-Level Speech Recognition

When an application uses the low-level speech-recognition interfaces, it is talking directly to the engine. This provides the application with much more control but also involves more work. Because the low-level API is more complex, this article won't go into detail about how the engine object is used. However, an architectural overview will give you an idea of the processes involved.

The low-level API consists of many more objects than the high-level interface (voice commands). Here's how the process works:

The application determines where the speech recognition's audio should come from and creates an audio-source object through which the engine acquires the data. Microsoft supplies an audio-source object that gets its audio from the multimedia wave-in device, but the application is able to use customized audio sources, such as an audio source which acquires audio from a .WAV file or a specialized hardware device.

The application, through a speech-recognition enumerator object (not shown here, but provided by Microsoft), locates a speech-recognition engine that it wants to use. It then creates an instance of the engine object and passes it the audio-source object.

The engine object has a dialog with the audio-source object to find a common data format for the digital audio data, such as pulse code modulation (PCM). Once a suitable format is established, the engine creates an audio-source notification sink that it passes to the audio-source object. From then on, the audio-source object submits digital audio data to the engine through the notification sink.

The application can then register a main notification sink that receives grammar-independent notifications, such as whether or not the user is speaking.

When it is ready, the application creates one or more grammar objects. These are similar to the Voice-menu object in voice commands but more flexible in syntax recognition.

To find out what words the user spoke; the application creates a grammar-notification sink for every grammar object. When the grammar object recognizes a word or phrase, or has other grammar-specific information for the application, it calls functions in the grammar-notification sinks. The application responds to the notifications and takes whatever actions are necessary.

Typically, when a grammar object recognizes speech, it sends the grammar-notification sink a string indicating what was spoken. However, the engine may know a lot more information than this, such as alternative phrases that may have been spoken, timing, or even who is speaking the phrase. An application can find this information by requesting a results object for the phrase and interrogating the results object for more information.

For more information look in the "Low-Level Speech Recognition API" section.


Low-Level Text-to-Speech

When an application uses the low-level text-to-speech interfaces, it is talking directly to the engine. As with speech recognition, this provides the application with a lot more control, but requires more work of it. Because the low-level API is more complex, this document won't go into detail about how objects are used. However, an architectural overview follows.

The low-level API consists of many more objects than the high-level API (voice text). Here's how the process works:

The application determines where the text-to-speech audio should be sent and creates an audio-destination object through which the engine sends the data. Microsoft supplies an audio-destination object that sends its audio to the multimedia wave-out device, but the application may use customized audio destinations, such as an audio destination that writes to a .WAV file.

The application, through a text-to-speech enumerator object (not shown here, but provided by Microsoft), locates a text-to-speech engine and voice that it wants to use. It then creates an instance of the engine object and passes it the audio-destination object.

The engine object has a dialog with the audio-destination object to find a common data format for the digital audio. Once an acceptable format is established, the engine creates an Audio-Destination Notification Sink that it passes to the audio-destination object. From then on, the audio-destination object submits status information to the engine through the notification sink.

The application can then register a Main Notification Sink that receives buffer-independent notifications, such as whether the synthesized voice is speaking and mouth positions for animation.

When it is ready, the application passes one or more text buffers down to the engine. These will be queued up and then spoken (to the audio destination) by the engine.

To find out what words are currently being spoken, the application can create a Buffer Notification Sink for every buffer object. When the engine speaks a word, reaches a bookmark, or some other event occurs, it calls functions in the Buffer Notification Sinks. The notification sink is released when the buffer is finished being read.

For more information look in the "Low-Level Text-to-Speech Section."


What is Text-to-Speech?

Text-to-speech is a process through which text is rendered into digital audio and "spoken." Most text-to-speech engines can be categorized by the method that they use to translate phonemes into audible sound.

Concatenated Word. Although Concatenated Word systems are really synthesizers, the are one of the most commonly used text-to-speech systems around. In a concatenated word engine, the application designer provides recordings for phrases and individual words. The engine pastes the recordings together to speak out a sentence or phrase. If you use voice-mail then you've heard one of these engines speaking, "[You have] [three] [new messages]." The engine has recordings for "You have", all of the digits, and "new messages".

Synthesis. A text-to-speech engine that uses synthesis generates sounds like those created by the human vocal cords and applies various filters to simulate throat length, mouth cavity, lip shape, and tongue position. The voice produced by current synthesis technology tends to sound less human than a voice produced by diphone concatenation, but it is possible to obtain different qualities of voice by changing a few parameters.

Diphone Concatenation. A text-to-speech engine that uses diphone concatenation links short digital-audio segments together and performs intersegment smoothing to produce a continuous sound. Each diphone consists of two phonemes, one that leads into the sound and one that finishes the sound. For example, the word "hello" consists of these phonemes: h eh l œ. The corresponding diphones are silence-h h-eh eh-l l-œ œ-silence.

Diphones are acquired by recording many hours of a human voice and painstakingly identifying the beginning and ending of phonemes. Although this technique can produce a more realistic voice, it takes a considerable amount of work to create a new voice and the voice is not localizable because the phonemes are specific to the speaker's language.


Why Use Text-to-Speech?

Text-to-speech should be used by an application when it has a need to audibly communicate information to the user and when digital audio recordings are inadequate. Generally, text-to-speech is better than audio recordings when:

Audio recordings are too large to store on disk or expensive to record.

Audio recording is impossible because the application doesn't know ahead of time what it will speak.


More Specific Reasons for Implementing Speech

Text-to-speech also offers a number of benefits. In general, text-to-speech is most useful for short phrases or for situations when prerecording is not practical. Text-to-speech has the following practical uses:

· Reading dynamic text. Text-to-speech is useful for phrases that vary too much to record and store using all possible alternatives. For example, speaking the time is a good use for text-to-speech, because the effort and storage involved in recording and concatenating all possible times is manageable.

· Proofreading. Audible proofreading of text and numbers helps the user catch typing errors missed by visual proofreading.

· Conserving storage space. Text-to-speech is useful for phrases that would occupy too much storage space if they were prerecorded in digital-audio format.

· Notifying the user of events. Text-to-speech works well for informational messages. For example, to inform the user that a print job is complete, an application could say "Printing complete" rather than displaying a message box and requiring the user to click OK. (This should be used for noncritical notifications in case the user turns the computer's sound off or is out of hearing range.)

· Providing audible feedback. Text-to-speech can provide audible feedback when visual feedback is inadequate or impossible. For example, the user's eyes might be busy with another task, such as transcribing data from a paper document. Users that have low vision may rely on text-to-speech as their sole means of feedback from the computer.


Potential Uses By Application Category

The specific use of text-to-speech will depend on the application. Here are some sample ideas and their uses:


Games and Edutainment

Text-to-speech is useful in games and edutainment to allow the characters in the application to really "talk" to the user instead of displaying speech baloons. Of course, it's also possible to have recordings of the speech. An application would use text-to-speech instead of recordings in the following cases:

· It's always possible to use concatenated word/phrase text-to-speech to replace recordings since it really just uses recordings of phrases or words. The application designer could easily have recordings of whole sentences played by the text-to-speech engine when a specific sentence string is passed in. If the application did this, it could easily used synthesized text-to-speech instead.

Synthesized text-to-speech inevitably sounds unnatural and wierd. However, it's very good for character voices that are supposed to be robots, aliens, or maybe even foreigners.

Of course, if the application cannot afford to have recordings of all the possible dialogs or if the dialogs cannot be recorded ahead of time then text-to-speech is the only alternative.


Telephony

Look in the "Telephony" section of the Microsoft Speech API documentation for a full description of telephony.


Current Speech Technology


Hardware and Software Requirements

A speech application requires certain hardware and software on the user's computer to run. Not all computers have the memory, speed, or speakers required to support speech, so it is a good idea to design the application so that speech is optional.

These hardware and software requirements should be considered when designing a speech application:

· Processor speed. Text-to-speech engines currently on the market typically require a 486/33 (DX or SX) or faster processor.

· Memory. On the average, text-to-speech uses about 1 MB of RAM.

· Sound card. Almost any sound card will work for speech recognition and text-to-speech, including Sound Blaster™, Media Vision™, ESS Technology, cards that are compatible with the Microsoft® Windows Sound System, and the audio hardware built into multimedia computers.

· Speakers. The user can choose between wearing headphones or using freestanding speakers. Headphones are useful in office cubicles. Some companies manufacture a combination headphone and microphone that can also be used for telephone conversations.

· Operating system. The Microsoft Speech application programming interface (API) requires either Windows 95 or Windows NT version 3.5.

· Text-to-speech engine. Text-to-speech software must be installed on the user's system. Many new audio-enabled computers and sound cards are bundled with speech-recognition and text-to-speech engines. As an alternative, many engine vendors offer retail packages for speech recognition or text-to-speech, and some license copies of their engines.

For a list of engine vendors that support the Speech API, see the ENGINE.DOC file included with the Speech Software Development Kit.


Limitations


Text-to-Speech Voice Quality

Most text-to-speech engines can render individual words successfully. However, as soon as the engine speaks a sentence, it is easy to identify the voice as synthesized because it lacks human prosody — that is, the inflection, accent, and timing of speech. For this reason, most text-to-speech voices are difficult to listen to and require concentration to understand, especially for more than a few words at a time.

Some engines allow an application to define text-to-speech segments with human prosody attached, making the synthesized voice much clearer. The engine provides this capability by prerecording a human voice and allowing the application developer to transfer its intonation and speed to the text being spoken.

In effect, this acts as a highly effective voice compression algorithm. Although text with prosody attached requires more storage than ASCII text (1K per minute compared to a few hundred bytes per minute), it requires considerably less storage than prerecorded speech, which requires at least 30K per minute. These factors also influence the quality of a synthesized voice:

· Emotion. Although many text-to-speech engines can parse and interpret punctuation, such as periods, commas, exclamation points, and questions, none of the engines that are currently available can render the sound of human emotion.

· Mispronunciation. Text-to-speech engines use a set of pronunciation rules to translate text into phonemes. This is fairly easy for languages with phonetic alphabets, but it is very difficult for the English language, especially if last names are to be pronounced correctly. (Pronunciation rules seldom fail on common words, but they almost always fail on odd or foreign-sounding names.)

If an engine mispronounces a word, the only way that the user can change the pronunciation is by entering either the phonemes, which is not an easy task, or by choosing a series of "sound-alike" words that combine to make the correct pronunciation.


Creating and Localizing Text-to-Speech Voices

Creating a new voice for an engine that uses synthesis can be done relatively quickly by altering a few parameters of an existing voice. However, although the pitch and timbre of the new voice is different, it uses the same speaking style and prosody rules as the existing voice.

Creating a new voice for a text-to-speech engine that uses diphone concatenation can take a considerable amount of work, because the diphones must be acquired by recording a human voice and identifying the beginning and ending of phonemes, which are specific to the speaker's language.

Whether a text-to-speech engine uses synthesis or diphone concatenation, localizing an engine for a new language requires a skilled linguist to design pronunciation and prosody rules and reprogram the engine to simulate the sound of the language's phonemes. In addition, diphone-concatenation systems require a new voice to be constructed for the new language. As a consequence, most engines support only five to ten major languages.


Application Design Considerations


Using Text-to-Speech for Short Phrases

An application should use text-to-speech only for short phrases or notifications, not for reading long passages of text. Because listening to a synthesized voice read more than a few sentences requires more concentration, a user can become irritated.


Presenting Important Information Visually

An application should communicate critical information visually as well as audibly, and it should not rely solely on text-to-speech to communicate important information. The user can miss spoken messages for a variety of reasons. For example, the user may not have speakers or headphones attached to the computer, the user may be distracted or out of earshot when the application speaks, or the user may simply have turned off text-to-speech.


Avoiding a Mix of Text-to-Speech and Recorded Voice

The synthesized voice provided by even the best text-to-speech engine is noticeably different from that provided by a digital-audio recording. Mixing the two in the same utterance can be disturbing to the user (and usually makes the text-to-speech voice sound worse by comparison).

For example, to have an application speak "The number is 56,738," you should not prerecord "The number is" and use text-to-speech to speak the numbers. You should either prerecord everything or use text-to-speech for everything.


Using Digital-Audio Recording Before Text-to-Speech

Prerecorded voice always sounds better than text-to-speech. If all of the phrases that the application speaks can be prerecorded and will not exceed the available storage space, you should use that option instead. In any case, you should not mix prerecorded voice with text-to-speech voice, because this emphasizes the synthesized quality of the text-to-speech voice.


Making Text-to-Speech Optional

An application should always allow the user to turn off text-to-speech. Some users work in environments in which a talking computer may distract coworkers or in which privacy may be important. Also, a user may simply dislike the sound of a synthesized voice.


Where the Engine Comes From

Of course, for text-to-speech to work on an end user's PC the system must have a text-to-speech engine installed on it. The application has two choices:

The application can come bundled with a text-to-speech engine and install it itself. This guarantees that text-to-speech will be installed and also guarantees a certain level of quality from the text-to-speech. However, if an application does this, royalties will need to be paid to the engine vendor.

Alternatively, an application can assume that the text-to-speech engine is already on the PC or that the user will purchase one if they wish to use text-to-speech. The user may already have text-to-speech because many PCs and sound cards will come bundled with an engine. Alternatively, the user may have purchased another application that included an engine. If the user has no text-to-speech engine installed then the application can tell the user that they need to purchase a text-to-speech engine and install it; Several engine vendors offer retail versions of their engines.


About Voice Text

Voice text is a feature that provides simple text-to-speech capabilities for Windows-based applications. You use voice text to convert words in a character string into speech played over the computer speaker or a telephone line.

Implementing voice text in an application involves the use of the voice-text object. The object manages the interaction between the text-to-speech engine that creates the speech and the computer speaker or telephone line that serves as the destination for the speech. Before an application can begin using voice text, it must create an instance of the voice-text object and register itself with the object. This section describes the voice-text object and explains how to use it to implement voice text in an application.

Some users do not wish to have the computer talk or to talk to the computer. For these users, Microsoft provides a speech control panel application that sets a registry entry indicating the user's preferences. An application can check this entry before trying to load its voice commands code, and save memory by not loading the code if it is not needed. For more information about the registry entry, see the section, "Common Elements of the Speech Application Programming Interface.".

The voice-text object is exposed as an OLE automation object, allowing it to be accessed and controlled through Visual Basic, Visual Basic for Applications (VBA), and other automation-aware application macro languages. You can take advantage of the OLE automation capabilities of the voice-text object to add voice text capabilities to your Visual Basic applications. For more information, see Appendix E, "Voice Commands, Text-to-Speech, and OLE Automation."


Object Creation

To create a voice-text object, an application calls the CoCreateInstance function with the CLSID_VTxt class identifier and the IID_IVoiceText interface identifier. An application must create a voice-text object for each audio destination that it needs to use. For example, if the application needs to play voice text over a telephone line as well as the computer speakers, it should create two voice-text objects.

After creating a voice-text object, CoCreateInstance returns a pointer to the IVoiceText interface for the object. An application uses the interface to register itself with the text-to-speech engine, to send text to the text-to-speech engine, and to control how the engine speaks the text.

Before an application can play voice text, it must register itself by calling the IVoiceText::Register member function. By registering, the application tells the engine which site to use as the destination of voice text and passes the voice-text object a pointer to the application's voice-text notification interface. The object calls the notification interface to notify the application about events related to voice text.


Voice-Text Site

A text-to-speech engine plays voice text through a voice-text site on the user's machine. A voice-text site consists of an audio-output device, such as the computer speaker or a telephone line, and a text-to-speech mode. An engine typically provides several different modes for playing speech, each representing a different voice. Each voice plays text in a different speaking style or personality.

An application must create a separate voice-text object for each site that it needs to use. To use a site other than the default (the computer speaker and the default mode of the engine), the application must obtain the identifier of the desired audio-output device and the globally unique identifier (GUID) of the desired mode, and then pass that information to the voice-text object when registering.

An application uses the waveOutGetNumDevs and waveOutGetDevCaps multimedia functions to obtain the device identifier of an audio-output device. To determine the available text-to-speech modes, an application creates a text-to-speech enumerator. The enumerator returns information about the modes provided by all engines in the user's computer, including the GUID of each mode.

For more information about multimedia and multimedia functions, see the documentation included with the Microsoft Win32 Software Development Kit (SDK). For information about enumerating the modes of a text-to-speech engine, see the section, "Low-Level Text-to-Speech API."


Site Attributes

Every voice-text site has a set of attributes that affect the interaction between the text-to-speech engine and the audio-output device. An application can query and set a site's attributes by using the IVTxtAttributes interface provided by the voice-text object. To get a pointer to the IVTxtAttributes interface, call the IVoiceText::QueryInterface member function with the IID_IVTxtAttributes interface identifier.

Because multiple applications can share the same voice-text site, changing a site attribute affects all applications registered to use the site, not just the application making the change. To receive notifications of attribute changes, an application must specify the VTXTF_ALLMESSAGES value when calling the IVoiceText::Register member function. Note, however, that attribute settings are saved between uses of the site, even if the computer is shut off in the meantime.


Audio-Output Device

An application can use the IVTxtAttributes::DeviceSet member function to set the audio-output device for a site. To change a device, an application must specify the device identifier of the new device. You can use the waveOutGetNumDevs and waveOutGetDevCaps multimedia functions to obtain the identifiers of the available audio-output devices in the system. The IVTxtAttributes::DeviceGet member function retrieves the device identifier of the current audio-output device for the site.


Text-to-Speech Mode

A text-to-speech engine typically provides an assortment of modes that it can use to play speech in different languages, dialects, and speaking styles. Each voice-text site uses a single text-to-speech mode. An application can set the mode for a voice-text site by specifying the GUID of the new site in a call to the IVTxtAttributes::TTSModeSet member function. To retrieve the GUID of the mode that the site is currently using, an application calls the IVTxtAttributes::TTSModeGet member function.


Enabled State

An application can enable or disable text-to-speech output for a voice-text site. Disabling a site prevents the engine from playing text through the site. An application uses the IVTxtAttributes::EnabledSet member function to enable or disable a voice-text site and the IVTxtAttributes::EnabledGet member function to discover if the site is currently enabled or disabled.

The user can use the Speech control panel application to turn the system's speech capabilities on and off. When the system's speech capabilities are turned on or off, the WM_SPEECHSTARTED or WM_SPEECHENDED message is sent to all top-level windows in the system. An application can use these messages to determine when to enable or disable its voice-text capabilities.


Average Talking Speed

A text-to-speech engine can speak text at various speeds. An application can set the engine's average talking speed by using the IVTxtAttributes::SpeedSet member function and can retrieve the current average speed by using the IVTxtAttributes::SpeedGet member function. The average talking speed is measured in words per minute.

To discover whether an engine is currently speaking text for a voice-text site, an application can call the IVTxtAttributes::IsSpeaking member function.


Speech

Once you have registered a voice-text site and notification interface, you can play text over the site by using the IVoiceText::Speak member function. Speak places a text string into the playback queue for the site. The engine removes the string from the queue and converts the words in the string into speech, using the space characters and punctuation in the string to determine the start and end of words and the inflections to use when speaking the words.

When calling Speak, an application specifies the address of a buffer that contains the text to speak, a pair of flags that indicate the type and priority level of the text, and any optional control tags. The text-to-speech engine uses the type flag and the control tags to set various qualities of the speaking voice. It uses the priority-level flag to determine how quickly to remove the text from the buffer and establish whether to preempt text that is already in the queue or is currently being spoken. For more information about control tags, see Appendix A, "Text-to-Speech Control Tags."

Once speech is started, all words in the given string are spoken sequentially unless the application stops the speech by using the IVoiceText::AudioPause or IVoiceText::StopSpeaking member function. If AudioPause is used, the application can resume the speech at the point it stopped by using the IVoiceText::AudioResume member function. If StopSpeaking is used, the speech cannot be resumed because all remaining words in the text are deleted from the playback queue.

An application can skip over words or replay portions of the speech by using the IVoiceText::AudioFastForward and IVoiceText::AudioRewind member functions.


Engine Dialog Boxes

A text-to-speech engine provides a set of dialog boxes that enable the user to configure the engine. By using the IVTxtDialogs interface provided by the voice-text object, an application can direct the engine to display its dialog boxes. The controls and capabilities of the dialog boxes are determined solely by the engine. Before an application can call any IVTxtDialogs member function, it must first register with the engine by calling the IVoiceText::Register member function.

When an application calls a member function of the IVTxtDialogs interface to display a dialog box, it specifies the handle of the parent window for the dialog box. By specifying NULL instead of a window handle, the application can determine whether the engine supports the given dialog box. The engine returns the NOERROR value if it supports the dialog box or the VTXTERR_NOTSUPPORTED error value if it does not. The application also specifies the title of the dialog box by specifying the address of a title string. If the application specifies NULL instead of a title string address, the engine uses the default title.

If a voice-navigation application such as Microsoft Voice is running, an application does not need to direct an engine to display its dialog boxes because the voice-navigation application gives the user access to them. For more information about voice-navigation applications, see the section, "Common Elements of the Speech Application Programming Interface.".


About Dialog Box

An engine's About dialog box contains information that identifies the text-to-speech engine and its manufacturer and contains the manufacturer's copyright notice. An application displays the About dialog box by using the IVTxtDialogs::AboutDlg member function.


General Dialog Box

The General dialog box gives the user control of the general features of the text-to-speech engine. The dialog box may also include controls that allow the user to open other dialog boxes provided by the engine. An application displays the General dialog box by using the IVTxtDialogs::GeneralDlg member function. Depending on the engine, the General dialog box may allow the user to perform the following tasks:

· Set the maximum amount of disk space allotted for text-to-speech output. (Typically, setting the disk space is relevant only for rendering concatenated wave files.)

· Set the speaker preferences supported by the engine.


Lexicon Dialog Box

A text-to-speech engine maintains a database of pronunciations called a pronunciation lexicon. The engine uses the pronunciation lexicon to determine how to pronounce words and phrases. The IVTxtDialogs::LexiconDlg member function displays a Lexicon dialog box that allows the user to view and edit the pronunciation lexicon. For example, the user can edit the phonetics of mispronounced words so that the engine pronounces them properly.


Translation Dialog Box

The Translation dialog box allows the user to control the translation techniques for symbols, currencies, abbreviations, and numbers. For example, the user can specify whether to speak "34" as "thirty-four," "three four," or the old English "four and thirty." This dialog box also allows the user to control the translation of abbreviations and case sensitivity and to manage how both affect the prosody of spoken text. An application displays the Translation dialog box by using the IVTxtDialogs::TranslateDlg member function.


Notification Interface

When an application calls the IVoiceText::Register member function to register its voice-text site, the application must also provide a pointer to its IVTxtNotifySink interface. The voice-text object calls this interface to notify the application of events related to voice text. To receive notifications, the application must create a Component Object Model (COM) object that supports the IVTxtNotifySink interface. The application developer determines how an application responds to notifications.

The voice-text object sends a variety of notifications to the IVTxtNotifySink interface. When calling Register, an application indicates whether the object should send all notifications or just a subset. A voice-text object can send the following notifications.

Notification | Description
AttribChanged | Sent when an application changes an attribute for a voice-text site. The notification includes a flag that identifies the attribute that changed.
Speak | Sent when an application puts text into the playback queue for the voice-text site.
SpeakingDone | Sent when speaking is finished and no text remains in the playback queue.
SpeakingStarted | Sent when the engine has started speaking text for a site.
Visual | Sent when the engine is speaking text for a site. The notification identifies the phoneme that the engine is currently speaking and the type of sentence that contains the phoneme (for example, a question, command, or statement).


Using Voice Text

This section contains examples demonstrating how to initialize an application to use voice text and how to place a text string into the playback buffer for the voice-text object.


Initializing an Application for Voice Text

To use voice text, you need to create a voice-text object and register your application with the object. You create a voice-text object by calling the CoCreateInstance function with the CLSID_VTxt class identifier and the IID_IVoiceText interface identifier. You must create a separate voice-text object for each site that your application needs to use.

CoCreateInstance returns a pointer to the IVoiceText interface for the voice-text object. Before an application can perform other voice-text tasks, it must register itself by calling the IVoiceText::Register member function. Register specifies the site that the object uses for audio output, and passes the address of the application's voice-text notification interface to the voice-text object.

The following example shows how to create a voice-text object and register the application with the object.

// BeginOLE - initializes OLE, creates a voice-text object, and

// registers the application with the object.

// Returns TRUE if successful or FALSE otherwise.

//

// Global variables:

// gpIVTxt - address of IVoiceText interface

// gpIVTxtAttr - address of IVTxtAttributes interface

// gpVTxtNotifySink - address of IVTxtNotifySink interface

BOOL BeginOLE()

{

HRESULT hRes;

// Initialize OLE.

if (FAILED(CoInitialize(NULL)))

return ReleaseInterfaces("CoInitialize() failed.");

// Create a voice-text object.

if (CoCreateInstance(CLSID_VTxt, NULL, CLSCTX_LOCAL_SERVER,

IID_IVoiceText, (LPVOID *) &gpIVTxt) != S_OK)

return ReleaseInterfaces(

"Error in CoCreateInstance for voice-text interface." );

// Get the address of the voice-text attributes interface.

hRes = gpIVTxt->QueryInterface(IID_IVTxtAttributes,

(LPVOID FAR *) &gpIVTxtAttr);

if (FAILED(hRes))

return ReleaseInterfaces(

"Failed to get voice-text attributes interface.");

// Create and register the voice-text notification sink.

gpVTxtNotifySink = new CIVTxtNotifySink;

if (gpVTxtNotifySink == NULL)

return ReleaseInterfaces(

"Out of memory for voice-text notification object.");

hRes = gpIVTxt->Register(NULL, "SRClock", gpVTxtNotifySink,

IID_IVTxtNotifySink, VTXTF_ALLMESSAGES, NULL);

if (FAILED(hRes))

return ReleaseInterfaces(

"Failed to register voice-text notification sink.");

return TRUE;

}

// ReleaseInterface - displays an error message and releases the

// voice-text object.

// Returns FALSE.

// szMsg - text of error message

//

// Global variable:

// gpIVTxt - address of voice-text interface

BOOL ReleaseInterfaces(const char *szMsg)

{

// If a message is specified, display it.

if (*szMsg)

MessageBox(NULL, szMsg, NULL, MB_OK);

if(gpIVTxt)

gpIVTxt->Release();

gpIVTxt = NULL;

return FALSE;

}


Speaking Text

To cause the text-to-speech engine to speak voice text , an application must place the text into the playback queue by calling the IVoiceText::Speak member function. The following example puts text into the playback queue for the voice-text object, indicating that the text is a normal-priority statement.

// SpeakString - puts a string in the playback queue.

// Returns the result of calling IVoiceText::Speak.

// szSpeak - address of buffer containing the text to speak

HRESULT SpeakString(char *szSpeak)

{

HRESULT hRes;

if( *szSpeak == NULL )

return 0;

// Put the text in the playback buffer.

hRes = gpIVTxt->Speak(szSpeak,

VTXTST_QUESTION | VTXTSP_NORMAL, NULL);

return hRes;

}


Processing Voice-Text Notifications

The voice-text object calls an application's IVTxtNotifySink interface to inform the application of voice-text events so that the application can respond to them. To receive notifications, the application must create a COM object that supports IVTxtNotifySink and must pass the address of the interface to the voice-text object when calling the IVoiceText::Register member function.

The IVTxtNotifySink interface consists of a set of member functions that correspond to voice-text events. When an event occurs on the voice-text site that the application is using, the voice-text object calls the member function that corresponds to the event. The application developer determines how the member functions process the event notifications.

The following example shows how to define an object class that implements the IVTxtNotifySink interface.

class CIVTxtNotifySink : public IVTxtNotifySink {

private:

HWND m_hwnd;

public:

CIVTxtNotifySink(void);

~CIVTxtNotifySink(void);

// IUnknown members

STDMETHODIMP QueryInterface (REFIID, LPVOID FAR *);

STDMETHODIMP_(ULONG) AddRef(void);

STDMETHODIMP_(ULONG) Release(void);

// IVTxtNotifySink members

STDMETHODIMP AttribChanged (DWORD;

STDMETHODIMP Speak (PSTR, PSTR, DWORD);

STDMETHODIMP SpeakingDone (void);

STDMETHODIMP SpeakingStarted (void);

STDMETHODIMP Visual (WCHAR, CHAR, DWORD, PTTSMOUTH);

};

typedef CIVTxtNotifySink * PCIVTxtNotifySink;


Objects

This section describes, in alphabetical order, the object, interfaces, member functions, structure, and error values used for voice text.


Voice-Text Object

Purpose

Manages the use of voice text by all applications that use a voice-text site. The voice-text object represents a voice-text site on the user's computer. A voice-text site consists of an audio device, such as a computer speaker or telephone line, and a text-to-speech mode provided by an engine. Internally, a voice-text site consists of two low-level text-to-speech objects: a text-to-speech engine object and an audio destination.

Creation

An application creates a voice-text object by calling the CoCreateInstance function with the CLSID_VTxt class identifier and the IID_IVoiceText interface identifier. CoCreateInstance returns a pointer to the IVoiceText interface for the voice-text object.

Interaction

The voice-text object calls an application's IVTxtNotifySink interface to notify the application of speech-related events. An application must create a COM object that supports the IVTxtNotifySink interface and must pass the address of the interface to the voice-text object when calling the IVoiceText::Register member function. The application developer determines how the functions in the interface respond to notifications.

A text-to-speech mode can be identified by using a text-to-speech enumerator, which enumerates the text-to-speech modes provided by all engines on the user's computer. For more information about text-to-speech enumerators, see the section, "Low-Level Text-to-Speech API."

Interfaces

The voice-text object supports the following interfaces.

· IVTxtAttributes. Controls attributes of a voice-text site, such as the audio device, the speaking speed, the text-to-speech mode, and whether text-to-speech is enabled for the site.

· IVTxtDialogs. Displays Windows dialog boxes that allow a user to configure the text-to-speech engine. These dialog boxes are defined by the engine.

· IVoiceText. Registers an application to use voice text on a site and controls the playback of text.


Voice-Text Quick-Look

This sections lists and briefly describes the voice-text interfaces and their member functions.


IUnknown

This interface is supported by all speech objects. IUnknown member functions are also included in all other speech interfaces.

Member function | Description
AddRef | Increments a reference count for every new copy of an interface pointer to a specified interface on a particular object.
QueryInterface | Returns a pointer to a specified interface on a particular object so that an application can query an object to determine what interface it supports.
Release | Decrements the reference count for the specified interface on a particular object.


IVoiceText

This interface is supported by the voice-text object.

Member function | Description
AudioFastForward | Advances audio playback by approximately one sentence or phrase.
AudioPause | Pauses text-to-speech output.
AudioResume | Resumes text-to-speech output after it has been paused.
AudioRewind | Backs up audio playback by approximately one sentence or phrase.
Register | Registers an application to use voice text on a site.
Speak | Starts playing the specified text.
StopSpeaking | Halts text that is currently being spoken and flushes all pending text from the queue.


IVTxtAttributes

This interface is supported by the voice-text object.

Member function | Description
DeviceGet | Returns the device ID of the wave-out audio device used for the site.
DeviceSet | Changes the wave-out audio device for the site.
EnabledGet | Indicates whether or not voice text is enabled for the site.
EnabledSet | Enables or disables all voice text for a site.
IsSpeaking | Indicates whether text is currently being spoken by the site.
SpeedGet | Gets the current average talking speed for a site, in words per minute.
SpeedSet | Sets the current average talking speed for a site, in words per minute.
TTSModeGet | Gets the GUID of the current text-to-speech mode for the site.
TTSModeSet | Sets the text-to-speech mode for the site.


IVTxtDialogs

This interface is supported by the voice-text object.

Member function | Description
AboutDlg | Displays an About dialog box that identifies the text-to-speech engine and contains the copyright notice.
GeneralDlg | Allows general control of the text-to-speech engine and should invoke some of the other dialog boxes.
LexiconDlg | Allows users to display and edit their dictionary entries.
TranslateDlg | Allows the end-user to control symbols, currencies, abbreviations, and number-translation techniques, as well as translation of abbreviations and case sensitivity, and how both affect the prosody of text.


IVTxtNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
AttribChanged | Notifies applications that a site attribute has changed.
Speak | Notifies applications on a site that an application has added voice text to the queue.
SpeakingDone | Notifies all applications on a site that speaking is finished and no voice text remains in the queue.
SpeakingStarted | Notifies all applications on a site that speaking has started.
Visual | Notifies applications on a site which phoneme is currently being spoken.


Voice-Text Structures

Structure | Description | Member functions
TTSMOUTH | Indicates the mouth position that corresponds to a phoneme. | IVTxtNotifySink::Visual
VTSITEINFO | Specifies an audio device, a text-to-speech mode, and the talking speed for a voice-text site, and indicates whether voice text is enabled or disabled for the site. | IVoiceText::Register


Interfaces and Member Functions

Voice-text interfaces contain member functions that allow an application to implement simple text-to-speech— the audio equivalent of the printf function. They include the following interfaces.

Interface | Description
IVTxtAttributes | Controls the attributes of a voice-text site.
IVTxtDialogs | Displays Windows dialog boxes that allow the user to configure the text-to-speech engine used by a site.
IVTxtNotifySink | Used by a voice-text object to notify an application that speaking has started or stopped or that an attribute has been changed for the site.
IVoiceText | Registers a voice-text site and controls the playback of spoken text.

This section describes, in alphabetical order, the interfaces and functions that support voice text.

In addition to the interfaces described in this section, each object supports IUnknown, which is a common interface supported by all OLE component objects. For more information about IUnknown, see the section, "Common Elements of the Speech Application Programming Interface."


IVTxtAttributes Interface

The IVTxtAttributes interface allows an application to retrieve and set the attributes of a voice-text site. The application must call the IVoiceText::Register member functions before it can call any of the member functions in the IVTxtAttributes interface.

A voice-text application may not need to use IVTxtAttributes member functions if a voice-navigation application is installed on the user's computer. The voice-navigation application typically monitors and controls the attributes of a voice-text site.

Changing an attribute of a site affects all applications using the site. Settings are saved between uses of the site, even if the user shuts down the computer in the meantime.

The IVTxtAttributes interface is supported by the voice-text object. It has these member functions:

DeviceGet | SpeedGet
DeviceSet | SpeedSet
EnabledGet | TTSModeGet
EnabledSet | TTSModeSet
IsSpeaking |


IVTxtAttributes::DeviceGet

HRESULT DeviceGet(DWORD *pdwDeviceID);

Retrieves the device identifier of the audio-output device that is used for a voice-text site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

pdwDeviceID

[out] Address of a variable that receives the device identifier of the audio-output device currently used by the site.

The device identifier for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVTxtAttributes::DeviceSet

HRESULT DeviceSet(DWORD dwDeviceID);

Sets the audio-output device that is used for a voice-text site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

dwDeviceID

[in] Device identifier for the new audio-output device to use for the site.

An application can obtain a list of devices by calling the waveOutGetNumDevs and waveOutGetDevCaps multimedia functions.

The device identifier for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the device.


IVTxtAttributes::EnabledGet

HRESULT EnabledGet(DWORD *dwEnabled);

Discovers whether voice text is enabled for a voice-text site. If voice text is disabled, no text-to-speech is played over the site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

dwEnabled

[out] TRUE if voice text is enabled for the site or FALSE if it is disabled.

Enabling or disabling voice text for a site affects all applications using a voice-text site.

Typically, an application disables voice text because the user does not want the computer to speak. You should involve the user when enabling or disabling voice text.

The enabled state for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVTxtAttributes::EnabledSet

HRESULT EnabledSet(DWORD dwEnabled);

Enables or disables voice text for a voice-text site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

dwEnabled

[in] TRUE to enable voice text or FALSE to disable it.

The enabled state for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the enabled state.


IVTxtAttributes::IsSpeaking

HRESULT IsSpeaking(BOOL *pfSpeaking);

Indicates whether text is currently being spoken by a voice-text site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

pfSpeaking

[out] Address of a variable that receives the current speaking status. The variable receives TRUE if the text-to-speech engine is speaking or FALSE if it is silent.

IsSpeaking corresponds to the ITTSNotifySink::AudioStart and AudioStop notifications. However, the voice text object does not send data resulting from multiple calls to the IVoiceText::Speak member function directly to the text-to-speech engine. Instead, the object keeps data from each call in a separate buffer so that the VTXTSP_HIGH and VTXTSP_VERYHIGH priority strings can be inserted into the queue at the proper positions. For example, a VTXTSP_VERYHIGH priority string may interrupt a high or normal priority string. The interrupted string resumes after the very high priority string has finished. As a result of this implementation, IsSpeaking returns FALSE for a short time between the end of one buffer in the queue and the start of the next buffer, because audio output has been temporarily suspended.


IVTxtAttributes::SpeedGet

HRESULT SpeedGet(DWORD *pdwSpeed);

Retrieves the current average talking speed for a voice-text site, in words per minute.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

pdwSpeed

[out] Address of a variable that receives the talking speed for a voice-text site.

The talking speed for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVTxtAttributes::SpeedSet

HRESULT SpeedSet(DWORD dwSpeed);

Sets the average talking speed for a voice-text site, in words per minute.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

dwSpeed

[in] New talking speed for the site. An application can specify TTSATTR_MINSPEED or TTSATTR_MAXSPEED for the minimum or maximum allowable value.

The talking speed for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the speed.


IVTxtAttributes::TTSModeGet

HRESULT TTSModeGet(GUID *pgVoice);

Retrieves the GUID of the current text-to-speech mode for a voice-text site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

pgVoice

[out] Address of a variable that receives the GUID assigned to the text-to-speech mode.

A text-to-speech engine typically provides an assortment of text-to-speech modes that can be used to play speech in difference voices. A voice-text site uses a single text-to-speech mode, represented internally by a low-level engine object.

The text-to-speech mode for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVTxtAttributes::TTSModeSet

HRESULT TTSModeSet(GUID gVoice);

Sets the text-to-speech mode for a voice-text site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

gVoice

[in] GUID of the text-to-speech mode to set for the site. If the mode does not exist, an error is returned and the mode is not changed.

The text-to-speech mode for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the mode.


IVTxtDialogs Interface

The IVTxtDialogs interface allows an application to display Windows dialog boxes to configure a text-to-speech engine. The controls and capabilities in these dialog boxes are determined by the particular text-to-speech engine that is being used. If a voice-navigation application is installed on the user's computer, an application will not have to use IVTxtDialogs member functions because the voice-navigation application will provide access to the engine's dialog boxes.

Before an application can call any IVTxtDialogs member function, it must call the IVoiceText::Register member function to register a voice-text site.

The IVTxtDialogs interface is supported by the voice-text object. It has these member functions:

AboutDlg

GeneralDlg

LexiconDlg

TranslateDlg


IVTxtDialogs::AboutDlg

HRESULT AboutDlg(HWND hWndParent, PTSTR pszTitle);

Displays an About dialog box that identifies the text-to-speech engine and contains the copyright notice.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_INVALIDWINDOW
VTXTERR_NOTSUPPORTED
VTXTERR_OUTOFMEM

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the About dialog box or VTXTERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVTxtDialogs::GeneralDlg

HRESULT GeneralDlg(HWND hWndParent, PTSTR pszTitle);

Displays a General dialog box that gives the user general control of the text-to-speech engine. The dialog box may also include controls that allow the user to open other dialog boxes provided by the engine.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_INVALIDWINDOW
VTXTERR_NOTSUPPORTED
VTXTERR_OUTOFMEM

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the General dialog box or VTXTERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Depending on the engine, the General dialog box may allow the user to perform the following tasks:

· Set the maximum amount of disk space allotted for text-to-speech output. (Typically, setting the disk space is relevant only for rendering concatenated wave files.)

· Set the speaker preferences supported by the engine.


IVTxtDialogs::LexiconDlg

HRESULT LexiconDlg(HWND hWndParent, PTSTR pszTitle);

Displays a Lexicon dialog box that allows the user to view and edit the pronunciation lexicon. For example, the user can edit the phonetics of mispronounced words so that the engine pronounces them properly.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_INVALIDWINDOW
VTXTERR_NOTSUPPORTED
VTXTERR_OUTOFMEM

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Lexicon dialog box or VTXTERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVTxtDialogs::TranslateDlg

HRESULT TranslateDlg(HWND hWndParent, PTSTR pszTitle);

Displays a Translation dialog box that allows the user to control symbols, currencies, abbreviations, and number-translation techniques, such as whether to speak "34" as "thirty-four," "three four," or the old English "four and thirty." This dialog box also allows the user to control the translation of abbreviations and case sensitivity, and to manage how both affect the prosody of spoken text.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_INVALIDWINDOW
VTXTERR_NOTSUPPORTED
VTXTERR_OUTOFMEM

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Translation dialog box or VTXTERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVTxtNotifySink Interface

The IVTxtNotifySink interface is used by a voice-text object to return status information to an application. This interface is passed to the voice-text object by an application when it calls the IVoiceText::Register member function to register for voice text.

The IVTxtNotifySink interface is supported by an application. It has these member functions:

AttribChanged

Speak

SpeakingDone

SpeakingStarted

Visual

Visual is optional and may or may not be supported by an engine, but it must be supported by the notification sink. An application can choose to receive all possible notifications or only those from SpeakingStarted and SpeakingDone. Notifications are sent to all applications on the site, not just the application that caused the notification to be sent.


IVTxtNotifySink::AttribChanged

HRESULT AttribChanged(DWORD dwAttribute);

Notifies all applications on a voice-text site that a process has changed an attribute of the site.

· Return value is ignored.

dwAttribute

[in] Site attribute that was changed. This parameter can be one of these values:

Value | Meaning
IVTNSAC_DEVICE | Audio-output device
IVTNSAC_ENABLED | Enabled state of voice text
IVTNSAC_SPEED | Average talking speed
IVTNSAC_TTSMODE | Text-to-speech mode
IVTNSAC_VOLUME | Baseline speaking volume

The notification is sent only to applications that set the dwFlags parameter of the IVoiceText::Register member function to the VTXTF_ALLMESSAGES value when the application registered to use voice text on the site.


IVTxtNotifySink::Speak

HRESULT Speak(PTSTR pszText, PTSTR pszApplication, DWORD dwType);

Notifies all applications on a voice-text site that an application has added text to the playback queue for the site.

· Return value is ignored.

pszText

[in] Address of the string added to the playback queue.

pszApplication

[in] Address of a string containing the name of the application that added text to the playback queue.

dwType

[in] Type of text added to the playback queue. For a list of possible values, see the description of the dwFlags parameter of the IVoiceText::Speak member function.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceText::Register member function to the VTXTF_ALLMESSAGES value when the application registered to use voice text on the site.


IVTxtNotifySink::SpeakingDone

HRESULT SpeakingDone(void);

Notifies all applications on a voice-text site that speaking is finished and no text remains in the playback queue.

· Return value is ignored.


IVTxtNotifySink::SpeakingStarted

HRESULT SpeakingStarted(void);

Notifies all applications on a voice-text site that speaking has started.

· Return value is ignored.


IVTxtNotifySink::Visual

HRESULT Visual(WCHAR cIPAPhoneme, TCHAR cEnginePhoneme, DWORD dwHints,

PTTSMOUTH pTTSMouth);

Notifies all applications on a voice-text site about the phoneme that is currently being spoken.

· Return value is ignored.

cIPAPhoneme

[in] International Phonetic Alphabet (IPA) phoneme that is being spoken. Note that this parameter is always a wide character (WCHAR) because IPA symbols are available only in the Unicode character set.

cEnginePhoneme

[in] Engine-specific phoneme that is being spoken.

dwHints

[in] Type of sentence in which the phoneme is being spoken. This parameter can be one of these values:

Value | Meaning
TTSNSHINT_COMMAND | The phoneme is part of a command.
TTSNSHINT_EMPHASIS | The phoneme is part of a word being emphasized.
TTSNSHINT_EXCLAMATION | The phoneme is part of an exclamation.
TTSNSHINT_QUESTION | The phoneme is part of a question.
TTSNSHINT_STATEMENT | The phoneme is part of a statement.

pTTSMouth

[in] Address of a TTSMOUTH structure that contains information about the mouth position that corresponds to the current phoneme. If the mouth position is unknown or not supported, this parameter is NULL.

This notification may or may not be supported by an engine. If it is supported, this notification is sent only to applications that set the dwFlags parameter of the IVoiceText::Register member function to the VTXTF_ALLMESSAGES value when the application registered to use voice text on the site.

An application can use the information returned by Visual to synchronize character animation with spoken text.


IVoiceText Interface

The IVoiceText interface allows an application to register with a voice-text object and to control how voice text is spoken. The application must call the IVoiceText::Register member function before it can call any of the other member functions in this interface or certain other member functions, such as those in the IVTxtDialogs interface.

The IVoiceText interface is supported by the voice-text object. It has these member functions:

AudioFastForward | Register
AudioPause | Speak
AudioResume | StopSpeaking
AudioRewind |


IVoiceText::AudioFastForward

HRESULT AudioFastForward(void);

Advances audio playback by approximately one sentence or phrase.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_NOTENABLED
VTXTERR_NOTENOUGHDATA
VTXTERR_OUTOFMEM

Voice text determines the beginning of a sentence or phrase by checking for terminal punctuation, such as a period or comma, at the end of the preceding sentence or phrase. AudioFastForward works better for some kinds of text than others — for example, prose with conventional punctuation rather than spreadsheet data.

If less than a sentence remains in the playback queue, AudioFastForward jumps to the end of the playback queue and stops playing.

If no text is playing, AudioFastForward returns NOERROR, but does nothing. To determine whether text is playing, an application can use the IVTxtAttributes::IsSpeaking member function.


IVoiceText::AudioPause

HRESULT AudioPause(void);

Pauses text-to-speech output for a voice-text site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_NOTENABLED
VTXTERR_OUTOFMEM

AudioPause affects all applications using the site, so the application should resume audio as soon as possible.

When a voice-text object is first created, text-to-speech output is not paused. Because pausing text-to-speech output affects all applications that use voice text on the site, an application should resume text-to-speech output as soon as possible by calling the IVoiceText::AudioResume member function.

When output has been paused, the IVTxtAttributes::IsSpeaking member function returns FALSE, even though the voice-text object still has data available in its queue and has not yet sent a IVTxtNotifySink::SpeakingDone notification.

No notifications are sent when audio is paused or resumed.


IVoiceText::AudioResume

HRESULT AudioResume(void);

Resumes text-to-speech output after it has been paused by the IVoiceText::AudioPause member function.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_NOTENABLED
VTXTERR_OUTOFMEM

AudioResume affects all applications using the site.


IVoiceText::AudioRewind

HRESULT AudioRewind(void);

Backs up audio playback by approximately one sentence or phrase.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_NOTENABLED
VTXTERR_NOTENOUGHDATA
VTXTERR_OUTOFMEM

Depending on the length of a passage of text, it may not be possible to rewind to the beginning of the passage. The voice-text object caches only a few paragraphs of text, and AudioRewind cannot back up past the beginning of the cache. If AudioRewind is called at the beginning of the cache, audio begins playing at the first sentence in the cache.

If AudioRewind is called after the end of a text stream has been reached, the audio begins playing again at approximately the last sentence in the stream.

If no text is playing, AudioRewind returns NOERROR, but does nothing. To determine whether text is playing, an application can use the IVTxtAttributes::IsSpeaking member function.


IVoiceText::Register

HRESULT Register(PTSTR pszSite, PTSTR pszApplication,
PIVTXTNOTIFYSINK pNotifyInterface, IID IIDNotifyInterface,
DWORD dwFlags, PVTSITEINFO pSiteInfo);

Registers an application to use voice text on a site.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDPARAM
VTXTERR_OUTOFMEM

pszSite

[in] Address of a string identifying the voice-text site that the application needs to use. This parameter can be a null string (" ") or NULL to use the computer, or "LineX" to use a phone line, where "X" is the number of the line. The site identifier is used to save settings for the site in the registry so that they remain in effect the next time an application registers to use the site. The site identifier can also be used to restore settings from the registry. If a site has not been used, the default attributes are likely to be set for the computer. All applications on a site share the attributes of the site, such as the instance of the text-to-speech engine.

If a particular site (such as "LineX") requires an output device other than the default, the device must be set the first time the site is used. To set the device, an application can use the IVTxtAttributes::DeviceSet member function, or specify the device in the VTSITEINFO structure pointed to by this parameter. During subsequent registrations, the same device is used unless it is changed using DeviceSet.

pszApplication

[in] Address of a string that identifies the application — for example, "Microsoft Word." An application can use this information to display the source of text. This parameter must not be NULL.

pNotifyInterface

[in] Address of the notification interface through which the voice-text object notifies the application about text-to-speech information. If this parameter is NULL, no notifications will be sent. The interface identifier is specified by IIDNotifyInterface.

Because passing the pointer to the voice-text object does not transfer ownership of the notification interface, the voice-text object must call the AddRef member function of the notification interface before returning from the call to Register. The voice-text object must also call the Release member function of the notification interface when it closes. The calling application must release any reference counts it holds on the notification interface after calling Register, unless it needs the notification object to be valid when the voice-text object releases it.

IIDNotifyInterface

[in] GUID of the interface used for notification. This parameter must be IID_IVTxtNotifySinkA (for ANSI) or IID_IVTxtNotifySinkW (for Unicode).

dwFlags

[in] Flag that indicates whether the application is to receive all notifications. If this parameter is the VTXTF_ALLMESSAGES value, all notifications are sent to pNotifyInterface. If this parameter is zero (0) or null, only the IVTxtNotifySink::SpeakingStarted and IVTxtNotifySink::SpeakingDone notifications are sent.

pSiteInfo

[in] Address of a VTSITEINFO structure that contains settings to apply to the site, such as the voice and talking speed. The settings are applied, even if the site is already open. If a VTSITEINFO structure is not specified, the voice-text object uses the settings from the registry. If there are no registry settings, it uses the default settings, typically those for the computer.

Telephony applications pass this information to ensure that the proper settings are selected. Other applications will set this parameter to NULL to leave the site settings unchanged.

An application must call Register before it can call other functions in the IVoiceText interface or any function in the IVTxtDialogs interface.

An application cannot call Register a second time for the same voice-text object. To change sites, the application must call the CoCreateInstance function to create a new voice-text object for the desired site.


IVoiceText::Speak

HRESULT Speak(PTSTR pszSpeak, DWORD dwFlags, PTSTR pszTags);

Starts playing the specified text.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_INVALIDPARAM
VTXTERR_NOTENABLED
VTXTERR_OUTOFMEM
VTXTERR_QUEUEFULL
VTXTERR_WAVEDEVICEBUSY

pszSpeak

[in] Address of a buffer that contains the text to speak. An application can free or modify the buffer as soon as Speak returns. The string pointed to by this parameter must not contain text-to-speech control tags, but is affected by tags in pszTags.

dwFlags

[in] Flags that indicate the type and priority of the text. This parameter is a combination of one type flag and one priority flag.

The type flag can be one of these values:

Value | Meaning
VTXTST_COMMAND | An instruction to the user, such as "Insert the disk."
VTXTST_NUMBERS | Text that is numeric and that should be read in numeric style.
VTXTST_QUESTION | A question, such as "Do you want to save the file?"
VTXTST_READING | Text that is being read from a document, such as an email message.
VTXTST_SPREADSHEET | Text that is being read from a spreadsheet, such as columns of numbers.
VTXTST_STATEMENT | A neutral, informative statement, such as "You have new messages." This is the default type of speech.
VTXTST_WARNING | A warning, such as "Your printer is out of paper."

The priority flag can be one of these values:

Value | Meaning
VTXTSP_HIGH | Play the text as soon as possible, after text that is currently being spoken but before any other text in the playback queue.
VTXTSP_NORMAL | Add the text to the end of the playback queue. This is the default priority.
VTXTSP_VERYHIGH | Play the text immediately, interrupting text that is currently being spoken, if any. The interrupted text resumes playing as soon as the very high priority text is finished, although the interrupted text may not be correctly synchronized.

For example, a text-to-speech engine speaks a warning with the highest priority if this parameter is set with VTXTST_WARNING and VTXTSP_VERYHIGH. Setting this parameter to zero is the same as specifying VTXTST_STATEMENT and VTXTSP_NORMAL.

pszTags

[in] Address of a buffer that contains text-to-speech control tags to change the voice, language, or context of the text specified by pszSpeak, or NULL to use the default settings for the text-to-speech voice. For more information about control tags, see Appendix A, "Text-to-Speech Control Tags."

If an application calls Speak when other text is being played, the specified text is added to the end of the playback queue, unless the application specifies a higher priority in dwFlags.

Calling Speak affects all applications using voice text on the site, because all applications share the same playback queue.

The type of speech specified by dwFlags is communicated to the text-to-speech engine though control tags. Support of most control tags is optional; the engine ignores unsupported tags.


IVoiceText::StopSpeaking

HRESULT StopSpeaking(void);

Halts text that is currently being spoken and flushes all pending text from the playback queue.

· Returns NOERROR if successful, or one of these error values:

VTXTERR_INVALIDMODE
VTXTERR_NOTENABLED
VTXTERR_OUTOFMEM

Calling StopSpeaking affects all applications using voice text on the site, because all applications share the same playback queue.


Structure

This section describes a structure that an application uses to implement voice text.


VTSITEINFO

typedef struct { // vtsi
DWORD dwDevice;
DWORD dwEnable;
DWORD dwSpeed;
GUID gModeID;
} VTSITEINFO, *PVTSITEINFO;

Specifies an audio device, a text-to-speech mode, and the talking speed for a voice-text site and indicates whether voice text is enabled or disabled for the site.

dwDevice

Device identifier of the wave-out audio device to be used by the site. The device identifier can be obtained by calling the waveOutGetNumDevs and waveOutGetDevCaps multimedia functions.

dwEnable

TRUE if voice text is to be enabled for the site or FALSE if voice text is to be disabled.

dwSpeed

Baseline average talking speed, in words per minute, for the text-to-speech mode to be used by the site.

gModeID

GUID that uniquely identifies the text-to-speech mode to be used by the site. The GUID for a text-to-speech mode is obtained from a text-to-speech enumerator object. For information about text-to-speech enumerators, see the section, "Low-Level Text-to-Speech API."

An application can specify the address of a VTSITEINFO structure in a call to the IVoiceText::Register member function to set the voice, speaking speed, and other attributes of a voice-text site, even if the site is already open. Telephony applications typically do this to ensure that the proper information is selected for the site.


Voice-Text Error Values

The following error values may be returned by member functions of the voice-text object. If no error occurs, the member functions return the NOERROR error value.

Error value | Description
VTXTERR_INVALIDMODE | The text-to-speech mode is invalid, or the application has not been registered to use the voice text on the site.
VTXTERR_INVALIDPARAM | A parameter is invalid.
VTXTERR_INVALIDWINDOW | A window is invalid.
VTXTERR_NOTENABLED | Voice text is not enabled.
VTXTERR_NOTENOUGHDATA | The text stream has been rewound beyond the beginning or fast-forwarded past the end of the stream.
VTXTERR_NOTSUPPORTED | The dialog box is not supported by the engine.
VTXTERR_OUTOFMEM | There is not enough memory.
VTXTERR_QUEUEFULL | The playback queue is full.
VTXTERR_WAVEDEVICEBUSY | The wave device is busy.

In addition to the error values described in the preceding list, member functions can return error values generated by lower-level component objects or calls to the Win32 API. For example, an IVoiceText function might return an AUDERR_ value generated by the audio destination object. For more information, see "Object Levels and Error Values."


Voice-Text Object

The properties and methods described in this section apply to the voice-text object.


AudioFastForward Method

object.AudioFastForward( )

Advances audio playback by approximately one sentence or phrase.

object

Voice-text object.


AudioPause Method

object.AudioPause( )

Pauses text-to-speech output for the current site. This affects all applications using the site, so the application should resume speech output as soon as possible by calling the AudioResume method.

object

Voice-text object.


AudioResume Method

object.AudioResume( )

Resumes text-to-speech engine after it has been paused by the AudioPause method.

object

Voice-text object.


AudioRewind Method

object.AudioRewind( )

Backs up audio playback by approximately one sentence or phrase.

object

Voice-text object.


Callback Property

object.Callback

Specifies the name of a class module containing methods that receive notifications from the voice-text automation object. The class module must contain a SpeakingStarted method and a SpeakingDone method.

· Example:

Vtxt.Callback = "TESTVCMD.VTxtSink"

object

Voice-text object.


Enabled Property

object.Enabled

Enables or disables voice text. Disabling a site prevents the engine from playing text through the site.

· Example:

Vtxt.Enabled = True

object

Voice-text object.

If Enabled is TRUE, voice text is enabled. If Enabled is FALSE, voice text is disabled.


IsSpeaking Property

object.IsSpeaking

Indicates whether the voice-text object is in the process of speaking text for the voice-text site. A site is global and may be in use by another application.

· Example:

Vtxt.IsSpeaking = True

object

Voice-text object.

If IsSpeaking is TRUE, the voice-text object is currently speaking text. If IsSpeaking is FALSE, the object is not speaking.

The IsSpeaking property is read-only; an application cannot set this property.


Register Method

object.Register(siteName as String, applicationName as String)

Registers an application with the voice-text object.

· Example:

Call Vtxt.Register("", "VB Test App")

object

Voice text object.

siteName

Name of the site to connect to, such as "Line 1". To use the default site ("Local PC"), specify an empty string.

applicationName

Name of the application — for example, "Microsoft Word". An application can use this information to display the source of text. This parameter cannot be NULL.


Speak Method

object.Speak(speakString as String, flags as Long)

Renders a text string into speech.

· Example:

Call Vtxt.Speak(ESpeakBuf.TEXT, vtxtst_STATEMENT)

object

Voice-text object.

speakString

Text to be spoken.

flags

Type of speech and its priority. The type of speech can be one of these values:

Value | Description
vtxtst_COMMAND | An instruction to the user, such as "Insert the disk."
vtxtst_NUMBERS | Text that is numeric and that should be read in numeric style.
vtxtst_QUESTION | A question, such as "Do you want to save the file?"
vtxtst_READING | Text that is being read from a document, such as an e-mail message.
vtxtst_SPREADSHEET | Text that is being read from a spreadsheet, such as columns of numbers.
vtxtst_STATEMENT | A neutral, informative statement, such as "You have new messages." This is the default type of speech.
vtxtst_WARNING | A warning, such as "Your printer is out of paper."

The priority can be one of these values:

Value | Description
vtxtsp_VERYHIGH | Play the text immediately, interrupting text that is currently being spoken, if any. The interrupted text resumes playing as soon as the very high priority text is finished, although the interrupted text may not be correctly synchronized.
vtxtsp_HIGH | Play the text as soon as possible, after text that is currently being spoken but before any other text in the queue.
vtxtsp_NORMAL | Add the text to the end of the queue. This is the default priority.

The Speak method calls the IVoiceText::Speak member function to speak the text.


Speed Property

object.Speed

Sets or gets the speed at which speech is spoken, in words per minute.

· Example:

Vtxt.Speed = ESpeed.TEXT

MySpeed = Vtxt.Speed

object

Voice-text object.

Setting the Speed property to 0 causes speech to be spoken at the slowest speed; setting it to – 1 to causes speech to be spoken at the fastest speed.


StopSpeaking Method

object.StopSpeaking( )

Interrupts the text that is currently being spoken by the voice-text object and deletes any buffers containing text waiting to be spoken.

object

Voice-text object.

The StopSpeaking method calls the IVoiceText::StopSpeaking member function to interrupt the text.


What is Command and Control recognition?

"Command & Control" speech recognition allows the user to speak a word, phrase, or sentence from a list of phrases that the computer is expecting to hear. For example, a user might be able to speak the command, "Send mail to Fred Smith", "Send mail to Bob Jones", or "Turn on the television."

The number of different commands a user might speak at any time can easily number in the hundreds. Furthermore, the commands are not just limited to a "list" but can also contain other fields, like "Send mail to <Name>" or "Call {digits}" or "Record the movie at {time}". With all of the possibilities, the user is able to speak thousands of different commands.

When using Command & Control technology, users talk in natural speech to their computer and don't need to train the computer to their voice, so it's easy to use.


Why Use Command and Control?

In general, use Command and Control recognition when:

It makes the application easier to use.

It makes features in the application easier to get to.

It makes the application more fun/realistic.

If an application uses speech recognition solely for the "Wow!" feature it will work well for demos but will not be used by real users.


More Specific Reasons for Implementing Speech

Command and Control recognition might be used in some of the following situations:

· Answering questions. Speech recognition assists the user in answering questions posed by the computer. All speech-recognition engines can easily identify "Yes," "No," and a few other short responses. For example, an application can easily be designed to accept voice responses to message boxes and wizard screens.

· Streamlining access to features. Speech recognition enables the user to speak any one of a potentially huge set of commands, which can make hard-to-reach features more accessible. For example, it is usually faster to say, "Change the font to Arial," "Use 13-point bold," or "Print the current page" than to maneuver through the corresponding dialog boxes.

· Activating macros. Keyboard-activated macros usually require the user to memorize a key sequence that may not be intuitive because the keyboard is overloaded with commands. Speech recognition allows the user to speak a more natural word or phrase to activate a macro. For example, "Spell check the paragraph" is easier for most users to remember than the ctrl+f5 key combination.

· Accessing Large Lists. Many applications provide large lists of names that users must scroll through to select the correct one. The file open dialog box or the "Start" menu provide such lists. In general, it's faster for a user to speak one of the names on the list, such as "Start running calculator" than to maneuver through the list to find it.

· Facilitating dialogue between the user and the computer. Speech recognition works very well in situations where the computer essentially asks the user "What do you want to do?" and branches according to the reply (somewhat like a wizard). For example, the user might reply, "I want to book a flight from New York to Boston." After the computer analyzes the reply, it clarifies any ambiguous words ("Did you say New York?"). Finally, the computer asks for any information that the user did not supply, such as "What day and time do you want to leave?"

· Providing hands-free computing. Speech recognition is an essential component of any application that requires hands-free operation; it also can provide an alternative to the keyboard for users who are unable or prefer not to use one. Users with repetitive-stress injuries or those who cannot type may use speech recognition as the sole means of controlling the computer.

· Humanizing the computer. Speech recognition can make the computer seem more like a person — that is, like someone whom the user talks to and who speaks back. This capability can make games more realistic and make educational or entertainment applications friendlier.


Potential Uses By Application Category

The specific use of command and control recognition will depend on the application. Here are some sample ideas and their uses:


Games and Edutainment

Game and edutainment software titles will be some of the heaviest users of Command & Control speech recognition in the near term. Christmas 1995 saw the appearance of several games and half a dozen language-learning titles that use speech recognition. Only high end machines sold in Christmas 1995 could run speech recognition. Because memory and CPU in machines will increase, and because of the introduction of Microsoft's speech API, many more games and edutainment titles in the Christmas '96 market will use speech recognition.

What do these titles use speech recognition for?

The most compelling use is interacting with characters. In most games, when a user "talks" to a character on the computer, they either type in a response, or click on the text of the response with their mouse. A silent response is a let-down because the computer's character is actually speaking through the sound card. After all, how many conversations do you have where the person you're talking to speaks and you stay mute and write out your response? Why not allow the user to really talk back to the character? It's a lot more fun and realistic.

Another use for speech recognition in games/titles is to avoid magic keystrokes. Many games have gotten so complex that virtually every key on the keyboard does something like "Look out the left window" or "Turn on targeting". Whenever I play these games I can only remember the most common keys, such as the ones for firing. I can't ever remember that F2 looks out over the left window and F3 looks out the right window. Why not just let the user speak, "Look out the left window?"


Data Entry

Many applications like databases front-ends and spreadsheets require that the user transfers data into their computer from a piece of paper. This is a problem because their user cannot look at both the computer screen, keyboard, and paper at the same time. Users will end up longer from paper, to keyboard, to computer screen just to enter one value.

The application can use speech recognition if the data is specific enough. While speech recognition cannot effectively be used to enter names, it's very good at entering numbers and selecting items out of a small (less than 100) list. Some recognizers can even handling spelling fairly well. If an application uses speech recognition, the user no longer has to look at the keyboard. If this is combined with text to speech then the user doesn't even need to look at the screen, and is able to focus on the paper.

Furthermore, because speech recognition is not as "modal" as a keyboard, some applications don't even need to require a specific field to have focus. If the form that is being filled in has fields with mutually exclusive data types - one field allows "male" or "female", the other is an age, and the third is a city - then speech recognition can hear the command and automatically determine which field to fill in. After all, if only one field accepts "New York City" as a valid entry and the user speaks "New York City" then the application knows which field to fill in.

Speech recognition can significantly speed up data entry.


Document Editing

Command and control recognition is useful for document editing when the user wishes to keep his hands on the keyboard to type, or mouse to drag and select. He/she can simultaneously speak commands for manipulating the data that he/she is working on. A word processor might provide commands like "bold", "italic", "change to times new roman font", "use bullet list text style," and "use 18 point type". A paint package might have "select eraser" or "chose a wider brush."

Of course, keyboard equivalents are possible for many of these commands, but how many users actually remember the keyboard equivalent?


Hardware and Software Requirements

A speech application requires certain hardware and software on the user's computer to run. Not all computers have the memory, speed, microphone, or speakers required to support speech, so it is a good idea to design the application so that speech is optional.

These hardware and software requirements should be considered when designing a speech application:

· Processor speed. The speech-recognition and text-to-speech engines currently on the market typically require a 486/33 (DX or SX) or faster processor.

· Memory. On the average, speech recognition for command and control consumes 1 megabyte (MB) of random-access memory (RAM) in addition to that required by the running application. Speech recognition for dictation requires an additional 8 MB. Text-to-speech uses about 1 MB of additional RAM.

· Sound card. Almost any sound card will work for speech recognition and text-to-speech, including Sound Blaster™, Media Vision™, ESS Technology, cards that are compatible with the Microsoft® Windows Sound System, and the audio hardware built into multimedia computers. A few speech-recognition engines still need a DSP (digital signal processor) card.

· Microphone. The user can choose between two kinds of microphone: either a close-talk or headset microphone that is held close to the mouth or a medium-distance microphone that rests on the computer 30 to 60 centimeters away from the speaker. A headset microphone is needed for noisy environments.

· Operating system. The Microsoft Speech application programming interface (API) requires either Windows 95 or Windows NT version 3.5.

· Speech-recognition and text-to-speech engine. Speech-recognition and text-to-speech software must be installed on the user's system. Many new audio-enabled computers and sound cards are bundled with speech-recognition and text-to-speech engines. As an alternative, many engine vendors offer retail packages for speech recognition or text-to-speech, and some license copies of their engines.

For a list of engine vendors that support the Speech API, see the ENGINE.DOC file included with the Speech Software Development Kit.


Limitations

Currently, even the most sophisticated speech-recognition engine has limitations that affect what it can recognize and how accurate the recognition will be. This may seem like an impenetrable list, but a savvy application can design around these limitations.


Microphones and sound cards

The microphone is the largest problem that speech recognition encounters. Microphones inherently have the following problems:

Not every user has a sound card. Over time more and more PCs will bundle with a sound card.

Not every user has a microphone. Over time more and more PCs will bundle a microphone.

Sound cards (being in the back) don't make it very easy for users to plug in the microphone.

Most microphones that come with computers are cheap. Because of that, the microphones don't do as well as more expensive microphones that retail for $50 to $100. Furthermore, many of the cheap microphones that are designed to be worn are uncomfortable. A user will not use a microphone if it is uncomfortable.

Users don't know how to use a microphone. If the microphone is a worn on their head they often wear it incorrectly, or if it sits on their desktop they will lean towards it to speak even though the microphone is designed for the user to speak from their normal sitting position; After all, everyone has seen rock stars on TV, and they are practically eating the microphone.

Most applications can do little about the microphone. Luckily, when the user installs the speech recognition engine, the engine should come with software that makes sure the user's microphone is correctly plugged in and working.


Problems with ambient noise

In general, the user should be using a microphone as close to his/her mouth as possible. This reduces noise coming from their environment. If the user is sitting in a quiet room they can afford to have the microphone sitting several feet away from them, but if they're working in a cubicle they must wear a headset microphone that positions the microphone only a few centimeters away from their mouth.

For a cubicle office worker to user speech recognition he/she must wear a headset microphone. Unfortunately, many people don't like to be tethered so this limits the utility of speech recognition for applications targeted at users in cubicles.


Problems with computer generated sounds

What's often worse that ambient noise are sounds generated by the user's computer and played out their 80-watt stereo system.

Several methods exist to make sure that the microphone isn't hearing the speakers:

If the user is wearing a close-talk headset, causing the microphone to be about a few centimeters away from the user's mouth, then the microphone is so close to the user that it won't hear the speakers. This solution is good for avid game players who want speech recognition, but can't hold anything in their hands because they already have a joystick.

Make sure the user is wearing headphones. That way the microphone won't pick up the sound coming from the computer.

Have a "push-to-talk" button. When the user pushes the button, the computer's audio output is muted and speech recognition is turned on. When the user releases the button speech recognition is turned off and the audio is turned back on. Users get the hang of this pretty quickly. If used in a wireless device, this also saves battery life.


Half-Duplex Sound Cards

Many sound cards are only "half duplex" (as opposed to "full duplex"). If a sound card is "half duplex" it cannot record and play audio at the same time. That means that speech recognition cannot be listening while the card is playing sound. Fortunately, with plug-and-play the number of full duplex sound cards is increasing.


Speech Recognition Likes to Hear

Speech recognition engines like to hear - no surprise. They like to hear so much that if the user is having a phone conversation in the room while speech recognition is listening then the recognizer will think that the user is talking to it and hear random words. Sometimes the speech recognizer even hears door slamming as words.

There are several ways to overcome this obstacle:

Allow the user to turn speech recognition on/off quickly and easily. This can be done by the keyboard, mouse, voice, or joystick.

Have a "push-to-talk" button. This can be a key on the keyboard, mouse button, hot-spot that the cursor has to be over, a joystick button, or anything else. The user presses the button to get the computer to listen. When the button isn't pressed the computer isn't listening.

Give the computer a name, like "Computer," and make the user speak it prior to speaking a common. It's kind of like "Simon Says". The computer will only act when it hears its name.

Have the computer verify every command with the user. The Microsoft Voice application will display the command that it heard and then in small text display, "Say 'Do it' to accept the command." The command is not actually acted upon unless the user says "Do it" within a few seconds of the command being spoken.


Command and Control Engines need exact commands

Before an application starts a command and control recognizer listening it must first give the recognizer a "list" of commands to listen for. The list might include commands like "minimize window," "make the font bold," "call extension <digit> <digit> <digit>," and "send mail to <name>."

If the user speaks the command as it is written they are going to get very good accuracy. However, if they word the command differently (and the application hasn't provided the alternate wording) then recognition will either not recognize anything or, even worse, it will recognize something completely different. So, if a user speaks, "bold that" instead of "make the font bold" there's a pretty good change the computer will hear "minimize window".

Applications can work around this problem by:

Make sure the command names are intuitive to users. For many operations like minimizing a window, nine out of ten users will say "minimize" or "minimize" window without prompting.

Show the command on the screen. Sometimes an application will be able to display a list of commands on the screen. Users will naturally speak the same text they see. Microsoft Voice uses the application names shown on the task-bar for the "Switch to <application>" command.

Use word spotting. Many speech recognizers can be told to just listen for one keyword, like "mail". This way the user can speak, "Send mail", or "Mail a letter," and the recognizer will get it. Of course, the user might say, "I don't want to send any mail" and the computer will still end up sending mail.

Have the computer verify every command with the user. The Microsoft Voice application will display the command that it heard and then in small text display, "Say 'Do it' to accept the command." The command is not actually acted upon unless the user says "Do it" within a few seconds of the command being spoken.

Over time speech recognizers will start applying natural language processing and this problem will go away.


Speech Recognizers make mistakes

Speech recognizers make mistakes, and will always make mistakes. The only thing that is changing is that every two years recognizers make half as many mistakes as they did before. But, no matter how great a recognizer is it will always make mistakes.

An application can minimize some of the misrecognitions by:

Design the list of commands that the computer is listening for so that commands sound different. As a rule of thumb, the more phonemes different between two commands the more different they sound to the computer. The two commands, "go" and "no" only differ by one phoneme so when the user says, "Go" the computer is likely to recognize "No." However, if the commands were "Go there" and "No way" recognition would be much better.

Don't provide speech commands that do really bad things like formatting a user's hard disk.

Verify commands that might be dangerous.

Have the computer verify every command with the user. The Microsoft Voice application will display the command that it heard and then in small text display, "Say 'Do it' to accept the command." The command is not actually acted upon unless the user says "Do it" within a few seconds of the command being spoken.


Other Problems

Some other problems crop up:

Having a user spell out words is a bad idea for most recognizers because they are too inaccurate. However, some recognizers are specially tuned for spelling and do a very good job.

An engine also cannot tell who is speaking, although some engines may be able to detect a change in the speaker. Voice-recognition algorithms exist that can be used to identify a speaker, but currently they cannot also determine what the speaker is saying.

In addition, an engine cannot detect multiple speakers talking over each other in the same digital-audio stream. This means that a dictation system used to transcribe a meeting will not perform accurately during times when two or more people are talking at once.

· Unlike a human being, an engine cannot hear a new word and guess its spelling.

· Localization of a speech-recognition engine is time-consuming and expensive, requiring extensive amounts of speech data and the skills of a trained linguist. If a language has strong dialects that each represent sizable markets, it is also necessary to localize the engine for each dialect. Consequently, most engines support only five or ten major languages—for example, European languages and Japanese, or possibly Korean.

· Speakers with accents or those speaking in nonstandard dialects can expect more misrecognitions until they train the engine to recognize their speech. Even then, the engine accuracy will not be as high as it would be for someone with the expected accent or dialect. An engine can be designed to recognize different accents or dialects, but this requires almost as much effort as porting the engine to a new language.


Application Design Considerations

Here are some design considerations for applications using command and control speech recognition.


Design Speech Recognition in From the Start

It's poor design to bolt speech recognition onto an application that is designed for a mouse and keyboard. Applications designed for just the keyboard and mouse get little benefit from speech recognition. After all, how many DOS applications that were designed for just the keyboard came up with effective uses for the mouse?


Do Not Replace the Keyboard and Mouse

Speech recognition is not a replacement for the keyboard and mouse. In some circumstances it is a better input device than they keyboard/mouse but not an all. Speech recognition makes a terrible pointing device, just like the mouse makes a terrible text entry device, or the keyboard is bad for drawing. When speech recognition systems were first bolted onto the PC, it was thought that speaking menu names would be really useful. As it turns out, very few users use speech recognition to access a window menu because the mouse is much faster and easier.

Generally speaking, every feature in an application should be accessible from all input devices, keyboard, mouse, and speech recognition. Users will naturally use whichever input mechanism provides them the quickest or easiest access to the feature. The ideal input device for a feature will vary from user to user.


Work Around Recognizer Limitations

Speech recognizers have a lot of limitations, as listed in the previous section. Make sure that the application isn't using or requiring speech recognition to be used for purposes where it performs poorly.


Communicate Speech Awareness

Since most applications today do not include speech recognition, users will find speech recognition a new technology. They probably won't assume that your application has it, and won't know how to use it.

When you design a speech-recognition application, it is important to communicate to the user that your application is speech-aware and to provide him or her with the commands it understands. It is also important to provide command sets that are consistent and complete.


Managing User Expectations

When users hear that they can speak to their computers, they instantly think of Star Trek and 2001: A Space Odyssey, expecting that the computer will correctly transcribe every word that they speak, understand it, and then act upon it in an intelligent manner.

You should convey as clearly as possible exactly what an application can and cannot do and emphasize that the user should speak clearly, using words the application understands.


Communicating the Command Set

A graphic user interface provides users with tremendous feedback about what they can do by displaying menus, buttons, and other controls on the screen. Furthermore, the keyboard and mouse typically do not send erroneous signals to the application.

This is not so with speech recognition. The number of voice commands that can be recognized at any given time can easily number in the hundreds, perhaps even thousands. Although cues for the most likely commands could be displayed on the screen, it is impossible to display the full set at once. To compensate, an application can provide mechanisms to scan through the large list of active commands or can prompt the user for the most common voice responses through visuals or text-to-speech. For example, the application might say "Do you want to save the file? Say Yes or No." If the application does not recognize a command, it can also provide more extensive help. For example, "Please say either Yes or No, or say Help if you need more help."


Providing Feedback to the User

Whenever a voice command is spoken, you should give some sort of feedback to the user indicating that the command was understood and acted upon. Visual indications are usually sufficient, but if it is impossible to have noticeable visuals, you should verify with a short text-to-speech or recorded phrase.


Breaking Up Long Series of Numbers

Most engines have a very high error rate for long series of digits that are spoken continuously. For phone numbers or other long series, either break the number into groups of four or fewer digits or have the user speak each digit as an isolated word.


Avoiding Speech for Directional Commands

Speech input should not be used as a means of moving the mouse cursor because it is inefficient and annoying for the user. For example, the user would need to repeat directional commands, such as up, many times in succession to move the cursor to the desired screen position.


Where the Engine Comes From

Of course, for speech recognition to work on an end user's PC the system must have a speech recognition engine installed on it. The application has two choices:

The application can come bundled with a speech recognition engine and install it itself. This guarantees that speech recognition will be installed and also guarantees a certain level of quality from the speech recognizer. However, if an application does this, royalties will need to be paid to the engine vendor.

Alternatively, an application can assume that the speech recognition engine is already on the PC or that the user will purchase one if they wish to use speech recognition. The user may already have speech recognition because many PCs and sound cards will come bundled with an engine. Alternatively, the user may have purchased another application that included an engine. If the user has no speech recognition engine installed then the application can tell the user that they need to purchase a speech recognition engine and install it; Several engine vendors offer retail versions of their engines.


About Voice Commands

Voice commands allow the user to control an application by speaking commands through an audio input device rather than by using the mouse or keyboard, giving the user hands-free control of the application. Voice commands involve the use of an audio input device, such as a microphone or a telephone, a speech-recognition engine, and a voice menu. When the user speaks a command into the audio input device, the speech-recognition engine attempts to transcribe the spoken input into text. If the engine succeeds, it compares the command text to that of the commands in the active voice menus. (A voice menu contains a list of commands to which an application can respond.) If the engine finds a matching command in a voice menu, it notifies the application of the match, and the application carries out the command.

Implementing voice commands in an application involves the use of three different objects: the voice-command object, the voice-menu object, and the voice-menu enumerator. This section describes these objects and explains how to use them to implement voice commands in an application.

Some users do not wish to have the computer speak or to talk to the computer. For these users, Microsoft provides a Speech control panel application that sets a registry entry indicating the user's preferences. An application can check this entry before trying to load its voice-command code and can save memory by not loading the code if it is not needed. For more information about the registry entry, see the section, "Common Elements of the Speech Application Programming Interface.".

The voice-command and voice-menu objects are exposed as OLE automation objects, allowing them to be accessed and controlled through Visual Basic, Visual Basic for Applications (VBA), and other automation-aware application macro languages. You can take advantage of the OLE automation capabilities of these objects to add voice commands and voice-text capabilities to your Visual Basic applications. For more information, see Appendix E, "Voice Commands, Text-to-Speech, and OLE Automation."


Voice-Command Object

The voice-command object manages the use of voice commands by all voice-aware applications in the system. The object manages the interaction between applications and the speech-recognition engine. It also manages aspects of the interaction between the engine and the audio input device. Before an application can begin using voice commands, it must create an instance of the voice-command object and register itself with the object.


Object Creation

To create a voice-command object, an application calls the CoCreateInstance function with the CLSID_VCmd class identifier and the IID_IVoiceCmd interface identifier. An application must create a voice-command object for each audio source that it needs to use. For example, if the application needs to recognize commands from a telephone line as well as the computer microphone, it should create two voice-command objects.

After creating a voice-command object, CoCreateInstance returns a pointer to the IVoiceCmd interface for the voice-command object. An application uses IVoiceCmd to register itself with the speech-recognition engine and to create voice-menu objects and voice-menu enumerators. For more information about voice-menu objects and voice-menu enumerators, see "Voice-Menu Object" later in this section and "Voice-Menu Enumerator" later in this section.

Before an application can perform other voice-command tasks, it must register itself by calling the IVoiceCmd::Register member function. By registering, the application tells the engine which site to use as the source of voice commands and passes the voice-command object a pointer to the application's voice-command notification interface. The object calls the notification interface to notify the application about events related to voice commands.


Voice-Command Site

A speech-recognition engine receives spoken audio input from the user through a voice-command site on the user's computer. A voice-command site consists of an audio input device, such as a microphone or telephone, and a speech-recognition mode. An engine typically provides several different modes for recognizing speech, each representing a different language or dialect.

An application must create a separate voice-command object for each voice-command site it needs to use. To use a site other than the default (the computer microphone and the default mode of the engine), the application must obtain the identifier of the desired audio input device and the globally unique identifier (GUID) of the desired mode and then pass that information to the voice-command object when registering.

An application can use the waveInGetNumDevs and waveInGetDevCaps multimedia functions to obtain the device identifier of an audio input device. To determine the available speech-recognition modes, an application creates a speech-recognition enumerator. The enumerator returns information about all modes provided by all engines in the user's system, including the GUID of each mode.

For more information about multimedia and multimedia functions, see the Microsoft Win32 Software Development Kit (SDK). For information about enumerating the modes of a speech-recognition engine, see the "Low-Level Speech Recognition" section.


Site Attributes

Every voice-command site has a set of attributes that affect the interaction between the speech-recognition engine and the audio input device. An application can query and set a site's attributes by using the IVCmdAttributes interface provided by the voice-command object. To get a pointer to IVCmdAttributes, call the IVoiceCmd::QueryInterface member function with the IID_IVCmdAttributes interface identifier. All speech interfaces include the IUnknown functions QueryInterface, AddRef, and Release, so if you have a pointer to any interface for an object, it is not necessary to call IUnknown to get a pointer to another of the object's interfaces.

Because multiple applications can share the same voice-command site, changing a site attribute affects all applications registered to use the site, not just the application making the change. To receive notification of attribute changes, an application must specify the VCMDRF_ALLMESSAGES value when calling the IVoiceCmd::Register member function. Note that attribute settings are saved between uses of the site, even if the computer is shut off in the meantime.

Audio Input Device

An application can use the IVCmdAttributes::DeviceSet member function to change the audio input device for a voice-command site. To change the device, the application must specify the device identifier of the new device. An application can use the waveInGetNumDevs and waveInGetDevCaps multimedia functions to obtain the identifiers of the available audio devices in the system. The IVCmdAttributes::DeviceGet member function retrieves the device identifier of the wave-in audio device for the site.

Speech-Recognition Mode

A speech-recognition engine typically provides an assortment of modes that it can use to recognize speech in different languages and dialects. Each voice-command site uses a single speech-recognition mode. An application can change the mode for a voice-command site by specifying the GUID of the new mode in a call to the IVCmdAttributes::SRModeSet member function. To retrieve the GUID of the mode that the site is currently using, an application can call the IVCmdAttributes::SRModeGet member function.

Speaker Name

One way an application can improve the recognition accuracy of an engine is to train the engine to take into account the unique qualities of a user's voice. By using the IVCmdDialogs interface of the voice-command object, an application can direct the engine to display its training dialog box.

Typically, an engine's training dialog box displays a sequence of words and phrases that the user must speak into the audio input device. The engine processes the user's spoken input and saves information that helps the engine improve its recognition accuracy for the user. The engine saves the user's name (that is, the speaker name) along with the user's training information and uses the name to load the information whenever the user becomes the speaker for a site.

An application changes the speaker name for a voice-command site by using the IVCmdAttributes::SpeakerSet member function. Changing the speaker name unloads all training for the previous speaker and loads the training for the new speaker. If no training exists for the new speaker, the application starts with the engine's default training. The IVCmdAttributes::SpeakerGet member function retrieves the name of the current speaker for a site.

Microphone Name

Some speech-recognition engines can improve recognition accuracy by optimizing themselves for use with particular types of microphones. Setting a microphone name for a site allows the engine to identify the type of the microphone, so it can optimize itself accordingly. An application uses the IVCmdAttributes::MicrophoneSet and IVCmdAttributes::MicrophoneGet member functions to set and retrieve the microphone name.

Awake State

A voice-command site can be either awake or asleep. When a site is awake, the speech-recognition engine "listens" for commands for all active voice menus associated with the site. When a site is asleep, the engine listens for commands only from sleep menus. Commands from sleep menus become active only when the site is asleep, and they become inactive when the site is awake. An application uses the IVCmdAttributes::AwakeStateSet and IVCmdAttributes::AwakeStateGet member functions to set and retrieve the awake state.

An application can use AwakeStateSet to enable the user to briefly suspend voice commands for a site. For example, the user might want to suspend recognition from the computer microphone during a telephone conversation and resume recognition when the conversation is finished. A sleep menu typically contains a "Wake up!" command that resumes speech recognition, but it may contain other commands as well.

Enabled State

The IVCmdAttributes::EnabledSet member function enables or disables speech recognition for a voice-command site. Disabling a site completely turns off speech recognition for the site so that the engine recognizes no audio input, not even commands on sleep menus. For example, a user might want to disable speech recognition from the computer microphone during a meeting so that speech recognition will stay off, even if somebody inadvertently speaks a command on a sleep menu. The IVCmdAttributes::EnabledGet member function finds out whether a site is enabled or disabled.

The user can use the Speech control panel application to turn the system's speech capabilities on and off. When the system's speech capabilities are turned on or off, the WM_SPEECHSTARTED or WM_SPEECHENDED message is sent to all top-level windows in the system. An application can use these messages to determine when to enable or disable its voice-command capabilities.

Automatic Gain

Many speech-recognition engines can automatically adjust the gain of the incoming audio signal for a voice-command site (if the audio device supports it). Gain refers to the increase in signaling power, measured in decibels (dB), that occurs as a signal is boosted by an electronic device.

An application can use the IVCmdAttributes::AutoGainEnableSet member function to set the speed with which the engine adjusts the signaling power. When calling AutoGainEnableSet, the application specifies a value from 0 to 100. A value of 0 disables automatic gain, and a value of 100 causes the voice-command object to set the gain to the value for the previous utterance so that if the next utterance is spoken at the same level, the gain is set perfectly. A value between 0 and 100 moderates the automatic gain adjustments on a linear scale. For example, a value of 50 adjusts the gain to 50% of the level for the previous utterance.

An application retrieves the current automatic gain value for a voice-command site by using the IVCmdAttributes::AutoGainEnableGet member function.

Threshold Level

An application sets and retrieves the threshold level of a speech-recognition engine used by a voice-command site by using the IVCmdAttributes interface with the ThresholdSet and ThresholdGet member functions. The threshold level is a value from 0 to 100 that indicates the point below which the engine rejects an utterance as unrecognized. A value of 0 indicates that the engine should match any utterance to the closest phrase match. A value of 100 indicates that the engine should be absolutely certain that an utterance is the recognized phrase. For example, suppose the engine is expecting "What is the time?" If the threshold is 100 and the user mumbles "What'z tha time" or has a cold, the command may not be recognized. However, if the threshold is too low and the user says a similar-sounding phrase that is not being listened for, such as "What is mine?" the engine may recognize it as "What is the time?"

If the command spoken by the user is not close enough to what the speech-recognition engine expects, the voice-command object notifies the application that the command was not recognized by calling the application's IVCmdNotifySink::CommandOther notification interface with a NULL string. For more information about the voice-command notification interface, see "Notification Interface" later in this section.


Voice-Menu Database

The voice-command object maintains a database of voice menus that have been created by applications. The first application to create a voice-command object causes the database to be opened. The database remains open until the last voice-command object is released.

An application can use a voice menu from the database or add new menus to the database by using voice-menu objects. It can enumerate the voice menus in the database by using a voice-menu enumerator. For more information about voice-menu objects and voice-menu enumerators, see "Voice-Menu Object" later in this section and "Voice-Menu Enumerator" later in this section.


Engine Dialog Boxes

A speech-recognition engine provides a set of dialog boxes that enable the user to configure the engine. By using the IVCmdDialogs interface provided by the voice-command object, an application can direct the engine to display one of its dialog boxes. The controls and capabilities of the dialog boxes are determined solely by the engine. Before an application can call any IVCmdDialogs member function, it must first register with the engine by calling the IVoiceCmd::Register member function.

When an application calls a member function of the IVCmdDialogs interface to display a dialog box, it specifies the handle of the parent window for the dialog box. By specifying NULL instead of a window handle, the application can determine whether the engine supports the given dialog box. The engine returns the NOERROR value if it supports the dialog box or the VCMDERR_NOTSUPPORTED error value if it does not.

The application also supplies a title for the dialog box by specifying the address of a title string. If the application specifies NULL instead of a title string address, the engine uses the default title.

If a voice-navigation application, such as Microsoft® Voice, is running, an application does not need to direct an engine to display its dialog boxes because the voice-navigation application gives the user access to them. For more information about voice-navigation applications, see the section, "Common Elements of the Speech Application Programming Interface.".

About Dialog Box

An engine's About dialog box contains information that identifies the speech-recognition engine and its manufacturer and that contains the manufacturer's copyright notice. An application displays the About dialog box by calling the IVCmdDialogs::AboutDlg member function.

General Dialog Box

An engine's General dialog box gives the user control of the general features of the speech-recognition engine. An application displays the General dialog box by using the IVCmdDialogs::GeneralDlg member function. Depending on the engine, the General dialog box may allow the user to perform the following tasks:

· Inform the engine of the speaker's gender or other speaker-specific information.

· Control how stringent the engine is about recognition.

· Control the maximum amount of disk space allotted for speaker information.

· Control optimizations that affect speech processing time.

· Export and import speaker information to and from a disk so that this information can be transferred between systems.

Lexicon Dialog Box

A speech-recognition engine maintains a database of pronunciations called a pronunciation lexicon, which an application can load into the Lexicon dialog box by using the IVCmdDialogs::LexiconDlg member function. The user edits the pronunciation lexicon by correcting the phonetics of mispronounced words and by loading and unloading personal exception dictionaries.

General Training Dialog Box

The IVCmdDialogs::TrainGeneralDlg member function displays the General Training dialog box, which trains a speech-recognition engine by having the user speak a preselected set of words and phrases. After the user speaks the words and phrases, recognition accuracy of the engine should be better for that particular speaker.

Microphone Training Dialog Box

The IVCmdDialogs::TrainMicDlg member function displays the Microphone Training dialog box, which trains the engine for use with a particular microphone so that recognition accuracy improves with the microphone. An application should display this dialog box at the request of the engine or whenever the speaker wants to display it. Training information is associated with both the speaker and the microphone. If the engine does not supply this dialog box, the speaker cannot train the engine for the microphone; however, the application will still work.


Voice-Menu Object

A voice-menu object contains interfaces that an application uses to manage its voice menus. A voice menu contains a set of commands that a speech-recognition engine can recognize and that the application can respond to. An application must create a voice-menu object to represent a voice menu; only then can the engine use the menu to recognize commands. An engine can use more than one voice menu for an application, but the application must create a separate voice-menu object for each menu.


Object Creation

After an application has registered its site with the voice-command object, it can create a voice-menu object by using the IVoiceCmd::MenuCreate member function. MenuCreate retrieves a pointer to an IVCmdMenu interface for the newly created voice-menu object. An application uses the IVCmdMenu member functions to obtain information about the voice menu, add and remove menu commands, and so on.

When calling MenuCreate, the application specifies the name of the voice menu that the new object is to represent and indicates whether the function should create a new menu or open an existing one from the voice-menu database. If the function creates a new menu, the menu initially contains no commands. The application must add commands by using the IVCmdMenu::Add member function.

A voice menu is stored in the database, unless it was created as a temporary menu. A temporary menu is destroyed when the voice-menu object is released. Typically, you will store a voice menu between invocations of the voice-command object, so you should first call MenuCreate with the VCMDMC_OPEN_EXISTING value. If MenuCreate returns an error, you should call it a second time with the VCMDMC_OPEN_ALWAYS value, which creates a new voice menu for the application.

An application can use the IVoiceCmd::MenuEnum member function to create a voice-menu enumerator object to enumerate the menus currently in the voice-menu database. For more information about voice-menu enumerators, see "Voice-Menu Enumerator" later in this section.


Voice Menus

A voice menu contains a set of one or more voice commands that can be recognized when the voice menu is active. This section describes the attributes of a voice menu.

Name and Language

Every voice menu has a name and a language. The name consists of two strings: the name of the application that uses the menu and a string that identifies the application state in which the menu is valid. The application name must be unique among all applications registered to use voice commands in the system. Typically, the state string is the title-bar text of the window or dialog box with which the voice menu is associated (for example, "Main Window"). An application uses the VCMDNAME structure to specify a voice menu's name.

A voice menu's language identifies the language and dialect that the speech-recognition engine should use for the menu. A language consists of a Win32 language identifier and a dialect string. An application uses the LANGUAGE structure to specify a voice menu's language.

Activation

Once a voice menu is created and has commands, it must be activated before its commands can be recognized. When a menu is active, the engine checks the menu for a matching command whenever the engine processes an utterance from the voice-command site. The engine does not check the menu when it is inactive. An application uses the IVCmdMenu interface with the Activate and Deactivate member functions to explicitly activate and deactivate a voice menu. The menu remains open after a Deactivate command, so the application only needs to call Activate to reactive the menu; the application does not need to reopen the menu.

When calling Activate, an application specifies whether the voice menu is to be associated with a particular window or is a global menu. When a voice menu is associated with a window, the menu is automatically activated whenever the window becomes the foreground window and is automatically deactivated when the user switches to another window. The application does not need to explicitly activate or deactivate the menu. Instead of being associated with a particular window, a voice menu can also be global. A global voice menu remains active at all times regardless of which window is in the foreground, until the application explicitly deactivates it.

A global voice menu is useful for an application such as a clock program. The user can, for example, ask what time it is and get a response no matter what other applications are active. Global menu commands have a lower priority in case of a recognition conflict, which happens when multiple commands with the same name appear in different menus.

When an application calls Activate, it specifies whether the voice menu should remain active when speech recognition is asleep for the voice-command site. By specifying the VWGFLAG_ASLEEP value, the application indicates that the menu should be active only when speech recognition is asleep and should be automatically deactivated when speech recognition is awake. A menu activated with the VWGFLAG_ASLEEP value is called a sleep menu. If the application does not specify this value, the menu is only active when speech recognition is awake.

Phrase Lists

An application can use the IVCmdMenu::ListSet member function to associate any number of phrase lists with a voice menu. A phrase list is a named set of words or phrases. By embedding the name of a list in the command string, the user can speak any word or phrase in the list at the given point in a command string. For example, a list named "myfriends" might contain the names "Linda," "Tom," and "Mary." If the list name were embedded in a command string, such as "Send mail to myfriends," the user could speak "Send mail to Linda," Send mail to Tom," or "Send mail to Mary."

Typically, a list contains information that changes dynamically at run time, such as the ten people to whom the user most recently sent electronic mail. For best results, a list should have fewer than 20 entries. Having more than 20 entries in a list can reduce the accuracy of recognition.

A list is associated with a menu rather than an individual command. The list must appear in at least one command string, but can be used by more than one command on the menu. The list persists until the voice-menu object is released. List entries are not automatically saved to disk. To preserve the list, call the IVCmdMenu::ListGet member function and take steps to store the result. ListGet retrieves the phrases stored in the current list for the selected voice menu.

The IVCmdMenu::ListSet member function is much faster than the IVCmdMenu interface's Add, Remove, or Set member functions because list entries are substituted when a command is recognized and the menu is not recompiled. This means that ListSet can be called on an active menu without affecting performance.

Training

The IVCmdMenu::TrainMenuDlg member function displays a modal dialog box to train the engine for the selected menu. Because the Train Menu dialog box is provided by the speech-recognition engine, the controls in the dialog box and the training it requests depend on the speech-recognition engine being used. Some engines may not offer this dialog box.

An application can call TrainMenuDlg to obtain training for the commands on a particular menu. In contrast, the training obtained by calling the IVCmdDialogs::TrainGeneralDlg member function applies to all commands, regardless of the menu they are on.


Voice-Menu Commands

A voice menu contains a list of voice commands to which an application can respond. An application uses the functions provided by a voice-menu object's IVCmdMenu interface to add commands to a voice menu, remove commands from the menu, modify commands, obtain information about commands, and so on.

A voice command is defined by a VCMDCOMMAND structure, which specifies the attributes of the command. This section describes the attributes common to all commands and describes the IVCmdMenu functions that an application uses to manage a voice menu's commands.

Command Position and Identifier

Each command in a voice menu has a position. The first command added to a menu is in position 1, the second is in position 2, and so on. New commands are always added to the end of the menu.

An application can assign a command identifier to each voice command in a menu. A command identifier is an application-defined doubleword value that uniquely identifies a command within its menu. When an application receives a notification from the voice-command object, the application can examine the command identifier to determine the command that is the subject of the notification.

The IVCmdMenu interface's Remove, Get, and Set member functions process commands either by their positions in the menu or by their identifiers, depending on whether the application specifies the VCMD_BY_POSITION or VCMD_BY_IDENTIFIER value. If VCMD_BY_POSITION is specified, the functions process one or more commands sequentially in the command set. If VCMD_BY_IDENTIFIER is specified, the functions process all of the commands with the identifier.

Command String

Every voice command must have a command string, such as "Open a file." During the speech-recognition process, an engine translates audio input into a string and compares it to the command strings in the active voice menus. A match constitutes a voice command. In languages, such as Japanese, that have both a phonemic and symbolic character set, the command string is a phonemic string.

Within the command string, the '' and '' characters indicate the name of a list of words or phrases that can be spoken at the given point in the command. For example, the command string "Send mail to name" contains a list called "name." To add phrases to the list, use the IVCmdMenu::ListSet member function. To retrieve the phrases stored in the current list for a voice menu, use the IVCmdMenu::ListGet member function.

Many speech-recognition engines support wildcards in command strings. Using wildcards allows the user to speak any words between certain key words in a phrase. For example, if an application defines a voice command of "{wild-card} mail {wild-card} name", the user could say, "Send mail to Fred," "I want to mail Fred," or even "Mail, yes, I want it to go to Fred."

Command Text

The command text is a string that an application displays to the user when he or she requests a list of the available voice commands for the application. If a command has no command text, the application displays the command string.

Most applications written for European languages do not use command text because these languages use only one character set. Applications written for languages that have both a phonemic character set and a symbolic character set, such as Japanese, store the phonemic representation of the command in the command string and the symbolic representation (which most users prefer to see) in the command text.

Category String

A voice command's category string indicates the category to which the command belongs. Commands in a voice menu should be organized into categories to help the user easily browse through lists of commands. This is similar to Windows menus, which organize commands under menu names such as "File," "Edit," "View," and so on. For best results, you should use 20 or fewer categories.

Command Description

A command's description is a string describing the action that the application performs in response to the command. The command description tells the user the purpose of the command. An application typically displays a command's description string along with its command text when the user requests a list of available voice commands for the application.

Enabled State

At any given instant, a voice command is either enabled or disabled. If a command is enabled, the engine includes it when attempting to match the user's utterance to a command in the voice menu. The engine excludes a command if it is disabled.

A command can be disabled either temporarily or permanently. When a command is temporarily disabled, the voice-command object sends a IVCmdNotifySink::CommandOther notification rather than a IVCmdNotifySink::CommandRecognize notification when the engine recognizes the disabled command. An application uses the IVCmdMenu::SetItem member function to temporarily disable and later reenable a voice command.

A voice command that is permanently disabled is not compiled into the voice menu. The engine cannot recognize a permanently disabled command, and no notifications are sent to the application when the user speaks the command. An application uses the IVCmdMenu::EnableItem member function to permanently disable and later reenable a voice command.

The IVCmdMenu::SetItem member function is much faster than the IVCmdMenu::EnableItem member function because the menu is not recompiled. This means that SetItem can be called on an active menu without affecting performance.

For best results, you should use the IVCmdMenu::Deactivate member function to deactivate the voice menu before calling EnableItem. Calling EnableItem on an active menu can be fairly slow because the menu must be deactivated, recompiled, and reactivated before the member function returns. If the menu is already deactivated when EnableItem is called, the menu is not recompiled until the application activates it again.

Action Data

Action data is application-defined data associated with a command in a voice menu. The engine passes the data to the application's notification interface when the command is recognized. An application can associate action data with each command in a voice menu.

Action data should be some unique identifier, so the application knows what (if anything) to do with the command when it is spoken. For example, action data might be an index into a table.

Action data is not interpreted by the voice-command object; it is up to the application to determine whether the data is valid and to act upon it. An application should always check the validity of the action data because it is susceptible to being changed, accidentally or intentionally, by other applications.

Command Operations

After creating a voice-menu object, the IVoiceCmd::MenuCreate member function retrieves the address of an IVCmdMenu interface. An application uses the member functions of IVCmdMenu to add, remove, and retrieve information about the voice commands in a voice menu.

If the voice menu is new, the application must add commands to the voice menu before it can use the menu. An application adds voice commands to a voice menu by filling an array of VCMDCOMMAND structures, copying the address and size of the array into an SDATA structure, and passing the address of the SDATA structure to the IVCmdMenu::Add member function. New commands are appended to the end of the menu after any existing commands.

To remove commands from a voice menu, an application uses the IVCmdMenu::Remove member function. It uses the IVCmdMenu::Set member function to modify a range of the commands in a menu.

An application should deactivate a voice menu by calling the IVCmdMenu::Deactivate member function before calling the IVCmdMenu interface's Add or Remove member function. Calling these member functions on an active menu can be slow because the engine must must deactivate, recompile, and reactivate the menu before the function returns. If the menu is already deactivated when the member function is called, the menu is not recompiled until the application activates it again.

The IVCmdMenu::Get member function retrieves information about a range of commands in a voice menu. The application passes the address of an SDATA structure to Get. Get fills an array of VCMDCOMMAND structures containing information about each command and then copies the address and size of the array into the SDATA structure. An application can use the command information retrieved by Get to modify commands in the menu, create a new menu based on an existing one, or "borrow" commands from one menu and add them to another.

Get uses the OLE task allocator to allocate memory for the command information and copies the address of the memory into the pData member of the SDATA structure. The application must free the memory by using the CoTaskMemFree function. If Get fails to allocate memory, the function sets the pData member to NULL and the dwSize member to zero.

A voice menu can contain any number of voice commands. An application uses the IVCmdMenu::Num member function to retrieve the number of commands in a voice menu.


Voice-Menu Enumerator

When an application creates a new voice menu by using the IVoiceCmd::MenuCreate member function with the VCMDMC_CREATE_NEW, VCMDMC_CREATE_ALWAYS, or VCMDMC_OPEN_ALWAYS value, the voice-command object stores the new menu in a database. When the application is restarted at a later time, it can save time by loading menus from the database instead of creating new ones.

An application can identify voice menus stored in the database by using a voice-menu enumerator. To create an enumerator, the application calls the IVoiceCmd::MenuEnum member function, specifying whether to enumerate all function menus in the database or only those belonging to a particular application and state. The application also specifies the type of menus to enumerate. For example, it can enumerate active menus, permanent menus, or open menus.

MenuEnum retrieves a pointer to an IVCmdEnum interface that includes the OLE IEnumX enumeration functions Clone, Next, Reset, and Skip. The application uses these functions to obtain a pointer to an array of VCMDNAME structures that contain the application names and states of the menus matching the enumeration criteria specified in the call to MenuEnum.

To use a voice menu from the database, an application must pass the address of a VCMDNAME structure that it received from the enumerator to the IVoiceCmd::MenuCreate member function. MenuCreate creates a voice-menu object that represents the voice menu.

A voice-menu enumerator persists until all references to it are released, even if the voice-command object is released first.


Notification Interface

When an application calls the IVoiceCmd::Register member function to register its voice-command site, the application must also provide a pointer to its IVCmdNotifySink interface. The voice-command object calls this interface to notify the application of speech-related events. To receive notifications, the application must create a Component Object Module (COM) object that supports the IVCmdNotifySink interface. You determine how your application responds to notifications.

The voice-command object sends a variety of notifications to the IVCmdNotifySink interface. When calling Register, an application indicates whether the object should send all notifications or just a subset. A voice-command object can send the following notifications.

Notification | Description
AttribChanged | Sent when an application changes an attribute for the voice-command site. The notification includes a flag that identifies the attribute that changed.
CommandOther | Sent when a spoken phrase was either recognized as being from another application's command set or was not recognized. Along with the notification, the application receives a pointer to the phrase. If the pointer has a value, the phrase was recognized as valid for some other voice-command object. If the pointer to the phrase is NULL, the phrase was not recognized.
CommandRecognize | Sent when the engine recognizes a spoken phrase as being from an active menu associated with the application. Along with the notification, the application receives the text of the phrase and the action data that was supplied by the application when it originally defined the command.
CommandStart | Sent when the engine begins recognition processing for a command.
Interference | Sent when the engine cannot properly recognize an utterance for a known reason. The notification includes a flag indicating the reason.
MenuActive | Sent when a menu has been activated or deactivated.
UtteranceBegin | Sent as soon as the speech-recognition engine has received any audio input that might be speech.
UtteranceEnd | Sent as soon as the speech-recognition engine senses that the user has stopped speaking.
VUMeter | A linear volume-unit (VU) level from 0 to 0xFFFF, sent between 8 and 16 times a second, that indicates the loudness of the digital-audio stream. This information can be used to provide feedback to the user that the voice-command object is listening.

Typically, the voice-command object calls UtteranceBegin when the user starts speaking and UtteranceEnd when the user stops speaking. After UtteranceEnd is called, a small amount of time may elapse while the speech-recognition engine processes the utterance. When the engine finishes processing, the voice-command object calls CommandRecognize if it recognized one of the application's commands or CommandOther if the command was unrecognized or was recognized as another application's command.


Using Voice Commands

This section contains examples demonstrating how to perform the following tasks:

· Initialize an application to use voice commands.

· Enumerate menus in the voice-menu database maintained by the voice-command object.

· Add commands to a voice menu.

· Respond to notifications from the voice-command object.


Initializing an Application for Voice Commands

To use voice commands, you need to create a voice-command object, register your application with the object, and then create a voice-menu object to manage your application's voice menus. You create a voice-command object by calling the CoCreateInstance function with the CLSID_VCmd class identifier and the IID_IVoiceCmd interface identifier. You must create a separate voice-command object for each site that your application needs to use.

CoCreateInstance returns a pointer to the IVoiceCmd interface for the voice-command object. Before it can perform other voice-command tasks, an application must register itself by calling the IVoiceCmd::Register member function. Register specifies the site that the object represents, and passes the address of the application's voice-command notification interface to the voice-command object.

After creating a voice-command object and registering the application, you can use the IVoiceCmd::MenuCreate member function to open a voice menu and create a voice-menu object to represent the menu. MenuCreate retrieves the address of the IVCmdMenu interface for the voice-menu object. You can use the interface's member functions to manage the menu and its commands.

The following example shows how to create a voice-command object, register an application, and create a voice-menu object. The example creates a temporary voice-menu object; that is, the object is not added to the voice-menu database maintained by the voice-command object.

// BeginOLE - initializes OLE, creates an instance of the voice-command

// object, registers the application with the object, and creates a

// temporary voice-menu object.

// Returns TRUE if successful or FALSE otherwise.

//

// Global variables and constants:

// gpIVoiceCommmand - address of the IVoiceCmd interface for the

// voice-command object

// gpIVCmdDialogs - address of the IVCmdDialog interface for the

// voice-command object

// gpIVCmdMenu - address of the IVCmdMenu interface for the

// voice-menu object

BOOL BeginOLE (void)

{

HRESULT hRes;

VCMDNAME VcmdName;

LANGUAGE Language;

PIVCMDATTRIBUTES pIVCmdAttributes;

// Initialize interface pointers to NULL.

gpIVoiceCommand = NULL;

gpIVCmdDialogs = NULL;

gpIVCmdMenu = NULL;

// Initialize OLE.

if (FAILED(CoInitialize(NULL)))

return FALSE;

// Create the voice-command object.

if (CoCreateInstance(CLSID_VCmd, NULL, CLSCTX_LOCAL_SERVER,

IID_IVoiceCmd, (LPVOID *) &gpIVoiceCommand) != S_OK)

goto fail;

// Retrieve the address of the voice-command object's IVCmdDialogs

// interface. The application uses this interface to direct the

// speech-recognition engine to display its configuration dialog boxes.

hRes = gpIVoiceCommand-QueryInterface(IID_IVCmdDialogs,

(LPVOID FAR *) &gpIVCmdDialogs);

if (hRes)

goto fail;

// Register the application with the voice-command object.

hRes = gpIVoiceCommand->Register(

"", // use the default site (computer microphone)

&gVCmdNotifySink, // address of application's notification interface

IID_IVCmdNotifySink, // GUID of the notification interface

VCMDRF_ALLMESSAGES, // send all notifications to the interface

NULL); // do not change the site settings

if (hRes)

goto fail;

// Use the voice command's IVCmdAttributes interface to ensure that

// speech recognition is awake and enabled for the whole system.

hRes = gpIVoiceCommand-QueryInterface(IID_IVCmdAttributes,

(LPVOID FAR *) &pIVCmdAttributes);

if (pIVCmdAttributes) {

pIVCmdAttributes-EnabledSet(TRUE);

pIVCmdAttributes-AwakeStateSet(TRUE);

pIVCmdAttributes-Release();

};

// Create a voice-menu object.

lstrcpy (VcmdName.szApplication, "My Demo Application");

lstrcpy (VcmdName.szState, "Main");

Language.LanguageID = LANG_ENGLISH;

lstrcpy (Language.szDialect, "US English");

hRes = gpIVoiceCommand->MenuCreate(

&VcmdName, // application name and state

&Language, // language and dialect

VCMDMC_CREATE_TEMP, // temporary menu (not stored in database)

&gpIVCmdMenu // receives address of voice-menu object

);

if (hRes)

goto fail;

return TRUE;

// If the BeginOLE function fails, release the IVoiceCmd, IVCmdDialogs,

// and IVCmdMenu interfaces and set the interface pointers to NULL.

fail:

if (gpIVoiceCommand)

gpIVoiceCommand->Release();

if (gpIVCmdDialogs)

gpIVCmdDialogs->Release();

if (gpIVCmdMenu)

gpIVCmdMenu->Release();

gpIVoiceCommand = NULL;

gpIVCmdDialogs = NULL;

gpIVCmdMenu = NULL;

return FALSE;

}


Enumerating Voice Menus

The voice-command object maintains a database of voice menus that have been created by applications using voice commands. You can identify the voice menus in the database by using a voice-menu enumerator, which you create by using the IVoiceCmd::MenuEnum member function. When calling MenuEnum, you specify whether to enumerate all menus in the database, or only those belonging to a particular application and state. You also specify whether to enumerate active menus, permanent menus, open menus, and so on.

MenuEnum returns a pointer to an IVCmdEnum interface. You use the member functions of this interface to identify the menus that match the enumeration criteria specified in the call to MenuEnum. You use the IVoiceCmd::MenuCreate member function to create a voice-menu object for a menu that you retrieved from the database.

The following example creates a voice-menu enumerator and uses it to identify menus in the database belonging to an application called "My Demo Application."

// DoEnum - uses a voice-menu enumerator to find an application's

// menus in the voice-menu database.

// Returns TRUE if successful or FALSE otherwise.

BOOL DoEnum(void) {

HRESULT hRes;

VCMDNAME VCName;

PIVCMDENUM pIVCmdEnum = NULL;

// Create a voice-menu enumerator object.

hRes = gpIVoiceCommand->MenuEnum(

VCMDEF_DATABASE, // enumerate menus in database

"My Demo Application", // enumerate menus for demo application

NULL, // enumerate menus for all states

&pIVCmdEnum); // receives address of enumerator

if (hRes)

goto fail;

while (S_OK == pIVCmdEnum->Next(1, &VCName, NULL))

// DoMenuObject is an application-defined function that uses

// MenuCreate to create a voice-menu object.

DoMenuObject(&VCName);

fail:

// Release the enumerator.

if (pIVCmdEnum)

pIVCmdEnum-Release();

pIVCmdEnum = NULL;

if (hRes)

return FALSE;

else

return TRUE;

}


Adding Commands to a Voice Menu

After you create a voice-menu object, you can add commands to the menu by filling an array of VCMDCOMMAND structures, copying the address and size of the array into an SDATA structure, and passing the address of the SDATA structure to the IVCmdMenu::Add member function.

The example in this section shows how to add a new set of commands to a voice-menu object. The example consists of three functions: UseCommands, GetCommands, and NextCommand. The UseCommands function deactivates the voice menu, replaces any existing commands in the menu with a new set, and reactivates the menu.

One of the parameters to UseCommands is the address of a buffer containing the list of command strings to enter. UseCommands passes the address of the command-string buffer to the GetCommands function along with the address of an SDATA structure. The GetCommands function converts the buffer to an array of VCMDCOMMAND structures and copies the address and size of the array into the SDATA structure. The NextCommand function is a helper routine that GetCommands uses to retrieve individual command strings from the command buffer passed to UseCommands.

// UseCommands - adds commands to a voice-menu object, replacing any

// existing commands.

// Returns the result code returned by a voice-menu function.

// pszCommands - address of a null-terminated string of commands

// separated by \n or \r characters

// pMenu - address of the IVCmdMenu interface for the voice-menu object

HRESULT UseCommands (char *pszCommands, PIVCMDMENU pMenu)

{

HRESULT hRes;

SDATA data;

DWORD dwNum, dwStart;

hRes = pMenu->Deactivate(); // deactivate the menu

if (hRes) return hRes;

// Retrieve the number of commands in the menu.

hRes = pMenu->Num(&dwNum);

if (hRes) return hRes;

// Remove the existing commands from the menu.

if (dwNum)

hRes = pMenu->Remove (1, dwNum, VCMD_BY_POSITION);

if (hRes) return hRes;

// Call GetCommands, an application-defined function that fills an SDATA

// structure with information about the commands to add to the menu.

if (!GetCommands(pszCommands, &data, &dwNum))

return ResultFromScode (E_OUTOFMEMORY);

// Add the commands to the menu.

hRes = pMenu->Add (dwNum, data, &dwStart);

if (hRes) return hRes;

// Free the command data.

free (data.pData);

// Reactivate the menu.

hRes = pMenu->Activate(ghwndDialog, 0);

return hRes;

}

// GetCommands - takes a block of memory containing command strings and

// converts it into a list of VCMDCOMMAND structures.

// Returns TRUE if successful or FALSE otherwise.

// pszMemory - address of a null-terminated string of commands

// separated by \n or \r characters

// pData - address of an SDATA structure that receives the address and

// size of the memory containing the VCMDCOMMAND structures. The

// caller must free the memory by using the free function.

// pdwNumCommands - receives the count of VCMDCOMMAND structures in the

// memory block

BOOL GetCommands(char *pszMemory, PSDATA pData, DWORD *pdwNumCommands)

{

PSTR pTemp;

DWORD dwTotal, dwSize, dwSizeDesc, dwSizeCat;

DWORD dwSizeCmd;

PVCMDCOMMAND pCmd, pCmdNew;

CHAR *pszBegin;

DWORD dwCmdSize;

DWORD dwCmds = 0; // current count

DWORD dwCount = 1; // command number

char szCat[] = "Main";

dwTotal = dwSize = 0;

pTemp = (PSTR) malloc(0);

if (!pTemp)

return FALSE;

pCmd = (PVCMDCOMMAND) pTemp;

for( ;; ) {

// Retrieve the next command from the list of command strings.

pszMemory = NextCommand (pszMemory, &pszBegin, &dwCmdSize);

if (!pszMemory)

break; // no more

dwSize = sizeof(VCMDCOMMAND); // size of header

dwSizeCmd = (dwCmdSize + 1); // get command-string length

// Align the command string on a doubleword boundary.

dwSizeCmd += 3;

dwSizeCmd &= (~3);

dwSize += dwSizeCmd;

dwSizeDesc = (dwCmdSize + 1); // get description length

// Align the description on a doubleword boundary.

dwSizeDesc += 3;

dwSizeDesc &= (~3);

dwSize += dwSizeDesc;

dwSizeCat = lstrlen(szCat) + 1; // get category length

// Align the category string on a doubleword boundary.

dwSizeCat += 3;

dwSizeCat &= (~3);

dwSize += dwSizeCat;

dwSize += sizeof(DWORD); // action indicator

// Accumulate the total size of the command.

dwTotal += dwSize;

// Reallocate enough memory to hold the command.

pTemp = (PSTR) realloc((PVOID) pCmd, dwTotal);

// Fill the new VCMDCOMMAND structure.

pCmd = (PVCMDCOMMAND) pTemp;

pTemp += (dwTotal-dwSize);

pCmdNew = (PVCMDCOMMAND) pTemp;

memset (pCmdNew, 0, dwSize);

pCmdNew->dwSize = dwSize;

pCmdNew->dwFlags = 0;

pCmdNew->dwAction = (DWORD) (pCmdNew->abData - (PBYTE) pTemp);

pCmdNew->dwActionSize = sizeof(DWORD);

pCmdNew->dwCommandText = NULL;

// Point past the header to the beginning of the command data.

pTemp += (pCmdNew->abData - (PBYTE) pTemp);

// Set the action index.

*(DWORD *) pTemp = dwCount++;

pTemp += sizeof(DWORD);

// Set the command string.

pCmdNew->dwCommand = (DWORD) ((PBYTE) pTemp - (PBYTE) pCmdNew);

strncpy(pTemp, pszBegin, dwCmdSize);

pTemp += dwSizeCmd;

// Set the description.

pCmdNew->dwDescription =

(DWORD) ((PBYTE) pTemp - (PBYTE) pCmdNew);

strncpy(pTemp, pszBegin, dwCmdSize);

pTemp += dwSizeDesc;

// Set the category.

pCmdNew->dwCategory = (DWORD) ((PBYTE) pTemp - (PBYTE) pCmdNew);

strcpy(pTemp, szCat);

// Increment the count of commands.

dwCmds++;

}

pData-pData = (PVOID) pCmd;

pData-dwSize = dwTotal;

*pdwNumCommands = dwCmds;

return TRUE;

}

// NextCommand - finds the next command in a block of memory.

// Returns the address of the next command string in the block, or NULL

// if no command string is found.

// pszMemory - address of a null-terminated string of commands

// separated by \n or \r characters

// pBegin - receives the address of the next command string

// pdwSize - receives the number of bytes in the command string, not

// including any terminating NULL character

CHAR * NextCommand (CHAR *pszMemory, PCHAR *pBegin, DWORD *pdwSize)

{

DWORD i;

for( ;; ) {

// Try to find a non-newline character.

while ((*pszMemory == '\n') || (*pszMemory == '\r')) {

if (*pszMemory == '\0')

return NULL;

pszMemory++;

};

// Try to find a newline character.

for (i = 0;

(pszMemory[i] != '\n') && (pszMemory[i] != '\r') &&

(pszMemory[i] != '\0'); i++);

if (!i) {

if (!pszMemory[i])

return NULL; // end

pszMemory++;

continue; // try again

};

// A string was found. Return it.

*pBegin = pszMemory;

*pdwSize = i;

return pszMemory + i;

};

}


Processing Voice-Command Notifications

The voice-command object calls an application's IVCmdNotifySink interface to inform the application of voice-command events so that the application can respond to them. To receive notifications, the application must create a COM object that supports the IVCmdNotifySink interface and must pass the address of the interface to the voice-command object when calling the IVoiceCmd::Register function.

The IVCmdNotifySink interface consists of a set of member functions that correspond to voice-command events. When an event occurs on the site that the application is using, the voice-command object calls the member function that corresponds to the event.

The following example shows how to define an object class that implements the IVCmdNotifySink interface.

class CIVCmdNotifySink : public IVCmdNotifySink {

private:

DWORD m_dwMsgCnt;

HWND m_hwnd;

public:

CIVCmdNotifySink(void);

~CIVCmdNotifySink(void);

// IUnknown members

STDMETHODIMP QueryInterface (REFIID, LPVOID FAR *);

STDMETHODIMP_(ULONG) AddRef(void);

STDMETHODIMP_(ULONG) Release(void);

// IVCmdNotifySink members

STDMETHODIMP CommandRecognize (DWORD, PVCMDNAME, DWORD,

DWORD, PVOID, DWORD,PSTR, PSTR);

STDMETHODIMP CommandOther (PVCMDNAME, PSTR);

STDMETHODIMP MenuActivate (PVCMDNAME, BOOL);

STDMETHODIMP UtteranceBegin (void);

STDMETHODIMP UtteranceEnd (void);

STDMETHODIMP CommandStart (void);

STDMETHODIMP VUMeter (WORD);

STDMETHODIMP AttribChanged (DWORD);

STDMETHODIMP Interference (DWORD);

};

typedef CIVCmdNotifySink * PCIVCmdNotifySink;

The following example shows a simple implementation of the IVCmdNotifySink::CommandRecognize member function. The voice-command object calls CommandRecognize when the engine recognizes an utterance from the user as a voice command from one of the application's voice menus. The example starts the Windows Calculator application when the user speaks the "Start Calculator" command.

STDMETHODIMP CIVCmdNotifySink::CommandRecognize(DWORD dwID,

PVCMDNAME pName, DWORD dwFlags, DWORD dwActionSize, PVOID pAction,

DWORD dwNumLists, PSTR pszListValues, PSTR pszCommand)

{

if (!lstrcmp(pszCommand, "Start Calculator"))

WinExec("calc.exe", SW_SHOWNORMAL);

return NOERROR;

}


Objects

Voice-command objects provide simple speech-recognition capabilities — sometimes called command and control — for applications. This section describes these objects and the interfaces that they support.

In addition to the interfaces listed in this section, each object supports IUnknown, which is a common interface supported by all OLE component objects. For information about IUnknown, see the section, "Common Elements of the Speech Application Programming Interface."


Voice-Command Object


Purpose

A voice-command object manages the use of voice commands by all voice-aware applications on a user's computer. The voice-command object represents a voice-command site consisting of an audio device and a speech-recognition mode provided by an engine. Internally, a voice-command site consists of two low-level speech-recognition objects: a speech-recognition engine object and an audio source.

The voice-command object can use any speech-recognition engine that supports lists. An engine that does not support lists cannot be used for voice commands.


Created by

You create a voice-command object by calling the CoCreateInstance function with the CLSID_VCmd class identifier and the IID_IVoiceCmd interface identifier. CoCreateInstance returns a pointer to the IVoiceCmd interface for the voice-command object.


Interacts with

The voice-command object calls an application's IVCmdNotifySink interface to notify the application of speech-related events. An application must create a COM object that supports the IVCmdNotifySink interface and must pass the address of the interface to the voice-command object when calling the IVoiceCmd::Register member function. The application developer determines how the functions in the interface respond to notifications.

An application creates a voice-menu object by using the IVoiceCmd::MenuCreate member function. It can enumerate available voice menus by using a voice-menu enumerator created by the IVoiceCmd::MenuEnum member function.


Interfaces

The voice-command object supports the following interfaces.

Interface | Description
IVCmdAttributes | Controls attributes of a voice-command site, such as the audio device, the microphone, the speech-recognition mode, and whether speech-recognition is awake and enabled for the site. To get the address of the IVCmdAttributes interface, use the IVoiceCmd::QueryInterface member function with the IID_IVCmdAttributes interface identifier.
IVCmdDialogs | Displays Windows dialog boxes that allow a user to configure the speech-recognition engine. These dialog boxes are defined by the engine.
IVoiceCmd | Registers an application to use voice commands on a site, allows an application to create voice-menu objects and voice-menu enumerators, and controls recognition of voice commands.


Voice-Menu Enumerator


Purpose

A voice-menu enumerator enumerates the voice menus stored in the voice-menu database.


Created by

You create a voice-menu enumerator by using the IVoiceCmd::MenuEnum member function. MenuEnum returns a pointer to an IVCmdEnum interface on the object. When calling MenuEnum, you can enumerate only menus that are already open, only those that are active, or all menus in the database.

The voice-menu enumerator persists until all references to it are released, even if the voice-command object is released first.


Interacts with

An application uses the OLE IEnumX enumeration functions Clone, Next, Reset, and Skip to obtain the address of a VCMDNAME structure that identifies the desired voice menu. The application passes the address of the structure to the IVoiceCmd::MenuCreate member function to create a voice-menu object.


Interfaces

The voice-menu enumerator supports the IVCmdEnum interface, which enumerates the voice menus stored in the voice-menu database, using VCMDNAME structures as the elements in the enumeration sequence.


Voice-Menu Object


Purpose

A voice-menu object represents an open voice menu, which is a set of voice commands that an application can recognize when the menu is active.


Created by

You create a voice-menu object by using the IVoiceCmd::MenuCreate member function with a pointer to a VCMDNAME structure that identifies the menu to open. MenuCreate returns a pointer to the IVCmdMenu interface on a voice-menu object for the menu.

When calling MenuCreate to create a new voice menu, you can specify whether the voice-command object stores the menu in the voice-menu database or deletes the menu when the voice-menu object is released.


Interface

The voice-menu object supports the IVCmdMenu interface, which allows an application to activate or deactivate a voice menu, request training for the menu, or define the commands in a voice menu, including retrieving and setting lists to use within the context of a command.


Voice-Command Quick-Look

This section lists and briefly describes the voice-command interfaces and their member functions.


IVCmdAttributes

This interface is supported by the voice-command object.

Member function | Description
AutoGainEnableGet | Gets the state of the automatic gain for the incoming audio signal.
AutoGainEnableSet | Sets the state of the automatic gain for the incoming audio stream.
AwakeStateGet | Indicates whether or not the site is awake for purposes of speech recognition.
AwakeStateSet | Temporarily suspends or resumes speech recognition for a site.
DeviceGet | Returns the device ID of the wave-in audio device used for the site.
DeviceSet | Changes the wave-in audio device for the site.
EnabledGet | Indicates whether or not speech recognition is enabled for the site.
EnabledSet | Enables or disables speech recognition for a site.
MicrophoneGet | Gets the name of the microphone used by the site.
MicrophoneSet | Sets the name of the current microphone for a site.
SpeakerGet | Gets the name of the current speaker for the site.
SpeakerSet | Sets the name of the current speaker for the site.
SRModeGet | Gets the globally unique identifier (GUID) of the speech-recognition mode used for the site.
SRModeSet | Sets the speech-recognition mode used by the site.
ThresholdGet | Gets the threshold level of the speech-recognition engine used by the site.
ThresholdSet | Sets the threshold level for the speech-recognition engine used by a site.


IVCmdDialogs

This interface is supported by the voice-command object.

Member function | Description
AboutDlg | Displays an About dialog box that identifies the speech-recognition engine and contains the copyright notice.
GeneralDlg | Allows general control of the engine and should invoke some of the other dialog boxes.
LexiconDlg | Allows the speaker to display and edit his or her pronunciation lexicon.
TrainGeneralDlg | Provides phonetic training, which trains the engine for a preselected set of words.


IVCmdEnum

This interface is supported by the voice-menu enumerator.

Member function | Description
Clone | Returns another enumerator containing the same enumeration state as the current one.
Next | Retrieves the specified number of items in the enumeration sequence.
Reset | Resets the enumeration sequence back to the beginning.
Skip | Skips a specified number of elements in the enumeration sequence.


IVCmdMenu

This interface is supported by the voice-menu object.

Member function | Description
Activate | Activates the voice menu so that its commands can be recognized.
Add | Adds one or more commands to a voice menu.
Deactivate | Deactivates the selected menu so that its commands are no longer recognized.
EnableItem | Permanently enables or disables a menu item. When a command is disabled by using EnableItem, it is not compiled into the menu.
Get | Gets information about one or more commands in a voice menu.
ListGet | Gets the phrases stored in the current list for the selected voice menu.
ListSet | Sets the phrases in a list for a voice command.
Num | Indicates the total number of commands on a voice menu.
Remove | Removes the specified commands from the voice menu.
Set | Sets information for one or more commands in a voice menu.
SetItem | Temporarily enables or disables a command on a voice menu. If a command is disabled by using SetItem, it is compiled into the menu but not recognized.
TrainMenuDlg | Displays a dialog box to train the engine for the selected menus.


IVCmdNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
AttribChanged | Notifies applications on a voice-command site that a site attribute has changed.
CommandOther | Notifies applications on a site that a command was recognized by another application, or that it was not recognized by any application.
CommandRecognize | Notifies applications on a site that an utterance was recognized as a command on a voice menu.
CommandStart | Notifies applications on a site that recognition processing has begun for a command.
Interference | Notifies the application that the engine cannot recognize speech properly for a known reason.
MenuActivate | Notifies applications on a site that a voice menu has been activated or deactivated.
UtteranceBegin | Notifies applications on a site that the speech-recognition has detected the beginning of an utterance or sound.
UtteranceEnd | Notifies applications on a site that an utterance has ended.
VUMeter | Notifies the application of the current VU level, which indicates the loudness of the digital-audio stream.


IVoiceCmd

This interface is supported by the voice-command object.

Member function | Description
CmdMimic | Supplies a voice-aware application with the equivalent of a spoken voice command.
MenuCreate | Creates a voice-menu object to represent a new or existing voice menu for an application.
MenuDelete | Deletes a menu from the voice-command database.
MenuEnum | Creates a voice-menu enumerator that allows an application to enumerate menus in the voice-command database.
Register | Registers an application to use voice commands on a site. An application must call this function before it can use voice commands.


Voice-Command Structures

Structure

Description

Member functions

LANGUAGE

Contains the name of a language and dialect.

IVoiceCmd::MenuCreate

SDATA

Contains a pointer to data and the data's size.

IVCmdMenu::Add
IVCmdMenu::Get
IVCmdMenu::ListGet
IVCmdMenu::ListSet
IVCmdMenu::Set

VCMDCOMMAND

Provides information about a command in a voice menu. A variable-length list of VCMDCOMMAND structures is passed to these functions in an SDATA structure.

IVCmdMenu::Add
IVCmdMenu::Get
IVCmdMenu::Set

VCMDNAME

Contains strings that uniquely identify the application and state to which a voice menu belongs.

IVCmdEnum::Clone
IVCmdEnum::Next
IVCmdEnum::Reset
IVCmdEnum::Skip
IVCmdNotifySink:
:CommandOther
IVCmdNotifySink:
:CommandRecognize
IVCmdNotifySink:
:MenuActivate
IVoiceCmd::CmdMimic
IVoiceCmd::MenuCreate
IVoiceCmd::MenuDelete

VCSITEINFO

Provides information about the audio device, speech-recognition mode, and other attributes of a voice-command site.

IVoiceCmd::Register


Interfaces and Member Functions

Voice-command interfaces contain functions that allow an application to implement simple speech recognition, or command and control. There are the following interfaces.

Interface | Description
IVCmdAttributes | Controls the attributes of a voice-command site.
IVCmdDialogs | Displays Windows dialog boxes that allow the user to configure the speech-recognition engine.
IVCmdEnum | Enumerates the menus in the voice-menu database.
IVCmdMenu | Allows an application to activate or deactivate a voice menu, request training for the menu, or define the commands in the menu.
IVCmdNotifySink | Used by a voice-command object to notify an application of a command or utterance, a menu activation, or a change in the current VU level on a voice-command site.
IVoiceCmd | Registers an application with a voice-command object and controls recognition of voice commands.

This section describes, in alphabetical order, the interfaces and functions that support voice commands.

In addition to the interfaces described in this section, each object supports IUnknown, which is a common interface supported by all OLE component objects. For information about IUnknown, see section, "Common Elements of the Speech Application Programming Interface."


IVCmdAttributes Interface

The IVCmdAttributes interface allows an application to retrieve and set the attributes of a voice-command site. The application must call the IVoiceCmd::Register member function before it can call any of the functions in IVCmdAttributes.

A voice-command application may not need to use IVCmdAttributes functions if a voice-navigation application is installed on the user's computer. The voice-navigation application typically monitors and controls the attributes of a voice-command site.

Changing an attribute of a site affects all applications using the site. Settings are saved between uses of the site, even if the user shuts off his or her computer in the meantime.

The IVCmdAttributes interface is supported by the voice commands object. It has these member functions:

AutoGainEnableGet | MicrophoneGet
AutoGainEnableSet | MicrophoneSet
AwakeStateGet | SpeakerGet
AwakeStateSet | SpeakerSet
DeviceGet | SRModeGet
DeviceSet | SRModeSet
EnabledGet | ThresholdGet
EnabledSet | ThresholdSet


IVCmdAttributes::AutoGainEnableGet

HRESULT AutoGainEnableGet(DWORD *pdwAutoGain);

Retrieves the current automatic gain value for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_UNEXPECTED
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM

pdwAutoGain

[out] Address of a variable that receives the automatic gain.

A value of 0 indicates that automatic gain is disabled. A value of 100 indicates that the gain is set perfectly if the next utterance is spoken at the same level. A value between 0 and 100 moderates the automatic adjustments on a linear scale. For example, a pdwAudioGain value of 50 adjusts the gain to 50% of the level for the previous utterance.

When a voice-command object sends an IVCmdNotifySink::UtteranceEnd notification while automatic gain is enabled, it increases or decreases the gain on the audio device according to the engine's recommendation.

If the audio device does not support setting the gain on input, calling AutoGainEnableGet has no effect.

The automatic gain for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::AutoGainEnableSet

HRESULT AutoGainEnableSet(DWORD dwAutoGain);

Sets the automatic gain value for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwAutoGain

[in] Automatic gain value. This parameter can be SRATTR_MINAUTOGAIN or SRATTR_MAXAUTOGAIN for the minimum or maximum allowable value.

A value of 0 disables automatic gain. A value of 100 causes the voice-command object to set the gain to the value for the previous utterance so that if the next utterance is spoken at the same level, the gain is set perfectly. A value between 0 and 100 moderates the automatic gain adjustments on a linear scale. For example, a dwAudioGain value of 50 adjusts the gain to 50% of the level for the previous utterance.

If a voice-navigation application is installed on the user's computer, an application may not need to set the gain. If the audio device does not support setting the gain on input, calling AutoGainEnableSet has no effect.

The automatic gain for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::AwakeStateGet

HRESULT AwakeStateGet(DWORD *pdwAwake);

Retrieves the awake state for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pdwAwake

[out] Address of a variable that receives the current state of speech recognition for the site. This parameter is TRUE if the site is awake or FALSE if it is asleep.

When the site is awake, it listens for commands from any active voice menu for the active application. When the site is asleep, it listens for commands only from sleep menus — those that were activated with the dwFlags parameter of the IVCmdMenu::Activate member function set to the VWGFLAG_ASLEEP value. Commands from such menus become active only when the site is asleep, and they become inactive when the site is awake. A sleep menu typically contains a "Wake up!" command that resumes speech recognition, and it may contain other commands.


IVCmdAttributes::AwakeStateSet

HRESULT AwakeStateSet(DWORD dwAwake);

Sets the awake state for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwAwake

[in] TRUE to cause the site to wake up or FALSE to cause it to go to sleep.

If a voice-navigation application is installed on the user's computer, suspending speech recognition by using AwakeStateSet will typically cause the voice-navigation application to activate a "wake up" menu.

Calling AwakeStateSet allows the user to temporarily suspend speech recognition for a site. For example, the user might want to suspend speech recognition from the computer microphone during a telephone conversation and resume recognition when the conversation is finished. The user resumes recognition by speaking an appropriate command from a sleep menu — for example, "Wake up!"

The sleep state for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the sleep state. However, it may call this function to make sure that speech recognition is awake. For example, if an application speaks (with voice text or text-to-speech) "Do you want to print the document?" it might enable and wake up speech recognition for the site to receive the user's reply. The application should then restore speech recognition to its previous state.


IVCmdAttributes::DeviceGet

HRESULT DeviceGet(DWORD *pdwDeviceID);

Retrieves the device identifier of the wave-in audio device used for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pdwDeviceID

[out] Address of a variable that receives the device identifier of the audio device currently used by the site.

An application can obtain a list of devices by calling the waveInGetNumDevs and waveInGetDevCaps multimedia functions. Most applications do not need to retrieve or set the audio device.

The device for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::DeviceSet

HRESULT DeviceSet(DWORD dwDeviceID);

Changes the wave-in audio device for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwDeviceID

[in] Device identifier for the new audio device to use for the site. This parameter can be the WAVE_MAPPER value.

The device identifier is stored in the site's VCSITEINFO structure and does not change between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the device.


IVCmdAttributes::EnabledGet

HRESULT EnabledGet(DWORD *dwEnabled);

Finds out whether speech recognition is enabled or disabled for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwEnabled

[out] TRUE if speech recognition is enabled for the site or FALSE if it is disabled.

When speech recognition is disabled, the engine does not recognize any command from any menu, whether speech recognition is awake or asleep or any menus are active. An application would use the IVCmdAttributes::EnabledSet member function to allow the user to turn speech recognition completely off, as opposed to suspending speech recognition temporarily by putting the site to sleep.

The enabled state for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::EnabledSet

HRESULT EnabledSet(DWORD dwEnabled);

Enables or disables speech recognition for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwEnabled

[in] TRUE to enable speech recognition or FALSE to disable it.

Whenever speech is turned on or off, the WM_SPEECHSTARTED or WM_SPEECHENDED message is sent to all top-level windows in the system. An application can use these messages to determine when to enable or disable its voice commands or voice-text capabilities.

Calling EnabledSet allows the user to completely turn off speech recognition for a site so that nothing is recognized, including commands on sleep menus. For example, the user might want to disable speech recognition from the computer microphone during a meeting so that speech recognition will stay off, even if somebody inadvertently speaks a command on a sleep menu.

If a voice-navigation application is installed on the user's computer, an application may not need to set the enabled state. However, it may call this function to make sure that speech recognition is awake. For example, if an application speaks (with voice text or text-to-speech) "Do you want to print the document?" it might enable and wake up speech recognition for the site to receive the user's reply. The application should then restore speech recognition to its previous state.

Note, however that, if speech recognition is disabled, it is probably because the user does not want to use it. It may not be appropriate to enable speech recognition under those circumstances.

The enabled state for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::MicrophoneGet

HRESULT MicrophoneGet(PTSTR pszMicrophone, DWORD dwSize,
DWORD *pdwNeeded);

Retrieves the name of the microphone used by a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pszMicrophone

[in/out] Address of a buffer that receives a string identifying the current microphone.

dwSize

[in] Size, in bytes, of the buffer specified by pszMicrophone. If the buffer is too small, the function returns an error and fills pdwNeeded with the number of bytes needed to store the microphone string.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the microphone string.

Some speech-recognition engines use the microphone name to improve the accuracy of recognition.

The microphone for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::MicrophoneSet

HRESULT MicrophoneSet(PTSTR pszMicrophone);

Sets the name of the current microphone for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pszMicrophone

[in] Address of the string that contains the name of the microphone to set. If the current microphone is unknown, this parameter can be an empty string.

The microphone name for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the microphone name.


IVCmdAttributes::SpeakerGet

HRESULT SpeakerGet(PTSTR pszSpeaker, DWORD dwSize, DWORD *pdwNeeded);

Retrieves the name of the current speaker for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pszSpeaker

[in/out] Address of a buffer that receives the name of the current speaker.

dwSize

[in] Size, in bytes, of the buffer specified by pszSpeaker. If the buffer is too small, the function returns an error and fills pdwNeeded with the number of bytes needed to store the speaker string.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the speaker string.

Changing the speaker name unloads all training for the previous speaker and loads the training for the new speaker. If no training exists for the new speaker, the application starts with default training.

The speaker name for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::SpeakerSet

HRESULT SpeakerSet(PTSTR pszSpeaker);

Sets the name of the current speaker for a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pszSpeaker

[in] Address of the string that contains the name of the speaker to set. If the speaker is unknown, this parameter can be an empty string.

The speaker name for a site is saved between uses of the site, even if the user shuts down the computer in the meantime. The string is case insensative.

If a voice-navigation application is installed on the user's computer, an application may not need to set the speaker name.


IVCmdAttributes::SRModeGet

HRESULT SRModeGet(GUID *pgMode);

Retrieves the GUID of the speech-recognition mode used for the site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM

pgMode

[out] Address of a variable that receives the unique GUID assigned to the speech-recognition mode.

A speech-recognition engine typically provides an assortment of modes that it can use to recognize speech in different languages or dialects. A voice-command site uses a single speech-recognition mode.

The speech-recognition mode for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::SRModeSet

HRESULT SRModeSet(GUID gMode);

Sets the speech-recognition mode used by a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

gMode

[in] GUID of the speech-recognition mode to set for the site. If the mode does not exist, an error is returned and the mode is not changed.

The speech-recognition mode for a site is saved between uses of the site, even if the user shuts down the computer in the meantime. If a voice-navigation application is installed on the user's computer, an application may not need to set the speech-recognition mode.

An application can use a speech-recognition enumerator to determine which speech-recognition modes are available. For information about the speech-recognition enumerator, see section, "Speech Recognition."


IVCmdAttributes::ThresholdGet

HRESULT ThresholdGet(DWORD *pdwThreshold);

Retrieves the threshold level of the speech-recognition engine used by a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM

pdwThreshold

[out] Address of a variable that receives the threshold level.

The threshold level is a value from 0 to 100 that indicates the point below which an engine rejects an utterance as unrecognized. A value of 0 indicates that the engine should match any utterance to the closest phrase match. A value of 100 indicates that the engine should be absolutely certain that an utterance is the recognized phrase. For example, suppose the engine is expecting "What is the time?" If the threshold is 100 and the user mumbles "What'z tha time" or has a cold, the command may not be recognized. However, if the threshold is too low and the user says a similar-sounding phrase that is not being listened for such as "What is mine?" the engine may recognize it as "What is the time?"

If the command spoken by the user is not close enough to what the speech-recognition engine expects, the voice-command object notifies the application that the command was not recognized by calling IVCmdNotifySink::CommandOther with a NULL phrase.

The threshold for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.


IVCmdAttributes::ThresholdSet

HRESULT ThresholdSet(DWORD dwThreshold);

Sets the threshold level for the speech-recognition engine used by a voice-command site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwThreshold

[in] Threshold level. An application can specify SRATTR_MINTHRESHOLD and SRATTR_MAXTHRESHOLD for minimum and maximum allowable values.

The threshold level is a value from 0 to 100 that indicates the point below which an utterance is rejected as unrecognized. A threshold level of 0 indicates that the engine should match any utterance to the closest phrase match. A value of 100 indicates that the engine should be absolutely certain that an utterance is the recognized phrase. If the value is out of range for the engine, an error is returned and the attribute is not changed.

The threshold for a site is saved between uses of the site, even if the user shuts down the computer in the meantime.

If a voice-navigation application is installed on the user's computer, an application may not need to set the threshold.


IVCmdDialogs Interface

The IVCmdDialogs interface allows an application to display Windows dialog boxes to configure a speech-recognition engine. The controls and capabilities in these dialog boxes are determined by the particular speech-recognition engine that is being used. If a voice-navigation application is installed on the user's computer, an application does not need to use IVCmdDialogs member functions because the voice-navigation application provides access to the engine's dialog boxes.

Before an application can call any IVCmdDialogs member function, it must first call the IVoiceCmd::Register member function to register the voice-command site.

The IVCmdDialogs interface is supported by the voice-command object. It has these member functions:

AboutDlg

GeneralDlg

LexiconDlg

TrainGeneralDlg

TrainMicDlg


IVCmdDialogs::AboutDlg

HRESULT AboutDlg(HWND hWndParent, PTSTR pszTitle);

Displays an About dialog box that identifies the speech-recognition engine and contains the copyright notice.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the About dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVCmdDialogs::GeneralDlg

HRESULT GeneralDlg(HWND hWndParent, PTSTR pszTitle);

Displays a General dialog box, which gives the user general control of the speech-recognition engine and provides the user with full access to engine-specific controls.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the General dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Depending on the engine, the General dialog box may allow users to perform these tasks:

· Inform the engine of the speaker's gender or other speaker-specific information.

· Control how stringent the engine is about recognition.

· Control the maximum amount of disk space allotted for speaker information.

· Control optimizations that reduce processing.

· Export and import speaker information to and from a disk so that this information can be transferred between systems.


IVCmdDialogs::LexiconDlg

HRESULT LexiconDlg(HWND hWndParent, PTSTR pszTitle);

Displays a Lexicon dialog box, which allows the user to display and edit his or her pronunciation lexicon. For example, the speaker can edit the phonetics of mispronounced words and load or unload personal exception dictionaries.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Lexicon dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVCmdDialogs::TrainGeneralDlg

HRESULT TrainGeneralDlg(HWND hWndParent, PTSTR pszTitle);

Displays a General Training dialog box, which trains a speech-recognition engine by having the user speak a preselected set of words. After the user speaks these words and phrases, recognition accuracy should be better for that particular user.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the General Training dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVCmdDialogs::TrainMicDlg

HRESULT TrainMicDlg(HWND hWndParent, PTSTR pszTitle);

Displays a Microphone Training dialog box, which trains the speech-recognition engine with training for a microphone. This dialog box should be displayed at the request of the engine or whenever the speaker wants to display it.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Microphone Training dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Training information will be associated with both the speaker and the microphone. Ideally, TrainMicDlg will train the engine only for the microphone and any surrounding sounds, but it is also likely to train for the speaker's voice.

If the engine does not supply this dialog box, the speaker cannot train it for the microphone; however, the application will still work.


IVCmdEnum Interface

The IVCmdEnum interface allows an application to find and open a menu from the voice-menu database. The set of menus is determined by the calling parameters of the IVoiceCmd::MenuEnum member function used to create the voice-menu enumerator. An application can determine which applications have voice menus, which states the menus are valid in, which menus are currently open, or which menus are currently active.

IVCmdEnum functions use VCMDNAME structures as the elements in the enumeration sequence.

The IVCmdEnum interface is supported by the voice-menu enumerator. It has these member functions:

Clone

Next

Reset

Skip

For information about Clone, Next, Reset, and Skip, see section, "Common Elements of the Speech Application Programming Interface."


IVCmdMenu Interface

The IVCmdMenu interface allows an application to activate or deactivate a voice menu, request training for the menu, or define the commands on the menu.

The IVCmdMenu interface is supported by the voice-menu object. It has these member functions:

Activate | ListSet
Add | Num
Deactivate | Remove
EnableItem | Set
Get | SetItem
ListGet | TrainMenuDlg


IVCmdMenu::Activate

HRESULT Activate(HWND hWndListening, DWORD dwFlags);

Activates a voice menu so that its commands can be recognized.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_CANTCREATEDATASTRUCTURES
VCMDERR_CANTINITDATASTRUCTURES
VCMDERR_CANTXTRACTWORDS
VCMDERR_INVALIDWINDOW
VCMDERR_MENUACTIVE
VCMDERR_MENUTOOCOMPLEX
VCMDERR_MENUWRONGLANGUAGE
VCMDERR_NOCACHEDATA
VCMDERR_NOENGINE
VCMDERR_NOGRAMMARINTERFACE
VCMDERR_OUTOFMEM
VCMDERR_TOOMANYMENUS

hWndListening

[in] Handle of the window associated with the voice menu. Whenever this window is the foreground window, the voice menu is automatically activated. Otherwise, it is deactivated. If this parameter is NULL, the voice menu is global (that is, it remains active regardless of the foreground window, until the application explicitly deactivates it).

dwFlags

[in] Flag that indicates whether the menu should be active when speech-recognition is "asleep" for the voice-command site. This parameter can be one of these values:

Value | Meaning
0 or NULL | The voice menu is active only when speech recognition is awake.
VWGFLAG_ASLEEP | The menu is active only when speech recognition is asleep and is automatically deactivated when speech recognition is awake.

Most applications set this parameter to zero. Typically, a sleep menu contains a command to resume speech recognition, such as "Wake up."

A global voice menu is useful for an application such as a clock program so that the user can ask what time it is and get a response no matter what else he or she is doing. Global voice-menu commands have a lower priority in case of a recognition conflict — for example, two commands with the same name in different menus.


IVCmdMenu::Add

HRESULT Add(DWORD dwCmdNum, SDATA dData, DWORD *pdwCmdStart);

Adds one or more commands to a voice menu. The added commands are appended to any existing commands in the menu.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwCmdNum

[in] Number of commands to add to the menu.

dData

[in] SDATA structure containing a list of VCMDCOMMAND structures that describe the voice commands to be added. Although they vary in size depending on the command data, the structures are contiguous within the list.

pdwCmdStart

[out] Address of a variable that receives the number of the first command added to the menu.

Commands are numbered sequentially from 1 to n. New commands are added to the end of the menu, so the first command added is numbered n+1.

For best results, you should deactivate the voice menu before calling Add. Otherwise, the menu must be deactivated, recompiled, and reactivated before Add returns. If the menu is already deactivated when Add is called, the menu is not recompiled until the application activates it again.

If a command string includes a list name, you can use IVCmdMenu::ListSet to set the phrases that the user can substitute for the list name when speaking the command.


IVCmdMenu::Deactivate

HRESULT Deactivate(void);

Deactivates an active voice menu so that the application no longer listens for its commands. The menu is still open, so the application can start listening for the menu's commands again by calling IVCmdMenu::Activate to reactivate the menu.

· Returns NOERROR if successful, or VCMDERR_OUTOFMEM if a low memory condition exists.


IVCmdMenu::EnableItem

HRESULT EnableItem(DWORD dwEnable, DWORD dwCmdNum, DWORD dwFlag);

Permanently enables or disables a menu item. When a command is disabled by using EnableItem, it is not compiled into the menu.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_OUTOFMEM

dwEnable

[in] TRUE to enable the command, or FALSE to disable it.

dwCmdNum

[in] Position or identifier of the command on the menu, depending on the value of dwFlag. Command positions are sequential, starting with 1 for the first command on the menu. The command identifier is specified in the dwID member of the VCMDCOMMAND structure that defines the command.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

For best results, you should deactivate the voice menu before calling EnableItem. Otherwise, the menu must be deactivated, recompiled, and reactivated before the function returns. If the menu is already deactivated when EnableItem is called, the menu is not recompiled until the application activates it again.


IVCmdMenu::Get

HRESULT Get(DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag,
PSDATA pdData, DWORD *pdwCmdNum);

Retrieves information about one or more commands in a voice menu.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwCmdStart

[in] Number of the first command to retrieve. Commands are numbered sequentially from 1 to n. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Either the number of commands to retrieve or the identifier of the commands, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the total number of commands in the menu, the function returns as many commands as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

pdData

[out] Address of an SDATA structure that receives the address and size of a buffer. The buffer contains a list of VCMDCOMMAND structures that describe the commands retrieved. Although they vary in size depending on the command data, the structures are contiguous within the list.

pdwCmdNum

[out] Address of a variable that receives the number of commands actually copied to the buffer.

The calling application allocates the SDATA structure and passes its address to Get. Get allocates memory (using the OLE task allocator) for the returned data and sets the pData member of SDATA to point to the memory. If the allocation fails, pData is set to NULL and the dwSize member is set to zero. The calling application must free the memory pointed to by pData as well as the SDATA structure itself.

The calling application must free the memory allocated by the member function by using the CoTaskMemFree function.


IVCmdMenu::ListGet

HRESULT ListGet(PTSTR pszList, PSDATA pdList, DWORD *pdwListNum);

Retrieves the phrases stored in the current list for the selected voice menu. A list is associated with a menu rather than an individual command. The list must appear in at least one command string, but can be used by more than one command on the menu.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDLIST
VCMDERR_OUTOFMEM

pszList

[in] Name of the list, such as "name" or "weekday." The list name must appear in the command string for at least one command on the menu. The command string is stored in the dwCommand member of the VCMDCOMMAND structure that defines the command.

pdList

[out] Address of an SDATA structure that receives the address and size of a buffer. The buffer contains a sequential list of null-terminated strings, one for each phrase in the list.

pdwListNum

[out] Address of a variable that receives the number of phrases that were copied to the buffer. If the list is empty, this parameter receives zero.

The calling application allocates the SDATA structure and passes its address to ListGet. ListGet allocates memory (using the OLE task allocator) for the returned data and sets the pData member of the SDATA structure to point to the memory. If the allocation fails, the pData member is set to NULL and the dwSize member is set to zero. The calling application must free the memory pointed to by pData, as well as the SDATA structure itself.

It is up to the calling application to free the memory allocated by the member function by using the CoTaskMemFree function.


IVCmdMenu::ListSet

HRESULT ListSet(PTSTR pszList, DWORD dwListNum, SDATA dList);

Sets the phrases in a list for a voice command. The user can speak any phrase in the list in place of the list name in the command string.

A command that uses a list must have the list name in brackets. Example:

"Send mail to <name>"

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_INVALIDLIST
VCMDERR_OUTOFMEM

pszList

[in] Address of the name of the list to set, such as "name" or "weekday." The list name must appear in the command string for at least one command on the menu. The command string is specified in the dwCommand member of the VCMDCOMMAND structure that defines the command.

dwListNum

[in] Number of phrases in the list.

dList

[in] SDATA structure that contains a pointer to a data buffer and the size of the buffer. The data buffer contains a sequential list of null-terminated strings, one for each phrase in the list.

Calling ListSet establishes a list of phrases that can be spoken in a voice command, such as "Send mail to name." Typically, the list contains information that changes dynamically at run time, such as the ten people to whom the user most recently sent electronic mail. For best results, a list should have fewer than 20 entries. Having more than 20 entries in a list can reduce the accuracy of recognition.

The list persists until the voice-menu object is released. List entries are not automatically saved to disk. To preserve the list, call the IVCmdMenu::ListGet member function and take steps to store the result.

ListSet is much faster than the IVCmdMenu interface's Add, Remove, or Set member functions because list entries are substituted when a command is recognized and the menu is not recompiled. This means that ListSet can be called on an active menu without affecting performance.


IVCmdMenu::Num

HRESULT Num(DWORD *pdwNumCmd);

Retrieves the total number of commands on a voice menu.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pdwNumCmd

[out] Address of a variable that receives the number of commands.


IVCmdMenu::Remove

HRESULT Remove(DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag);

Removes the specified commands from the voice menu.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwCmdStart

[in] Number of the first command in the menu to remove. Command positions are sequential, starting with 1 for the first command on the menu. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Number of commands to remove or the identifier of the commands, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the total number of commands in the menu, the function removes as many commands as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

For best results, you should deactivate the voice menu before calling Remove. Otherwise, the menu must be deactivated, recompiled, and reactivated before Remove returns. If the menu is already deactivated when Remove is called, the menu is not recompiled until the application activates it again.


IVCmdMenu::Set

HRESULT Set(DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag,
SDATA dData);

Sets information for one or more commands in a voice menu.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwCmdStart

[in] Number of the first command to set in the voice menu. Command positions are sequential, starting with 1 for the first command on the menu. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Either the number of commands to set or the identifier of the commands, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the number of commands in the menu, the function sets as many commands as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

dData

[in] SDATA structure that contains a pointer to a data buffer and the size of the buffer. The data buffer contains a list of VCMDCOMMAND structures that describe the voice commands to set. Although they vary in size depending on the command data, the structures are contiguous within the list.

For best results, you should deactivate the voice menu before calling Set. Calling Set on an active menu can be fairly slow because the menu must be deactivated, recompiled, and reactivated before Set returns. If the menu is already deactivated when Set is called, the menu is not recompiled until the application activates it again.


IVCmdMenu::SetItem

HRESULT SetItem(DWORD dwEnable, DWORD dwCmdNum, DWORD dwFlag);

Temporarily enables or disables a command on a voice menu. If a command is disabled by using SetItem, the voice-command object sends a CommandOther notification rather than a CommandRecognize notification when it "recognizes" the disabled command.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_OUTOFMEM

dwEnable

[in] TRUE to enable the command or FALSE to disable it.

dwCmdNum

[in] Position or identifier of the command on the menu, depending on the value of dwFlag. Command positions are sequential, starting with 1 for the first command on the menu.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

SetItem is much faster than the IVCmdMenu::EnableItem member function because the menu is not recompiled. This means that SetItem can be called on an active menu without affecting performance.


IVCmdMenu::TrainMenuDlg

HRESULT TrainMenuDlg (HWND hWndParent, PTSTR pszTitle);

Displays a modal dialog box used to train the engine for the selected menu. The controls in the Train Menu dialog box and the training it requests depend upon the speech-recognition engine being used. Some engines may not offer this dialog box.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window for the dialog box. If this parameter is NULL, the engine returns NOERROR if it supports the dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

This dialog box is provided by the speech-recognition engine. An application can call TrainMenuDlg to obtain training for the commands on a particular menu. In contrast, the training obtained by calling the IVCmdDialogs::TrainGeneralDlg member function applies to all commands, regardless of the menu they are on.


IVCmdNotifySink Interface

The IVCmdNotifySink interface is used by a voice-command object to return status information to an application. This interface is passed to the voice-command object by an application when it calls the IVoiceCmd::Register member function to register itself to use voice commands.

Notifications are sent to all applications on the voice-command site, not just the application that caused the notification to be sent. An application can choose to receive all possible notifications or only the IVCmdNotifySink::CommandRecognize notification.

The IVCmdNotifySink interface has these member functions:

AttribChanged | MenuActivate
CommandOther | UtteranceBegin
CommandRecognize | UtteranceEnd
CommandStart | VUMeter
Interference |


IVCmdNotifySink::AttribChanged

HRESULT AttribChanged(DWORD dwAttribute);

Notifies applications on a voice-command site that a site attribute has changed.

· Return value is ignored.

dwAttribute

[in] Site attribute that was changed. This parameter can be one of these values:

Value | Meaning
IVCNSAC_AWAKE | Awake state.
IVCNSAC_AUTOGAINENABLE | Automatic gain.
IVCNSAC_DEVICE | Wave-in audio device.
IVCNSAC_ENABLED | Enabled state.
IVCNSAC_MICROPHONE | Current microphone.
IVCNSAC_ORIGINAPP | The application receiving this notification originated the attribute change.
IVCNSAC_SPEAKER | Name of the current speaker.
IVCNSAC_SRMODE | Speech-recognition mode.
IVCNSAC_THRESHOLD | Confidence threshold.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.

dwAttribute includes the IVCNSAC_ORIGINAPP value only if the application sets an attribute by calling the IVCmdAttributes interface's EnabledSet, AwakeStateSet, DeviceSet, or SRModeSet member function.


IVCmdNotifySink::CommandOther

HRESULT CommandOther(PVCMDNAME pName, PTSTR pszCommand);

Sent when a spoken phrase was either recognized as being from another application's command set or was not recognized. Along with the notification, the application receives the address of the phrase.

· Return value is ignored.

pName

[in] Address of a VCMDNAME structure that contains the name of the voice menu that has the recognized command. If this parameter contains NULL, the command was not recognized.

pszCommand

[in] Address of the command string. If this parameter contains NULL, the command was not recognized.

An application can use the CommandOther notification to monitor utterances and inform the user what was heard. An application should not rely on this notification for information about the recognition of its own commands. Most applications ignore this notification.

The command string contains the words actually spoken by the user. If the command contains a list name, the command string may not match the words of the command. For example, the string pointed to by pszCommand might be "Send mail to Fred" whereas the command string is "Send mail to name."

The notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.

If two or more voice menus contain the same phrase and this phrase is recognized, it is indeterminate which of the menus will cause the engine to call the IVCmdNotifySink::CommandRecognize notification and which will cause it to call CommandOther. This happens only if the menus are all global or all window specific.


IVCmdNotifySink::CommandRecognize

HRESULT CommandRecognize(DWORD dwID, PVCMDNAME pvCmdName, DWORD dwFlags,
DWORD dwActionSize, PVOID pAction, DWORD dwNumLists,
PTSTR pszListValues, PTSTR pszCommand);

Sent when a spoken phrase was recognized as being from the application's command set. Along with the notification, the application receives the text of the phrase and the action data that was supplied by the application when it originally defined the command.

· Return value is ignored.

dwID

[in] Identifier of the command that was recognized. The command identifier is stored in the dwID member of the VCMDCOMMAND structure that defines the command.

pvCmdName

[in] Address of a VCMDNAME structure containing the voice menu that has the recognized command.

dwFlags

[in] VCMDCMD_VERIFY if the application should request verification from the user or NULL if verification is not required. To request verification, the application should display a dialog box. An application would typically require verification for a destructive or irreversible command such as "Format disk."

dwActionSize

[in] Size of the data in pAction.

pAction

[in] Address of a string that contains action data to accompany the recognized command. The action data is obtained from the VCMDCOMMAND structure for the command.

dwNumLists

[in] Size, in bytes, of the list data for the command. If a command does not contain any list fields, this parameter is zero.

pszListValues

[in] Address of a list of one or more null-terminated strings that correspond to the phrase from each list in the order that they occur in the command. For example, if the command is "Set the time to number AM or PM," this parameter points to "Ten\0PM" (the last '\0' is implicit in C notation).

pszCommand

[in] Address of the command string for the command that was recognized.

You should not use the contents of pszCommand to identify the recognized command. Instead, use the data in pAction or the identifier in dwID to determine which command was recognized. The pszCommand string may not contain the same string that you specified in the VCMDCOMMAND structure because it is possible for the user to edit the text for commands for your application using Microsoft Voice or another voice-aware application.

The notification is sent to all applications that are registered on the voice-command site, regardless of the settings of the dwFlags parameter of the IVoiceCmd::Register member function when the application registered to use voice commands.

If two or more global voice menus (or two or more window-specific voice menus) contain the same phrase and the engine recognizes that phrase, the engine calls CommandRecognize for one menu and IVCmdNotifySink::CommandOther for the other. The engine determines which notification to call for each menu; an application cannot determine which notification will be called.


IVCmdNotifySink::CommandStart

HRESULT CommandStart(void);

Notifies applications on a voice-command site that recognition processing has begun for a command.

· Return value is ignored.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.

An application can use CommandStart to indicate that it is processing what the user said. Most applications will ignore this notification.


IVCmdNotifySink::Interference

HRESULT Interference(DWORD dwType);

Notifies the application that the engine cannot recognize speech properly for a known reason.

· Return value is ignored.

dwType

[in] Type of interference. This parameter can be one of these values:

Value | Meaning
SRMSGINT_AUDIODATA_STARTED | The engine has resumed receiving audio data from the audio source.
SRMSGINT_AUDIODATA_STOPPED | The engine has stopped receiving audio data from the audio source.
SRMSGINT_NOISE | The background noise is too high.
SRMSGINT_NOSIGNAL | The engine cannot detect a signal, possibly because the microphone is off or unplugged.
SRMSGINT_TOOLOUD | The speaker is too loud; recognition results may be degraded.
SRMSGINT_TOOQUIET | The speaker is too quiet; recognition results may be degraded.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVCmdNotifySink::MenuActivate

HRESULT MenuActivate(PVCMDNAME pName, BOOL bActive);

Notifies applications on a voice-command site that a voice menu has been activated or deactivated, either explicitly by the application or as a result of the user switching to or from the window associated with the menu.

· Return value is ignored.

pName

[in] Address of a VCMDNAME structure identifying the voice menu that was activated or deactivated.

bActive

[in] TRUE if the voice menu was activated or FALSE if the menu was deactivated.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_ALLBUTVUMETER value when the application registered to use voice commands on the site. Most applications will ignore this notification.


IVCmdNotifySink::UtteranceBegin

HRESULT UtteranceBegin(void);

Notifies applications on a voice-command site that the speech-recognition engine has detected the beginning of an utterance or sound.

· Return value is ignored.

Detecting the beginning of an utterance does not necessarily have any bearing on the recognition of commands. An UtteranceBegin notification implicitly signals the end of silence, so an application can use this notification to detect long periods of silence and react accordingly.

This notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVCmdNotifySink::UtteranceEnd

HRESULT UtteranceEnd(void);

Notifies applications on a voice-command site that an utterance is finished.

· Return value is ignored.

A speech-recognition engine typically sends this notification 0.25 to 0.5 of a second after speech stops. This notification does not necessarily have any bearing on the recognition of commands. An UtteranceEnd notification implicitly signals the beginning of silence, so an application can use this notification to detect long periods of silence and react accordingly.

This notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVCmdNotifySink::VUMeter

HRESULT VUMeter(WORD wLevel);

Notifies the application of the current VU level, which indicates the loudness of the digital-audio stream.

· Return value is ignored.

wLevel

[in] Value from 0 to 0xFFFF that indicates a linear level. 0xFFFF is the loudest volume that the VU meter will register.

If the speech-recognition engine for the site supports a VU meter, the engine sends this notification about 8 times per second.

This notification is sent only to applications that set the dwFlags parameter of the IVoiceCmd::Register member function to the VCMDRF_ALLMESSAGES or VCMDRF_VUMETER value when the application registered to use voice commands on the site.


IVoiceCmd Interface

The IVoiceCmd interface allows an application to register with a voice-command object and to control recognition of speech. The application must call the IVoiceCmd::Register member function before it can call certain other functions, such as those in IVCmdDialogs.

The IVoiceCmd interface is supported by the voice-command object. It has these member functions:

CmdMimic

MenuCreate

MenuDelete

MenuEnum

Register


IVoiceCmd::CmdMimic

HRESULT CmdMimic(PVCMDNAME pMenu, PTSTR pszCommand);

Supplies a voice-aware application with the equivalent of a spoken voice command.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_CANNOTMIMIC
VCMDERR_INVALIDCHAR
VCMDERR_MENUDOESNOTEXIST
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pMenu

[in] Address of a VCMDNAME structure identifying the menu that contains the command to mimic.

pszCommand

[in] Address of a string that contains the command to mimic.

CmdMimic parses the command string and eliminates white space and punctuation, and then the member function compares the result with each command on the voice menu until it finds a match. The comparison is case-insensitive, and the command string can include phrases from lists. If the string matches a command in the voice menu, it is recognized. Otherwise, the function returns an error.

An application can call CmdMimic to play back voice macros to another application, somewhat like playing back keystrokes and mouse messages in Windows.

The voice menu must be active before an application can mimic its commands.


IVoiceCmd::MenuCreate

HRESULT MenuCreate(PVCMDNAME pName, PLANGUAGE pLanguage, DWORD dwFlags,
PIVCMDMENU *ppIVCmdMenu);

Creates a voice-menu object to represent a new or existing voice menu for an application.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_CANTCREATESTORAGE
VCMDERR_MENUDOESNOTEXIST
VCMDERR_MENUEXIST
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

pName

[in] Address of a VCMDNAME structure that identifies the menu to create. The VCMDNAME structure contains an application name, such as "Excel," and a state name, such as "Main menu" or "File Open dialog box."

pLanguage

[in] Address of a LANGUAGE structure that indicates the language to use for the menu. If this parameter is NULL, the default language for the site's speech-recognition mode is used.

dwFlags

[in] Flag that indicates how to create the menu. This parameter can be one of these values:

Value | Meaning
VCMDMC_CREATE_ALWAYS | Creates an empty menu with the given name. If a menu by that name already exists in the voice-menu database, it is erased. The new menu is stored in the database when the menu object is released.
VCMDMC_CREATE_NEW | Creates an empty menu with the given name. If a menu by that name already exists in the voice-menu database, the function returns an error. The new menu is stored in the database when the menu object is released.
VCMDMC_CREATE_TEMP | Creates an empty menu with the given name. If a menu by that name already exists in the voice-menu database, the function returns an error. The new menu is temporary and is discarded when the menu object is released.
VCMDMC_OPEN_ALWAYS | Opens an existing menu with the given name. If the menu does not exist, the function creates a new, empty menu. The new menu is stored in the database when the menu object is released.
VCMDMC_OPEN_EXISTING | Opens an existing menu. If the menu does not exist, the function returns an error.

ppIVCmdMenu

[out] Address of an IVCmdMenu interface for the newly created voice-menu object. The application uses the pointer to this interface to call IVCmdMenu functions on the voice-menu object. If an error occurs, this parameter receives NULL.

An application can create a voice-menu object by loading an existing voice menu from the voice-menu database or creating a new voice menu. A voice menu need not be stored in the database; an application can create a temporary voice menu by setting dwFlags to the VCMDMC_CREATE_TEMP value. A temporary voice menu persists until the menu object is released.

An application can create more than one voice-menu object to represent the same menu — either one of its own menus or a menu for another application. For example, one application might do this to enumerate another application's menus.

More than one application can use the same voice-menu object. For example, Application A might call the IVoiceCmd::CmdMimic member function on a voice-menu object that represents a menu for Application B, while Application B uses the same menu object to recognize commands spoken by the user.


IVoiceCmd::MenuDelete

HRESULT MenuDelete(PVCMDNAME pName);

Deletes a menu from the voice-menu database. A menu cannot be deleted if it is currently open and the application is actively listening for its commands.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_MENUACTIVE
VCMDERR_MENUDOESNOTEXIST
VCMDERR_MENUOPEN
VCMDERR_OUTOFMEM

pName

[in] Address of a VCMDNAME structure that identifies the menu to delete.

This function deletes the storage in the database for the menu (if it exists) and releases the voice-menu object that was created by the IVoiceCmd::MenuCreate member function. After a menu is deleted, the pointer to its IVCmdMenu interface is invalid, so it should be set to NULL.


IVoiceCmd::MenuEnum

HRESULT MenuEnum(DWORD dwFlags, PTSTR pszApplicationFilter,
PTSTR pszStateFilter, PIVCMDENUM *ppiVCmdEnum);

Creates a voice-menu enumerator that allows an application to enumerate menus in the voice-menu database.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

dwFlags

[in] Indicates whether to enumerate active menus or open menus (those that have voice-menu objects, whether or not they are also active.) This parameter can be certain combinations of these values:

Value | Meaning
VCMDEF_ACTIVE | Enumerates only active menus.
VCMDEF_DATABASE | Enumerates all menus in the voice commands database.
VCMDEF_PERMANENT | Enumerates only permanent menus.
VCMDEF_SELECTED | Enumerates open menus, whether or not they are also active.
VCMDEF_TEMPORARY | Enumerates only temporary menus.

VCMDEF_ACTIVE and VCMDEF_SELECTED are mutually exclusive, as are VCMDEF_TEMPORARY and VCMDEF_PERMANENT. If both values are specified, the function returns an error. VCMDEF_TEMPORARY and VCMDEF_PERMANENT are ignored if neither VCMDEF_ACTIVE and VCMDEF_SELECTED are specified. In other words, they do not apply if you want to enumerate the menus in the database.

By definition, if a menu is active, it is selected.

pszApplicationFilter

[in] Address of the name of the application for which to enumerate menus. This name is the same as that in the szApplication member of the VCMDNAME structure passed to the IVoiceCmd::MenuCreate member function. If this parameter is NULL, menus for all applications, except those eliminated by dwFlags and pszStateFilter, are enumerated.

pszStateFilter

[in] Address of a string that contains the name of the state for which to enumerate menus. This is the same as in the szState member of the VCMDNAME structure passed to MenuCreate. If pszApplicationFilter is NULL, all menus except those eliminated by dwFlags and this parameter are enumerated.

ppiVCmdEnum

[out] Address of a variable that receives a pointer to an IVCmdEnum interface for the newly created voice-menu enumerator. If an error occurs, this parameter receives NULL.

An application can use a voice-menu enumerator to find and modify unknown menus or to show menu status to the user.

The voice-menu enumerator persists until all references to it are released, even if the voice-command object is released.


IVoiceCmd::Register

HRESULT Register(PTSTR pszSite, PIVCMDNOTIFYSINK pNotifyInterface,
IID IIDNotifyInterface, DWORD dwFlags, PVCSITEINFO pSiteInfo);

Registers an application to use voice commands on a site. An application must call this function before it can use voice commands.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_CANTCREATEAUDIODEVICE
VCMDERR_CANTCREATESRENUM
VCMDERR_CANTSELECTENGINE
VCMDERR_CANTSETDEVICE
VCMDERR_INVALIDMODE
VCMDERR_NOFINDINTERFACE
VCMDERR_NOSITEINFO
VCMDERR_OUTOFMEM
VCMDERR_SRFINDFAILED
VCMDERR_VALUEOUTOFRANGE

pszSite

[in] Address of a string that identifies the site the application wants to use. This parameter can be a null string ("") or NULL to use the computer, or "LineX" to use a phone line, where "X" is the number of the line. The site identifier is used to save settings for the site in the registry so that they remain in effect the next time an application registers to use the site. The site identifier can also be used to restore settings from the registry. If a site has not been used, the default attributes are likely to be set for the computer. All applications on a site share the attributes of that site, such as the instance of the speech-recognition engine.

-If a particular site (such as "LineX") requires an input device other than the default, the device must be set the first time the site is used. You can use the IVCmdAttributes::DeviceSet member function to set the device or specify the device in the VCSITEINFO structure pointed to by pSiteInfo. During subsequent registrations, this same device is used unless it is changed by using DeviceSet.

pNotifyInterface

[in] Address of the notification interface that receives notifications from the voice-command object. The interface identifier is specified by IIDNotifyInterface. If this parameter is NULL, no notifications will be sent.

Because passing the pointer to the voice-command object does not transfer ownership of the notification interface, the voice-command object must call the AddRef member function of the notification interface before returning from the call to Register. The voice-command object must also call the Release member function of the notification interface when it closes. The calling application must release any reference counts it holds on the notification interface after calling Register, unless it needs the notification object to be valid when the voice-command object releases it.

IIDNotifyInterface

[in] GUID that uniquely identifies the type of notification sink being passed to the voice-command object. It must be IID_IVCmdNotifySinkA (for ANSI) or IID_IVCmdNotifySinkW (for Unicode).

dwFlags

[in] Flag that indicates whether the application is to receive all notifications. This parameter can be one of these values:

Value | Meaning
VCMDRF_ALLMESSAGES | Sends all notifications to pNotifyInterface.
VCMDRF_ALLBUTVUMETER | Sends all but VUMeter notifications to pNotifyInterface.
VCMDRF_VUMETER | Sends VUMeter notifications to pNotifyInterface.
VCMDRF_NOMESSAGES | Does not send notifications.

If dwFlags is 0 (zero) or NULL, only the IVCmdNotifySink::CommandRecognize notification is sent.

pSiteInfo

[in] Address of a VCSITEINFO structure that contains settings to apply to the site, such as the speaker, confidence threshold, and speech-recognition mode. The settings are applied even if the site is already open. If this parameter is NULL, the voice-command object uses the settings from the registry. If there are no registry settings, it uses the default settings, typically those for the computer.

Telephony applications will pass this information to ensure that the proper settings are selected. Other applications will set this parameter to NULL to leave the site settings unchanged from previous values.

An application cannot call Register a second time for the same voice-command object. If a voice-command object is already registered, calling Register returns an error. To change sites, the application must call CoCreateInstance to create a new voice-command object for the desired site.

An application must call Register before it can call any of the following member functions:

IVCmdMenu::Deactivate
IVCmdMenu::ListGet
IVCmdMenu::ListSet
IVCmdMenu::TrainMenuDlg
Any IVCmdDialogs member function


Structures

Certain voice-command functions use structures to store and retrieve information at run time about a particular voice menu, command, or voice-command site. This section describes, in alphabetical order, the structures used by voice-command functions.


VCMDCOMMAND

typedef struct { // vccmd
DWORD dwSize;
DWORD dwFlags;
DWORD dwID;
DWORD dwCommand;
DWORD dwDescription;
DWORD dwCategory;
DWORD dwCommandText;
DWORD dwAction;
DWORD dwActionSize;
BYTE abData[];
} VCMDCOMMAND, *PVCMDCOMMAND;

Provides information about a command in a voice menu.

dwSize

Size, in bytes, of the VCMDCOMMAND structure, including the amount allocated for abData. The contents of abData must be doubleword-aligned, so round dwSize up to the nearest whole doubleword.

dwFlags

Flags that indicate information about the command. This member can be a combination of these values:

Value | Meaning
VCMDCMD_DISABLED_PERM | The command was disabled by using the IVCmdMenu::EnableItem member function so that voice commands will not recognize it. The command is not compiled into the voice menu.
VCMDCMD_DISABLED_TEMP | The command was disabled by using the IVCmdMenu::SetItem member function. The command is still compiled into the voice menu, however, so it can be reenabled without recompilation of the menu.
VCMDCMD_VERIFY | The application should prompt the user to verify the command before carrying it out. For example, this value should be set for a "Delete File" command. This value may be combined with either of the other values.
VCMDCMD_CANTRENAME | (New for 3.0) This indicates that the command is automatically generated and that navigator applications (such as Microsoft Voice) shouldn't allow users to rename the command. For example: A set of commands that are generated by extracting all of the menu items in the currently running application would have this flag set since there would be little point in users renaming them.

dwID

Command identifier. This member can be used to identify a command to modify, or it can be used for notifications.

dwCommand

Offset from the beginning of this structure to first character of the voice command string (ANSI or Unicode, depending on which character set was used in the application). For example, the voice command string might be "Open the file" and the character at the offset specified by dwCommand would be 'O'. In languages such as Japanese that have both a phonemic and symbolic character set, the dwCommand member is the offset to a phonemic string.

Within the command string, the following characters have special meaning:

Character | Meaning
 | Indicates the name of a list of words or phrases that can be spoken at this point in the command. For example, the command string "Send mail to name" contains a list called "name." To add phrases to the list, use the IVCmdMenu::ListSet member function.
{ } | Reserved for future use.
[ ] | Reserved for future use.

dwDescription

Offset from the beginning of the structure to first character of a string that describes the action performed by the command.

dwCategory

Offset from the beginning of the structure to the first character of a string that indicates the category to which the command belongs.

Commands in a voice menu should be organized in different categories to help the user browse through lists of commands more easily. This is similar in concept to Windows menus, which organize commands under menu names such as "File," "Edit," "View," and so on. For best results, you should use 20 or fewer categories.

dwCommandText

Offset from the beginning of the structure to the first character of the command text, which is the string that is displayed to the user when he or she requests a list of available voice commands. If this member is NULL, an application uses the text pointed to by dwCommand, which is the voice-command string used in the application's user interface.

Most applications written for European languages will set this member to NULL because the language uses only one character set. Applications written for languages that have both a phonemic and symbolic character set, such as Japanese, will store the phonemic representation of the command in dwCommand and the symbolic representation (which is preferred by the user) in this member.

dwAction

Offset from the beginning of the structure to the first byte of a block of data that is sent to the application when the command is spoken.

Data passed with a command is not interpreted by voice commands; it is up to the application to determine whether the data is valid and to act upon it. Always check the validity of the data, because it is susceptible to being changed — accidentally or intentionally — by other applications, just as other applications can change an .INI file or registry file.

dwActionSize

Number of bytes required to store the block of data indicated by dwAction.

abData

Array of type BYTE that contains the command string, its description, its category, and additional data (if any) to pass to the application along with the command. Because all of the items in abData are doubleword-aligned, the size of abData should be a multiple of 4. All strings should be null-terminated (\0).

Because the items indicated by offsets into abData are doubleword-aligned, the offsets specified by dwCommand, dwDescription, dwCategory, dwAction, and dwActionSize should be multiples of 4.


VCMDNAME

typedef struct { // vcn
TCHAR szApplication[VCMD_APPLEN];
TCHAR szState[VCMD_STATELEN];
} VCMDNAME, *PVCMDNAME;

Contains strings that uniquely identify the application and state to which a voice menu belongs.

szApplication

Name of the application — for example, "Microsoft Word." The application name must be unique among all applications registered to use voice commands on the user's computer.

szState

Unique name of the application state in which the voice command set is valid. An application state usually corresponds to an active window or dialog box — for example, "Main Window" or "Set Font Dialog."


VCSITEINFO

typedef struct { // vcsi
DWORD dwAutoGainEnable;
DWORD dwAwakeState;
DWORD dwThreshold;
DWORD dwDevice;
DWORD dwEnable;
TCHAR szMicrophone[VCMD_MICLEN];
TCHAR szSpeaker[VCMD_SPEAKERLEN];
GUID gModeID;
} VCSITEINFO, *PVCSITEINFO

Provides information about the audio device, speech-recognition mode, and other attributes of a voice-command site.

dwAutoGainEnable

Value from 0 to 100 that indicates the state of the automatic gain for the incoming audio stream to be used by the site.

dwAwakeState

TRUE if the site is awake for purposes of speech-recognition or FALSE if the site is asleep.

dwThreshold

Value from 0 to 100 that indicates the recognition threshold for the speech-recognition engine to be used by the site.

dwDevice

Device identifier of the wave-in audio device to be used by the site. The device identifier can be obtained by calling the waveInGetNumDevs and waveInGetDevCaps multimedia functions.

dwEnable

TRUE if speech recognition is enabled for the site or FALSE if speech recognition is disabled.

szMicrophone

Name of the current microphone for the audio source to be used by the site.

szSpeaker

Name of the current speaker for the site.

gModeID

GUID that uniquely identifies the speech-recognition mode to be used by the site. The GUID for a speech-recognition mode can be obtained by using a speech-recognition enumerator. For more information about speech-recognition enumerators, see section, "Low-Level Speech Recognition API."

An application can pass a pointer to a VCSITEINFO structure with the IVoiceCmd::Register function to set the audio device, speech-recognition mode, and other attributes of a voice-command site, even if the site is already open.


Constants

This section describes the command flags and error values used with voice-command functions.


Command Flags

The following flags are used with the member functions of the IVCmdMenu interface to identify a command in a voice-command menu:

Value | Meaning
VCMD_BY_IDENTIFIER | The dwCmdNum is the command identifier of the command.
VCMD_BY_POSITION | The dwCmdNum parameter is the position in the list of commands.


Voice-Command Error Values

The following errors may be returned by functions of the voice-command objects. If no error occurs, the functions return NOERROR.

Error value | Description
VCMDERR_CANNOTMIMIC | No command was found to match the input string.
VCMDERR_CANTCREATEAUDIODEVICE
 | An audio source object cannot be created for speech recognition.
VCMDERR_CANTCREATEDATASTRUCTURES
 | The internal data structures needed to build a grammar cannot be created.
VCMDERR_CANTCREATESRENUM
 | A speech-recognition enumerator object cannot be created.
VCMDERR_CANTCREATESTORAGE
 | Space cannot be created in the database for the menu.
VCMDERR_CANTINITDATASTRUCTURES
 | The internal data structures needed to build a grammar cannot be initialized.
VCMDERR_CANTSELECTENGINE
 | A speech-recognition engine cannot be selected.
VCMDERR_CANTSETDEVICE | The device identifier cannot be set into the audio source object.
VCMDERR_CANTXTRACTWORDS
 | The unique words needed by the engine grammar object cannot be extracted.
VCMDERR_INVALIDCHAR | There is an invalid character in the voice menu.
VCMDERR_INVALIDLIST | There was an invalid list passed to ListSet or ListGet.
VCMDERR_INVALIDMODE | No site exists, the site does not support this attribute, or the voice command object is already registered with a site.
VCMDERR_INVALIDWINDOW | The window is invalid.
VCMDERR_MENUACTIVE | The menu is currently active and cannot be deleted.
VCMDERR_MENUDOESNOTEXIST
 | The requested menu does not exist in the database.
VCMDERR_MENUEXIST | The requested new or temporary menu already exists in the database.
VCMDERR_MENUOPEN | The menu exists but has not been released, so it cannot be deleted.
VCMDERR_MENUTOOCOMPLEX
 | The menu is too complex to use.
VCMDERR_MENUWRONGLANGUAGE
 | The language is not supported by the speech-recognition engine.
VCMDERR_NOCACHEDATA | There is no internal cache entry for this menu.
VCMDERR_NOENGINE | No speech-recognition engine is active.
VCMDERR_NOFINDINTERFACE | The speech-recognition enumerator interface does not have a Find interface.
VCMDERR_NOGRAMMARINTERFACE
 | No engine grammar object is available.
VCMDERR_NOSITEINFO | Site information to open this site is not available.
VCMDERR_NOTSUPPORTED | The function is not supported by the engine.
VCMDERR_OUTOFDISK | No disk space is left to write information.
VCMDERR_OUTOFMEM | There is no more memory.
VCMDERR_SRFINDFAILED | The speech-recognition object could not find an appropriate mode.
VCMDERR_TOOMANYMENUS | There are too many active menus.
VCMDERR_VALUEOUTOFRANGE
 | The value is out of range.

In addition to the error values described in the preceding list, member functions can return error values generated by lower-level component objects or calls to the Win32 API. For example, an IVoiceCmd function might return an AUDERR_ value generated by the audio source object. For more information, see "Object Levels and Error Values".


OLE Automation


About OLE Automation

The voice-command and voice-text objects are exposed as OLE automation objects, allowing them to be accessed and controlled through Visual Basic, Visual Basic for Applications (VBA), and other automation-aware application macro languages. You can take advantage of the OLE automation capabilities of these objects to add voice-command and voice-text capabilities to your Visual Basic applications.

This appendix describes the properties and methods that Visual Basic programmers can use to manipulate the voice-command objects.

Note that any version of Visual Basic or VBA that supports OLE automation can be used with voice-command and voice-text object. However only Visual Basic version 4.0 supports the custom asynchronous notification mechanism. Using previous versions of Visual Basic requires the use of a polling mechanism to periodically check to see if a voice command was recognized or another event occurred.

All OLE automation methods described in this appendix set a generic error #440, "OLE Automation error" if they fail.

For examples of Visual Basic applications that use the voice-command and voice-text objects, see the sample applications included with the Speech Software Development Kit.

For more information about OLE automation, see the Microsoft Win32 Software Development Kit (SDK).


Voice-Command Object

The properties and methods described in this section apply to the voice-command object.


Awake Property

object.Awake

Awakens or puts speech recognition to sleep for a site. Setting the Awake property to TRUE awakens speech recognition for the site; setting it to FALSE puts speech recognition to sleep.

· Example:

Vcmd.Awake = False

object

Voice-command object.

When speech recognition is asleep, it recognizes commands only from sleep menus, which are designated to be active when speech recognition is asleep and inactive otherwise. For more information, see the description of the IVCmdMenu::Activate and IVCmdAttributes::EnabledSet member functions.


Callback Property

object.Callback

Specifies the name of a class module containing methods that receive notifications from the voice-command automation object. The class module must contain a CommandRecognize method and a CommandOther method.

· Example:

Vcmd.Callback = "TESTVCMD.VCmdSink"

object

Voice-command object.

The CommandRecognize and CommandOther methods have the following syntax:

Function CommandRecognize(szCommand as String, dwID as Long)

Function CommandOther(szCommand as String, szApp as String, szState as String)

For more information, see the description of the IVCmdNotifySink::CommandOther and IVCmdNotifySink::CommandRecognize member functions.

In this example, "TESTVCMD.VCmdSink" is a ProgID where "TESTVCMD" is derived from the Visual Basic project name, and "VCmdSink" is derived from the Name property of the class module. For more information, see section of the Visual Basic 4.0 documentation that describes how to create an OLE server.


CommandSpoken Property

object.CommandSpoken

Identifies the voice command most recently recognized. An application must call the MenuCreate method before checking this property.

· Example:

CmdHeard = Vcmd.CommandSpoken

If CmdHeard > 0 Then

LCommand.ListIndex = CmdHeard - 1

End If

CmdHeard = 0

object

Voice-command object.

The voice-command object sets the CommandSpoken property to the command's unique identifier when the user speaks the command.

If the application has a mechanism to receive notifications, CommandSpoken can be ignored. For more information, see the description of the Callback property.

If there is no way to receive notifications, the application must continually poll this attribute to see if a command was spoken. The application usually polls this attribute twice per second. After the application has found a valid value, it should write a zero into the attribute to clear it for future commands.


MenuCreate Method

object.MenuCreate(applicationName as String, state as String, languageID as Integer, dialect as String, flags as Long)

Creates a new set of voice commands for the given application and state. This method returns a voice-menu object.

· Example:

Dim VMenu As Object

.

.

.

Set Vmenu = Vcmd.MenuCreate("My Application", "MyMenu",
1033&, "", vcmdmc_CREATE_ALWAYS)

object

Voice-command object.

applicationName

Name of the application — for example, "Microsoft Word". The application name must be unique among all applications registered to use voice commands on the user's system.

state

Unique name of the application state in which the voice command set is valid. An application state usually corresponds to an active window or dialog box — for example, "Main Window" or "Set Font Dialog".

languageID

Language identifier as specified for Win32. The first 10 bits indicate the primary language, such as English, French, Spanish, and so forth. The last 6 bits indicate the sublanguage, which is essentially a locale setting. For example, Portuguese has two sublanguages, Brazilian and Standard. For a list of primary and sublanguage identifiers, see the documentation for the MAKELANGID macro in the Win32 SDK.

dialect

Dialect specific to the language — for example, "Texas" or "New York City". The dialect parameter can be "Standard" (or its localized equivalent) for the standard dialect used by the television and radio news media.

flags

Flag that indicates how to create the menu. It can be one of these values:

Value | Description
vcmdmc_CREATE_NEW | Creates an empty menu with the given name. If a menu by that name already exists in the voice-command database, the method fails. The new menu is stored in the database when the voice-menu object is released.
vcmdmc_CREATE_ALWAYS | Creates an empty menu with the given name. If a menu by that name already exists in the voice-command database, it is erased. The new menu is stored in the database when the voice-menu object is released.
vcmdmc_CREATE_TEMP | Creates an empty menu with the given name. If a menu by that name already exists in the voice-command database, the method fails. The new menu is temporary and is discarded when the voice-menu object is released.
vcmdmc_OPEN_ALWAYS | Opens an existing menu with the given name. If the menu does not exist, the method creates a new, empty menu. The new menu is stored in the database when the voice-menu object is released.
vcmdmc_OPEN_EXISTING | Opens an existing menu. If the menu does not exist, the method fails.


Register Method

object.Register(siteName as String)

Registers an application with the voice-command object. An application must call this method before calling any other method or setting any property.

· Example:

Call Vcmd.Register("")

object

Voice-command object.

siteName

Name of the site to connect to, such as "Line 1". To use the default site ("Local PC"), specify an empty string.


Voice-Menu Object

The properties and methods described in this section apply to the voice-menu object.


Active Property

object.Active

Activates or deactivates the voice-command menu. An application must call the MenuCreate method before setting this property.

· Example:

Vmenu.Active = True

object

Voice-menu object.

When the Active property is set to TRUE, the voice-command menu is activated as long as speech recognition is awake for the site . Setting the Active property to FALSE deactivates the current voice-command menu.


Add Method

object.Add(identifier as Long, command as String, category as String, description as String)

Adds a new command to a voice-command menu created by the MenuCreate method.

· Example:

Call Vmenu.Add(CmdNum, "Copy")

object

Voice-menu object.

identifier

Unique number for the command. The CommandSpoken property is set to this number when the command is recognized.

command

Voice-command string — for example, "Open the file".

category

String that indicates the category to which the command belongs. Commands in a voice menu should be organized in different categories to help the user browse through lists of commands more easily. This is similar in concept to Windows menus, which organize commands under menu names such as "File," "Edit," "View," and so on. For best results, you should use 20 or fewer categories.

description

String that describes the action performed by the command.

The Add method calls the IVCmdMenu::Add member function to add the command to the menu.


hWndMenu Property

object.hWndMenu

Sets the window handle for a voice menu. Whenever this window is the foreground window, the voice menu is automatically activated; otherwise, it is deactivated. If this property is set to NULL, the menu is global — that is, it remains active regardless of the foreground window until the application explicitly deactivates it.

object

Voice-menu object.

This property should be set before setting the Active property on the voice-menu object.

This property defaults to NULL, resulting in the voice-menu being a global voice-menu that is active regardless of which window currently has the input focus. For more information, see the description of the IVCmdMenu::Activate member function. The hWndMenu property is equivalent to the hWndListening parameter of IVCmdMenu::Activate.


ListSet Method

object.ListSet(listName as String, numberOfElements as Long, listData as String)

Sets the phrases in a list for a voice command. A list is a named set of words or phrases. By embedding the name of a list in a command string, the user can speak any word or phrase in the list at the given point in a command string. For example, a list named "myfriends" might contain the names Linda, Tom, and Mary. If the list name were embedded in a command string such as, "Send mail to <Names>," the user could say "Send mail to Linda," "Send mail to Tom," or "Send mail to Mary."

· Example:

Dim Names

Dim szNULL as String

szNULL = Chr(0)

.

.

.

Names = "Larry" & szNULL & "Mike" & szNULL & "Gib" & szNULL & "Doug" & szNULL & "George" & szNull

Call Vmenu.ListSet("name", 5, Names)

object

Voice-menu object.

listName

Name of the list to set, such as "name" or "week day". The list name must appear in the command string for at least one command on the menu.

numberOfElements

Number of phrases (null-terminated strings) in the list specified by listData.

listData

List of null-terminated strings. Each string is a possible phrase for a voice command.

The ListSet method calls the IVCmdMenu::ListSet member function to set a list for a voice command.

Using ListSet establishes a list of phrases that can be spoken in a voice command, such as "Send mail to <name>." Typically, the list contains information that changes dynamically at run time, such as the 10 people to whom the user most recently sent electronic mail. For best results, a list should have fewer than 20 entries. Having more than 20 entries in a list can reduce the accuracy of recognition.


Remove Method

object.Remove(identifier as Long)

Removes a command from a voice-command menu created by the MenuCreate method.

· Example:

Call Vmenu.Remove(identifier)

object

Voice-menu object.

identifier

Identifier of the command to remove. This is the same unique identifier used by the Add method to add the command.

The Remove method calls the IVCmdMenu::Remove member function to remove the command from the menu.


Version 3.0

Voice Dictation requires version 3.0 or higher of the speech API. An application knows if version 3.0 or higher is installed because a QueryInterface for the Voice Dictation object will fail under 1.0 and 2.0.


Why Use Dictation?

The "Voice Commands" module allows applications to easily use speech recognition for Command & Control speech recognition. Applications merely have to provide a list of commands that they understand, and they will be notified when a command is spoken. The list of available commands is called a "Voice Menu" because it is very similar to a Windows Menu, which is a list of commands (menu items) that the user can select with a mouse or keyboard.

We are using a similar metaphor for the "Voice Dictation" module that allows users to dictate into applications. The Voice Dictation Module's infrastructure and interfaces are similar to an edit box, just like Voice Commands is similar to a windows menu.

The "Voice Dictation" object allows applications to easily provide dictation. The object uses the metaphor of an invisible edit box. Whenever the user speaks a word, the text is entered directly into the invisible edit box. The application is then notified that the text in the edit box has changed, and how it has changed. Since the invisible edit box cannot be since, the application needs to display any changes to the user. An application will most likely maintain its own rich-text control and be forced to keep the invisible dictation "edit box" and rich-text synchronized.

Of course, if an application merely wanted to transcribe the words spoken by the user, it would be easier to paste the PhraseFinish() results from a dictation engine directly into a text box. However, Voice Dictation provides more functionality than just transcription. It provides automatic text formatting (capitalization, spacing), translation of punctuation words into symbols, built in glossary entries, limited commands, and a GUI so users can correct the dictation engine. It has the potential to support more functionality in the future, especially when continuous dictation becomes widespread.


Hardware and Software Requirements

A dictation application requires certain hardware and software on the user's computer to run. Not all computers have the memory, speed, microphone, or speakers required to support speech, so it is a good idea to design the application so that speech is optional.

These hardware and software requirements should be considered when designing a speech application:

· Processor speed. The speech-recognition and text-to-speech engines currently on the market typically require a Pentium 60 or faster processor.

· Memory. On the average, speech recognition for dictation consumes 4 to 8 megabytes (MB) of random-access memory (RAM) in addition to that required by the running application.

· Sound card. Almost any sound card will work for speech recognition and text-to-speech, including Sound Blaster™, Media Vision™, ESS Technology, cards that are compatible with the Microsoft® Windows Sound System, and the audio hardware built into multimedia computers. A few speech-recognition engines still need a DSP (digital signal processor) card.

· Microphone. The user can choose between two kinds of microphone: either a close-talk or headset microphone that is held close to the mouth or a medium-distance microphone that rests on the computer 30 to 60 centimeters away from the speaker. A headset microphone is needed for noisy environments. Dictation works best with close-talk microphones.

· Operating system. The Microsoft Speech application programming interface (API) requires either Windows 95 or Windows NT version 3.5.

· Speech-recognition and text-to-speech engine. Speech-recognition and text-to-speech software must be installed on the user's system. Many new audio-enabled computers and sound cards are bundled with speech-recognition and text-to-speech engines. As an alternative, many engine vendors offer retail packages for speech recognition or text-to-speech, and some license copies of their engines.

For a list of engine vendors that support the Speech API, see the ENGINE.DOC file included with the Speech Software Development Kit.


Microphones and sound cards

The microphone is the largest problem that speech recognition encounters. Microphones inherently have the following problems:

Not every user has a sound card. Over time more and more PCs will bundle with a sound card.

Not every user has a microphone. Over time more and more PCs will bundle a microphone.

Sound cards (being in the back) don't make it very easy for users to plug in the microphone.

Most microphones that come with computers are cheap. Because of that, the microphones don't do as well as more expensive microphones that retail for $50 to $100. Furthermore, many of the cheap microphones that are designed to be worn are uncomfortable. A user will not use a microphone if it is uncomfortable.

Users don't know how to use a microphone. If the microphone is a worn on their head they often wear it incorrectly, or if it sits on their desktop they will lean towards it to speak even though the microphone is designed for the user to speak from their normal sitting position; After all, everyone has seen rock stars on TV, and they are practically eating the microphone.

Most applications can do little about the microphone. Luckily, when the user installs the speech recognition engine, the engine should come with software that makes sure the user's microphone is correctly plugged in and working.

Most users of dictation will wear close-talk microphones for maximum accuracy. This alleviates a number of problems that Command and Control recognition encounters because the user is wearing a headset microphone.


Speech Recognizers make mistakes

Speech recognizers make mistakes, and will always make mistakes. The only thing that is changing is that every two years recognizers make half as many mistakes as they did before. But, no matter how great a recognizer is it will always make mistakes.

To make matters worse, dictation engines make misrecognitions that are correctly spelled, often grammatically correct, but means nothing. Unfortunately, sometimes the misrecognition mistakes mean something completely different than the user intended. While people have become accustomed to typos they are not accustomed attributing strange wording to speech errors.

An application can minimize some of the misrecognitions by:

Make it as easy as possible for users to correct mistakes.

Provide easy access to the "Correction Window" so the user can correct mistakes that the recognizer made.

Allow the user to train the speech recognition system to his/her voice.


Is it a Command?

When speech recognition is listening for dictation user's will over want to interject commands such as "cross-out" to delete the previous word or "capitalize-that". Applications should make sure that:

If a command is just one word, make sure that it does not replace a word that people like to dictate.

If a command is multiple words that it isn't a phrase that people like to dictate.


Finite Number of Words

Speech recognizers limit the number of words that are being listened for to someplace between 20,000 and 100,000 words. Because of this, one out of every fifty words a user speaks isn't recognized because it isn't in the 20,000 - 100,000 words supported by the engine.

Applications can reduce the error rate of an engine if the application tells the engine about what words the engine should expect.


Other Problems

Some other problems crop up:

Having a user spell out words is a bad idea for most recognizers because they are too inaccurate.

An engine also cannot tell who is speaking, although some engines may be able to detect a change in the speaker. Voice-recognition algorithms exist that can be used to identify a speaker, but currently they cannot also determine what the speaker is saying.

In addition, an engine cannot detect multiple speakers talking over each other in the same digital-audio stream. This means that a dictation system used to transcribe a meeting will not perform accurately during times when two or more people are talking at once.

· Unlike a human being, an engine cannot hear a new word and guess its spelling.

· Localization of a speech-recognition engine is time-consuming and expensive, requiring extensive amounts of speech data and the skills of a trained linguist. If a language has strong dialects that each represent sizable markets, it is also necessary to localize the engine for each dialect. Consequently, most engines support only five or ten major languages—for example, European languages and Japanese, or possibly Korean.

· Speakers with accents or those speaking in nonstandard dialects can expect more misrecognitions until they train the engine to recognize their speech. Even then, the engine accuracy will not be as high as it would be for someone with the expected accent or dialect. An engine can be designed to recognize different accents or dialects, but this requires almost as much effort as porting the engine to a new language.


Do Not Replace the Keyboard and Mouse

Most dictation systems provide discrete dictation, allowing users to speak up to 50 words per minute. While this is faster than hunt-and-peck typists, touch typists can type at least 70 words per minute. Discrete dictation will not be used by touch typists. Continuous dictation allows up to 120 words per minute.


About Voice Dictation

Some elements of Voice Dictation that you should know of are:


Topics

A unique "topic" should exist for each classification of document that an application can have. Example topics might be "E-mail", "Formal Writing", or "C-Code", three text-entry mediums where the style of language used is vastly different. Voice Dictation will store information on topics away so that any information the recognizer learns will be stored, along with any glossary entries that the user generates specifically for the topic.

Adding a new topic might take a lot of hard disk or memory, so unless an application has a very good reason for creating a new topic, it should use an existing topic. Topics cannot be merged. Much of the information stored within a topic data structures is engine dependent so switching dictation engine vendors may cause a loss of some data.


Dictation Session

Whenever an application wishes to use dictation, it must instantiate a "Dictation Session." Each session must be based upon a "Topic" and have a window handle associated with it. The dictation session will only be actively listening when that window has the keyboard focus. It is possible for an application to have multiple sessions, or multiple applications each to have their own sessions. As a user switches between the applications he/she will be dictating into a different session.


Correction Window

The application can also have the dictation object provide some GUI, although the application doesn't necessarily need the GUI.

The "Correction Window" displays some user-interface that allows the user to correct the last word spoken and/or the currently selected word. It will look something like this: (The Correction Window GUI may look different for continuous dictation systems.)


Glossary Entries

A glossary entry is one or more words that, if dictated, will be replaced by a text string. It is a textual translation of a symbol. Thus, a user could have a glossary entry for "copyright" that converts it to "©", or "a t and t" to "AT&T". Since users can add glossary entries, they are designed so they're easy for users to understand and modify.

Applications can provide their own glossary entries for a specific site, and even more complex interactions by providing their own commands. Voice Dictation will have some built in glossary entries.


Commands

Voice Dictation supports some command & control ability while the user dictates. Applications should use commands for non-textual actions, or textual-modifications that are too complex for glossary entries. Applications will be able to specify commands to have Voice Dictation listen for. Users, through an application's user-interface, can modify these commands.

Voice Dictation will also provide built-in commands. We will have a limited number in the first version but will expand the built-in commands as resources and technology (continuous dictation) allow. Applications cannot add built-in commands, although they can rename or disable them if they don't use them.


Inverse Text Normalization

"Inverse text normalization" is a feature in the voice dictation module that automatically formats text spacing, capitalization, and converts some phrases to a more symbolic form. Example: Inverse text normalization is what converts the spoken phrase, "the time is eight oh five p m period" to "The time is 8:05 PM." Applications can control the inverse text normalization by passing a Voice Dictation context free grammar into the Voice Dictation module. A description of the Voice Dictation Context Free Grammar is given later.


Using Voice Dictation

An application wishing to write to the Voice Dictation API will need to incorporate the following code.

When the application starts up, it will need to call:

CoCreateInstance (..., CLSID_VDct, ..., &gpIVoiceDictation);

gpIVoiceDictation->QueryInterface (..., IID_IVDCTTEXT, &gpIVDctText);

gpIVoiceDictation->QueryInterface (..., IID_IVDCTGUI, &gpIVDctGUI);

If the application has not created its own topic, and does not wish to use a common topic, it needs to generate on. It calls:

gpIVoiceDictation->TopicAddString ("My App's Topic", NULL,

"E-mail", NULL, NULL, NULL);

The application can provide more information, such as sample words or documents for the topic. More information might increase the initial accuracy of the topic.

Once the application knows a topic, it calls Register() so that the Voice Dictation knows what topic to use:

gpIVoiceDictation->Register ("My Application", "My App's Topic",

NULL, NULL, &IVDctNotifySink, IID_IVDCTNOTIFYSINK, 0);

If an application were loading in a saved dictation session it would pass in an Istorage where the session information was saved.

The IVDctNotifySink is the notification sink code supplied by the application. It's member functions will be called to notify the application when the user speaks and text is added/removed from the virtual buffer. We will describe some of the interfaces as they're needed.

When the application wishes to listen, it calls:

gpIVoiceDictation->Activate (hWnd);

hWnd is the window handle of the window. The dictation engine might not really be listening until a global flag of EnabledSet() is turned on. The application usually does not have to turn this on since the system will have some UI that allows the user to turn it on. From the user's perspective he/she will then be turning on speech recognition for the entire PC, not just one application at a time.

Now that the application is registered, it creates its own edit box, rich edit box, or custom control that accepts text entry. This will be intimately tied into the virtual text edit box supplied by Voice Dictation. Whenever the user speaks some text that will be entered into the virtual edit box, the application will be called with, IVDctNotifySink.TextChanged() or TextSelChanged(). This means that something has changed and the application should synchronize it's visible edit box with Voice Dictation's invisible one so that the user will see the changes. To synchronize, the application calls:

gpIVDctText->Lock();

gpIVDctText->GetChanges (&dwOldStart, &dwOldEnd, &dwNewStart, &dwNewEnd);

gpIVDctText->TextGet (dwNewStart, dwNewEnd - dwNewStart, pszTemp);

Replace the text from dwOldStart to dwOldEnd in the visible edit box with pszTemp;

gpIVDctText->TextSelGet (&dwSelStart, &dwSelSize);

Set the text selection from dwSelStart to (dwSelStart + dwSelSize) in the visible

edit box;

gpIVDctText->UnLock();

Get the screen coordinates of the visible selection and put into rectSel;

gpIVDctGUI->SetSelRect (rectSel);

This code locks down the invisible text buffer so that it is not changed by inverse text normalization or new recognitions. It then finds out what text has changed since the last time the application synchronized and displays those changes to the user. The correction user-interface is then moved so that it's adjacent to the selected text.

Likewise, the application has to notify the Voice Dictation object when the user changes the selection or text.

If the user changes the selection with a mouse or keyboard then the application calls:

gpIVDctText->Lock();

gpIVDctText->GetChanges (&dwOldStart, &dwOldEnd, &dwNewStart, &dwNewEnd);

if the text has changed then the application should update the UI;

gpIVDctText->TextSelSet (dwSelStart, dwSelSize);

gpIVDctText->UnLock();

Get the screen coordinates of the visible selection and put into rectSel;

gpIVDctGUI->SetSelRect (rectSel);

If the user types in some new text, pastes in text, or does anything to change the text, the application should call:

gpIVDctText->Lock();

gpIVDctText->GetChanges (&dwOldStart, &dwOldEnd, &dwNewStart, &dwNewEnd);

if the text has changed then the application should update the UI;

gpIVDctText->TextSet (dwStart, dwNumCharsReplace, szNewText, reason);

gpIVDctText->TextSelSet (dwSelStart, dwSelSize);

gpIVDctText->UnLock();

Get the screen coordinates of the visible selection and put into rectSel;

gpIVDctGUI->SetSelRect (rectSel);

When the application closes down it just releases all of the interfaces.

That's all an application has to do for minimal functionality. Keeping the text buffers synchronized is tricky, but once that problem is solved, dictation is easy. The application doesn't have to worry about any more UI.

Some applications might want to take advantage of even more features such as:

Application Specific Commands - An application might want to provide its own commands that will be active during dictation. A good one is "line-up" or "line-down"; These cannot be built-in commands since the virtual edit box doesn't know where a line begins/ends.

Topic-Specific Glossary Entries - An application might want to add glossary entries to the topic so that the user won't have to add them him/herself.

Save all of the audio and speech recognition correction information into a file so the user can re-load it and do more correction.

If an application wants to maintain tags or marks within the text it can put in "bookmarks" between characters, and always be able to find their current position. This could be used to indicate changes in font.

Of course, there are many more options. Read the rest of the documentation for more information.


Objects

Voice-dictation objects provide dictation for applications. This section describes these objects and the interfaces that they support.

In addition to the interfaces listed in this section, each object supports IUnknown, which is a common interface supported by all OLE component objects. For information about IUnknown, see the section, "Common Elements of the Speech Application Programming Interface."


Voice Dictation Object


Purpose

A voice-dictation object manages the use of dictation by all voice-aware applications on a user's computer. The voice dictation object represents a voice dictation site consisting of an audio device and a speech-recognition mode provided by an engine. Internally, a voice-command site consists of two low-level speech-recognition objects: a speech-recognition engine object and an audio source.

The voice-dictation object can use any speech-recognition engine that supports dictation or limited-domain grammars.


Created by

You create a voice-dictation object by calling the CoCreateInstance function with the CLSID_VDct class identifier and the IID_IVoiceDictation interface identifier. CoCreateInstance returns a pointer to the IVoiceDictation interface for the voice-dictation object.


Interacts with

The voice-dictation object calls an application's IVDctNotifySink interface to notify the application of speech-related events. An application must create a COM object that supports the IVDctNotifySink interface and must pass the address of the interface to the voice-dictation object when calling the IVoiceDictation::Register member function. The application developer determines how the functions in the interface respond to notifications.


Interfaces

The voice-command object supports the following interfaces.

Interface | Description
ILexPronounce | Modify the pronunciation of words in the engine.
ISRSpeaker | Load, save, and restore speaker information.
IVDctAttributes | Controls attributes of a voice-dictation site, such as the audio device, the microphone, the speech-recognition mode, and whether speech-recognition is awake and enabled for the site. To get the address of the IVDctAttributes interface, use the IVoiceDictation::QueryInterface member function with the IID_IVDctAttributes interface identifier.
IVDctCommands | Controls which commands are to be recognized during dictation.
IVDctDialogs | Displays Windows dialog boxes that allow a user to configure the speech-recognition engine. These dialog boxes are defined by the engine.
IVDctGlossary | Edit, modify, or rename glossary entries used in dictation.
IVDctGUI | Controls the position of the correction window.
IVDctInvTextNorm | Allows an application to set and control the inverse text normalization done by the Voice Dictation module.
IVDctText | Allows the application to get, set, move, or otherwise modify the text in the virtual text field.
IVoiceDictation | Registers an application to use voice dictation on a site, allows an application to create topics.


Results Objects


Purpose

The results objects provide the application with more detailed information about the recognition.


Created by

You create a results object by using the IVDctText::ResultsGet member function.

All results objects should be released before the final interface to the Voice Dictation object is released.


Interfaces

See the section, "Low-Level Speech Recognition API" for more details.


Voice Dictation Quick-Look

This section lists and briefly describes the voice-dictation interfaces and their member functions.


ILexPronounce

See the section, "Common Elements."


ISRSpeaker

See the section, "Low-Level Speech Recognition API."


IVDctAttributes

This interface is supported by the voice-dictation object.

Member function | Description
AutoGainEnableGet | Gets the state of the automatic gain for the incoming audio stream.
AutoGainEnableSet | Sets the state of the automatic gain for the incoming audio stream.
EchoGet | Gets the state of echo canceling on the incoming audio stream.
EchoSet | Sets the state of echo canceling on the incoming audio stream.
EnergyFloorGet | Gets the value of the current noise floor expected by a speech-recognition engine from the audio stream.
EnergyFloorSet | Sets the value of the current noise floor expected by a speech-recognition engine from the audio stream.
IsAnyoneDictating | For a given window, this returns whether or not an application has turned on dictation for the window.
MemoryGet | Returns the amount of memory that the voice dictation session can use.
MemorySet | Sets the amount of memory that the voice dictation sessions can use.
MicrophoneGet | Gets the name of the current microphone for this audio source.
MicrophoneSet | Sets the name of the current microphone for this audio source.
ModeSet | Sets the dictation mode.
ModeGet | Gets the dictation mode.
RealTimeGet | Gets the real-time setting for the engine.
RealTimeSet | Sets the real-time setting for the engine.
SpeakerGet | Gets the name of the current speaker for an audio source.
SpeakerSet | Sets the name of the current speaker for an audio source.
ThresholdGet | Gets the threshold level of the speech-recognition engine.
ThresholdSet | Sets the threshold level of the speech-recognition engine.
TimeOutGet | Gets the current time-out values for a speech-recognition engine.
TimeOutSet | Sets the time-out values for a speech-recognition engine.


IVDctCommands

This interface is supported by the voice-dictation object.

Member function | Description
Add | Adds one or more commands to voice dictation.
Get | Gets information about one or more commands in voice dictation.
Num | Indicates the total number of commands in voice dictation.
Remove | Removes the specified commands from voice dictation.
Set | Sets information for one or more commands in voice dictation.


IVDctDialogs

This interface is supported by the speech-recognition engine.

Member function | Description
AboutDlg | Displays an About dialog box that identifies the speech-recognition engine and contains the copyright notice.
GeneralDlg | Allows general control of the engine and should invoke some of the other dialog boxes.
LexiconDlg | Allows the speaker to display and edit his or her pronunciation lexicon.
TrainMicDlg | Provides the engine with training for the microphone.
TrainGeneralDlg | Provides phonetic training, which trains the engine for a preselected set of words.
TrainTopicDlg | Provides training of words specific to the topic.


IVDctGlossary

This interface is supported by the voice-dictation object.

Member function | Description
Add | Adds one or more glossary entries to voice dictation.
Get | Gets information about one or more glossary entries in voice dictation.
Num | Indicates the total number of glossary entries in voice dictation.
Remove | Removes the specified glossary entries from voice dictation.
Set | Sets information for one or more glossary entries in voice dictation.


IVDctGUI

This interface is supported by the voice-dictation object.

Member function | Description
FlagsGet | Get control flags for the correction window.
FlagsSet | Set control flags for the correction window.
ReplaceUI | Replaces the correction window user interface with the application's own.
SetSelRect | Sets the selection and insertion rectangle.


IVDctInvTextNorm

This interface is supported by the voice-dictation object.

Member function | Description
OptionsEnum | Enumerates the options for inverse text normalization.
OptionsGet | Indicates if an inverse text normalization switch is on or off.
OptionsSet | Turns an inverse text normalization switch on or off.


IVDctNotifySink

This interface is supported by the application.

Member function | Description
AttribChanged | Indicates that an attribute has been modified and which one.
Command | Notifies the application when a command for the application is spoken.
Dictating | Sent when an application starts/stops dictation.
Interference | Notify what kind of interference with the audio signal is happening.
PhraseFinish | Indicates that a speech recognition phrase is finished.
PhraseHypothesis | Hypothesis for what might be spoken.
PhraseStart | Indicates that a speech recognition phrase is being processed.
TextChanged | Called when some text in the buffer has changed.
TextSelChanged | Called when the current selection has changed.
Training | Indicates that the speaker needs to train the system.
UtteranceBegin | Speech has started.
UtteranceEnd | Speech has finished.


IVDctText

This interface is supported by the voice-dictation object.

Member function | Description
AutoLockGet | Gets the state of automatic locking.
AutoLockSet | Turns automatic locking on/off.
BookmarkAdd | Places a bookmark in the text.
BookmarkEnum | Enumerates the bookmarks within a section of text.
BookmarkMove | Moves a bookmark within the text.
BookmarkQuery | Finds a bookmark's position.
BookmarkRemove | Removes a bookmark from the text.
FX | Performs built in effects on the text.
GetChanges | Returns what changes occurred since the last call to UnLock().
Hint | Sends a hint down to the speech recognizer.
Lock | Locks the text object so only the application can modify the text.
ResultsGet | Returns the results object associated with a word.
TextGet | Gets text from the internal buffer.
TextMove | Moves text within the internal buffer.
TextRemove | Removes text from the internal buffer.
TextSelGet | Gets the current text selection.
TextSelSet | Sets the current text selection.
TextSet | Writes text to the internal buffer.
UnLock | Unlocks the text object after it was locked.
Words | Tells the recognizer to expect specific words.


IVoiceDictation

This interface is supported by the voice-dictation object.

Member function | Description
Activate | Starts dictation listening.
CFGSet | Sets the context free grammar to be used for inverse text normalization.
Deactivate | Stops dictation listening.
Register | Registers a notification sink with the dictation object.
SessionSerialize | Writes a dictation session to disk.
SiteInfoGet | Gets the dictation information pertaining to a site.
SiteInfoSet | Sets the dictation information pertaining to a site.
TopicAddGrammar | Adds a new topic. Takes a grammar object to define the topic.
TopicAddString | Adds a new topic. Parameters are a several strings such as the topic name and language.
TopicDeserialize | Reads all of the topic data to a file.
TopicEnum | Enumerates the topics stored on the PC's hard disk.
TopicRemove | Removes a topic from disk.
TopicSerialize | Writes all of the topic data to a file.


Interfaces and Member Functions

Voice-dictation interfaces contain functions that allow an application to implement dictation. There are the following interfaces.

Interface | Description
ILexPronounce | Modify the pronunciation of words in the engine.
ISRSpeaker | Load, save, and restore speaker information.
IVDctAttributes | Controls attributes of a voice-dictation site, such as the audio device, the microphone, the speech-recognition mode, and whether speech-recognition is awake and enabled for the site. To get the address of the IVDctAttributes interface, use the IVoiceDictation::QueryInterface member function with the IID_IVDctAttributes interface identifier.
IVDctCommands | Controls which commands are to be recognized during dictation.
IVDctDialogs | Displays Windows dialog boxes that allow a user to configure the speech-recognition engine. These dialog boxes are defined by the engine.
IVDctGlossary | Edit, modify, or rename glossary entries used in dictation.
IVDctGUI | Controls the position of the correction window.
IVDctText | Allows the application to get, set, move, or otherwise modify the text in the virtual text field.
IVoiceDictation | Registers an application to use voice dictation on a site, allows an application to create topics.

This section describes, in alphabetical order, the interfaces and functions that support voice dictation.

In addition to the interfaces described in this section, each object supports IUnknown, which is a common interface supported by all OLE component objects. For information about IUnknown, see section, "Common Elements of the Speech Application Programming Interface."


ILexPronounce

See the section, "Common Elements" for a description of the interface.


ISRSpeaker

See the section, "Low-Level Speech Recognition API" for a description of the interface.


IVDctAttributes

The IVDctAttributes interface allows an application to retrieve and set the attributes of a voice-dictation site. The application must call the IVoiceDictation::Register member function before it can call any of the functions in IVDctAttributes.

A voice-dictation application may not need to use IVDctAttributes functions if a voice-navigation application is installed on the user's computer. The voice-navigation application typically monitors and controls the attributes of a voice-dictation site.

Changing an attribute of a site affects all applications using the site unless specified. Settings are saved between uses of the site, even if the user shuts off his or her computer in the meantime.

The IVDctAttributes interface is supported by the voice commands object. It has these member functions:


IVDctAttributes::AutoGainEnableGet

HRESULT AutoGainEnableGet(DWORD *pdwAutoGain);

Retrieves the state of the automatic gain for the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwAutoGain

[out] Address of a variable that receives the automatic gain.

A value of 0 indicates that automatic gain is disabled. A value of 100 indicates that the speech-recognition engine always adjusts the gain, so that if the next utterance is spoken at the same level, the gain is set perfectly. A value between 0 and 100 moderates the automatic adjustments on a linear scale. For example, a pdwAudioGain value of 50 adjusts the gain to half the extent that the engine otherwise would.

When automatic gain is enabled, the speech-recognition engine may adjust the level at the end of an utterance and increase or decrease the gain. The engine adjusts the gain by using the IAudio::LevelGet and IAudio::LevelSet member functions to communicate with the audio-source object that represents the incoming audio stream.

If the audio source does not support IAudio::LevelSet, the application cannot set the automatic gain.


IVDctAttributes::AutoGainEnableSet

HRESULT AutoGainEnableSet(DWORD dwAutoGain);

Sets the state of the automatic gain for the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwAutoGain

[in] Automatic gain value. This parameter can be SRATTR_MINAUTOGAIN or SRATTR_MAXAUTOGAIN for minimum or maximum allowable values, respectively.

A value of 0 disables automatic gain. A value of 100 causes the speech-recognition engine to always adjust the gain. Values between 0 and 100 moderate the gain on a linear scale. If the value is out of range for the engine, an error is returned and the gain is not changed.


IVDctAttributes::EchoGet

HRESULT EchoGet(BOOL *pfEchoOn);

Retrieves the state of echo canceling on the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pfEchoOn

[out] Address of a variable that receives the state of echo canceling. The variable receives TRUE if the engine expects echo canceling on the incoming audio stream, or FALSE if the engine expects a clean signal that hasn't undergone echo canceling. When pfEchoOn is TRUE, the engine typically ignores low-level audio signals.

The state of echo canceling indicates whether the signal should be treated as though it has residual signal on it. In general, echo canceling causes the speech-recognition engine to ignore low-level audio signals so that it doesn't attempt to recognize the residual signal of the echo cancellation.


IVDctAttributes::EchoSet

HRESULT EchoSet(BOOL fEchoOn);

Sets the state of echo canceling on the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

fEchoOn

[in] State of echo canceling. TRUE turns on echo canceling, FALSE turns it off.


IVDctAttributes::EnergyFloorGet

HRESULT EnergyFloorGet(WORD *pwEnergy);

Retrieves the value of the current noise floor expected by a speech-recognition engine from the audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pwEnergy

[out] Address of a variable that receives the expended energy of the noise floor, in negative decibels. For example, a value of 30 indicates an energy floor of –30 dB.

The noise floor is the noise value in the signal-to-noise ratio (SNR) for a particular environment. In general, the higher the noise value in the ratio, the more sensitive the speech-recognition engine is to background noise. For example, a quiet office would have a high SNR (low floor), whereas a noisy factory would have a low SNR (high floor).

An application might send this value to a speech-recognition engine if the application has information about the SNR to expect from the audio stream. The engine can use the value internally to adjust its calculated noise floor in expectation of the audio stream's SNR.


IVDctAttributes::EnergyFloorSet

HRESULT EnergyFloorSet(WORD pwEnergy);

Sets the value of the current noise floor expected by a speech-recognition engine from the audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pwEnergy

[in] Expended energy of the noise floor, in negative decibels. For example, a value of 30 indicates an energy floor of –30 dB. An application can specify SRATTR_MINENERGYFLOOR or SRATTR_MAXENERGYFLOOR for minimum or maximum allowable values.


IVDctAttributes::IsAnyoneDictating

HRESULT IsAnyoneDictating(HWND hWnd, TCHAR *pszName, DWORD dwNameSize, DWORD *pdwNameNeeded);

An application passes in a window handle, and this returns whether or not the application (or anyone) has activated dictation for the window handle, and which application activated dictation.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
S_FALSE

hWnd

[in] Window which dictation is active for. If this is NULL then it checks to see if a global dictation session is activated.

pszName

[out] Pointer to a string to be filled with the name. This will not be filled in if *pdwNameNeeded > dwNameSize.

dwNameSize

[in] Number of bytes available in the data buffer pointed to by pszName.

pdwNameNeeded

[out] Filled in with the number of bytes needed for the entire name.


IVDctAttributes::MemoryGet

HRESULT MemoryGet(VDCTMEMORY *pDctMemory);

MemoryGet fills in a VDCTMEMORY structure with the memory constraints used for the dictation session.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pDctMemory

[out] Pointer to a VDCTMEMORY structure that is filled in.


IVDctAttributes::MemorySet

HRESULT MemorySet(VDCTMEMORY *pDctMemory);

MemoryGet tells Voice Dictation how much memory to use for the results objects (used for correction) for the current session. If the memory constraints are tightened MemorySet does not free up any allocated resources. Changing this will affect the settings for this session only. The changes are not permanent across invocations.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pDctMemory

[in] Pointer to a VDCTMEMORY structure specifying memory constraints.


IVDctAttributes::MicrophoneGet

HRESULT MicrophoneGet(TCHAR *pszMicrophone, DWORD dwMicrophoneSize,

DWORD *pdwNeeded);

Retrieves the name of the current microphone for this audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszMicrophone

[in/out] Address of a buffer that receives the name of the current microphone for an audio source. If the buffer receives an empty string, the current microphone is unknown.

dwMicrophoneSize

[in] Size, in bytes, of the buffer specified by pszMicrophone. If the specified size is too small, the function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the microphone string.

An application can use the microphone name to save or retrieve information about a particular microphone, such as the name of the microphone used to train the engine or the conditions in which the microphone is recording. For example, an application could use this information to preserve the original training when the user changes microphones.


IVDctAttributes::MicrophoneSet

HRESULT MicrophoneSet(TCHAR *pszMicrophone);

Sets the name of the current microphone for an audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszMicrophone

[in] Address of a buffer that contains the name of the microphone to set. If the current microphone is unknown, pszMicrophone can be an empty string.


IVDctAttributes::ModeGet

HRESULT ModeGet(DWORD *pdwMode);

This returns the current mode of operation on the current site. The mode affects both Voice Dictation and Voice Commands.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pdwMode

[out] Mode value. These can be VSRMODE_XXX values.


IVDctAttributes::ModeSet

HRESULT ModeSet(DWORD dwMode);

This sets the current mode of operation on the current site. The mode affects both Voice Dictation and Voice Commands.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwMode

[in] Mode value. These can be VSRMODE_XXX values.


IVDctAttributes::RealTimeGet

HRESULT RealTimeGet(DWORD *pdwRealTime);

Retrieves the real-time setting for the engine. The real-time setting is the percentage of processor time that the engine expects to use during constant speech.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwRealTime

[out] Address of a variable that receives the real-time setting. This value can be more than 100 for non-real-time applications (for example, transcribing prerecorded speech).

For most engines, the amount of processor time required diminishes markedly during periods of silence. For example, if pdwRealTime is 100, the engine takes one full minute of processor time to process one minute of speech. If pdwRealTime is 50, the engine takes 30 seconds of processor time to process the same minute of speech. This value is difficult to compute precisely, so it should be regarded as an estimate.


IVDctAttributes::RealTimeSet

HRESULT RealTimeSet(DWORD dwRealTime);

Sets the real-time setting for the engine. The real-time setting is the percentage of processor time that the engine expects to use during constant speech.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwRealTime

[in] New real-time setting. An application can specify SRATTR_MINREALTIME or SRATTR_MAXREALTIME for minimum or maximum allowable values. If the value is out of range for the engine, the function returns an error and the real-time setting is not changed.

The engine tries to meet the real-time expectation, but a real-time setting of 1 is nearly impossible on current machines.


IVDctAttributes::SpeakerGet

HRESULT SpeakerGet(TCHAR *pszSpeaker, DWORD dwSpeakerSize,

DWORD *pdwNeeded);

Retrieves the name of the current speaker for an audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszSpeaker

[in/out] Address of a buffer that receives the name of the current speaker. If the buffer receives an empty string, the current speaker is unknown.

dwSpeakerSize

[in] Size, in bytes, of the buffer specified by pszSpeaker. If the specified size is too small, the function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the speaker string.


IVDctAttributes::SpeakerSet

HRESULT SpeakerSet(TCHAR *pszSpeaker);

Sets the name of the current speaker for an audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszSpeaker

[in] Address of a buffer that contains the speaker's name to set. If pszSpeaker is a new name, the engine creates a new speaker profile. If the speaker is unknown, pszSpeaker can be an empty string.


IVDctAttributes::ThresholdGet

HRESULT ThresholdGet(DWORD *pdwThreshold);

Retrieves the threshold level of the speech-recognition engine. The threshold level indicates how confident the speech-recognition engine must be about the validity of a given recognition in order to accept it. If the engine is not confident enough, it notifies the application that the phrase was not recognized.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwThreshold

[out] Address of a variable that receives the threshold.

The threshold level is a value from 0 to 100 that indicates the threshold below which an utterance is rejected as unrecognized. A value of 0 indicates that the engine matches any utterance to the closest phrase match. A value of 100 indicates that the engine must be absolutely certain that an utterance is the recognized phrase. The optimum threshold for the average user should be 50.


IVDctAttributes::ThresholdSet

HRESULT ThresholdSet(DWORD dwThreshold);

Sets the threshold level for a speech-recognition engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwThreshold

[in] Threshold level. If the value is out of range for the engine, an error is returned and the attribute is not changed. An application can specify SRATTR_MINTHRESHOLD or SRATTR_MAXTHRESHOLD for minimum or maximum allowable values.

The threshold level is a value from 0 to 100 that indicates the threshold below which an utterance is rejected as unrecognized. A value of 0 indicates that the engine matches any utterance to the closest phrase match. A value of 100 indicates that the engine must have determined absolutely that an utterance is the recognized phrase. The optimum threshold for the average user should be 50.


IVDctAttributes::TimeOutGet

HRESULT TimeOutGet(DWORD *pdwIncomplete, DWORD *pdwComplete);

Retrieves the current time-out values for a speech-recognition engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwIncomplete

[out] Address of a variable that receives the number of milliseconds that the speech-recognition engine waits before discarding an incomplete phrase because the user has stopped speaking. For example, if pdwIncomplete is 2000, a user speaking "Send mail to " could pause for 2 seconds before the engine would assume that the user has stopped speaking the phrase.

pdwComplete

[out] Address of a variable that receives the number of milliseconds that the speech-recognition engine waits before regarding a phrase as complete after the user has stopped speaking. For example, if pdwComplete is 500, a user speaking "Send mail to Fred" would see results 0.5 seconds after finishing the phrase.


IVDctAttributes::TimeOutSet

HRESULT TimeOutSet(DWORD dwIncomplete, DWORD dwComplete);

Sets the time-out values for a speech-recognition engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwIncomplete

[in] Number of milliseconds that the speech-recognition engine should wait before discarding an incomplete phrase because the user has stopped speaking. An application can specify SRATTR_MINTOINCOMPLETE or SRATTR_MAXTOINCOMPLETE for minimum or maximum allowable values.

dwComplete

[in] Number of milliseconds that the speech-recognition engine should wait before regarding a phrase as complete after the user has stopped speaking. An application can specify SRATTR_MINTOCOMPLETE or SRATTR_MAXTOCOMPLETE for minimum or maximum allowable values.

If a time-out value is set to a value that is out of range for the engine, an error is returned and the attribute is not changed.


IVDctCommands

IVDctCommands allows applications to add, remove, and rename commands that are available during dictation.

Member function | Description
Add | Adds one or more commands to voice dictation.
Get | Gets information about one or more commands in voice dictation.
Num | Indicates the total number of commands in voice dictation.
Remove | Removes the specified commands from voice dictation.
Set | Sets information for one or more commands in voice dictation.


IVDctCommands::Add

HRESULT Add(BOOL fGlobal, DWORD dwCmdNum, SDATA dData, DWORD *pdwCmdStart);

Adds one or more commands to for dictation to support during the current dictation session. The added commands are appended to any existing commands.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the commands are added to the global command set that is used in every application; Global commands are saved across invocations. If this is FALSE it only provides commands for the application. In general, applications should not modify the global command set.

dwCmdNum

[in] Number of commands to add.

dData

[in] SDATA structure containing a list of VCMDCOMMAND structures that describe the voice commands to be added. Although they vary in size depending on the command data, the structures are contiguous within the list.

pdwCmdStart

[out] Address of a variable that receives the number of the first command added to the menu.

Commands are numbered sequentially from 1 to n. New commands are added to the end of the menu, so the first command added is numbered n+1. Unlike Voice Commands, Voice Dictation commands do not support lists.


IVDctCommands::Get

HRESULT Get(BOOL fGlobal, DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag,
PSDATA pdData, DWORD *pdwCmdNum);

Retrieves information about one or more commands.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the commands are read from the global command set that is used in every application. If this is FALSE it accesses the commands for the application. In general, applications should not modify the global command set.

dwCmdStart

[in] Number of the first command to retrieve. Commands are numbered sequentially from 1 to n. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Either the number of commands to retrieve or the identifier of the commands, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the total number of commands in the menu, the function returns as many commands as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

pdData

[out] Address of an SDATA structure that receives the address and size of a buffer. The buffer contains a list of VCMDCOMMAND structures that describe the commands retrieved. Although they vary in size depending on the command data, the structures are contiguous within the list.

pdwCmdNum

[out] Address of a variable that receives the number of commands actually copied to the buffer.

The calling application allocates the SDATA structure and passes its address to Get. Get allocates memory (using the OLE task allocator) for the returned data and sets the pData member of SDATA to point to the memory. If the allocation fails, pData is set to NULL and the dwSize member is set to zero. The calling application must free the memory pointed to by pData as well as the SDATA structure itself.

The calling application must free the memory allocated by the member function by using the CoTaskMemFree function.


IVDctCommands::Num

HRESULT Num(BOOL fGlobal, DWORD *pdwNumCmd);

Retrieves the total number of commands in the global or application specific command set.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the commands are read from the global command set that is used in every application. If this is FALSE it accesses the commands for the application. In general, applications should not modify the global command set.

pdwNumCmd

[out] Address of a variable that receives the number of commands.


IVDctCommands::Remove

HRESULT Remove(BOOL fGlobal, DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag);

Removes the specified commands from either the global or application specific commands.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the commands are removed from the global command set that is used in every application; Global commands are saved across invocations. If this is FALSE it only removes commands for the application. In general, applications should not modify the global command set.

dwCmdStart

[in] Number of the first command in the menu to remove. Command positions are sequential, starting with 1 for the first command on the menu. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Number of commands to remove or the identifier of the commands, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the total number of commands in the menu, the function removes as many commands as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION


IVDctCommands::Set

HRESULT Set(BOOL fGlobal, DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag,
SDATA dData);

Sets information for one or more commands in either the global or application-specific command set.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the commands are set in the global command set that is used in every application; Global commands are saved across invocations. If this is FALSE it only changes commands for the application. In general, applications should not modify the global command set.

dwCmdStart

[in] Number of the first command to set. Command positions are sequential, starting with 1 for the first command on the menu. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Either the number of commands to set or the identifier of the commands, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the number of commands in the menu, the function sets as many commands as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

dData

[in] SDATA structure that contains a pointer to a data buffer and the size of the buffer. The data buffer contains a list of VCMDCOMMAND structures that describe the voice commands to set. Although they vary in size depending on the command data, the structures are contiguous within the list.


IVDctDialogs

The IVDctDialogs interface allows an application to display Windows dialog boxes to configure a speech-recognition engine. The controls and capabilities in these dialog boxes are determined by the particular speech-recognition engine that is being used. If a voice-navigation application is installed on the user's computer, an application does not need to use IVDctDialogs member functions because the voice-navigation application provides access to the engine's dialog boxes.

Before an application can call any IVDctDialogs member function, it must first call the IVoiceDictation::Register member function to register the voice-dictation site.

The IVDctDialogs interface is supported by the voice-dictation object. It has these member functions:

Member function | Description
AboutDlg | Displays an About dialog box that identifies the speech-recognition engine and contains the copyright notice.
GeneralDlg | Allows general control of the engine and should invoke some of the other dialog boxes.
LexiconDlg | Allows the speaker to display and edit his or her pronunciation lexicon.
TrainMicDlg | Provides the engine with training for the microphone.
TrainGeneralDlg | Provides phonetic training, which trains the engine for a preselected set of words.
TrainTopicDlg | Provides training of words specific to the topic.


IVDctDialogs::AboutDlg

HRESULT AboutDlg(HWND hWndParent, PTSTR pszTitle);

Displays an About dialog box that identifies the speech-recognition engine and contains the copyright notice.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the About dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVDctDialogs::GeneralDlg

HRESULT GeneralDlg(HWND hWndParent, PTSTR pszTitle);

Displays a General dialog box, which gives the user general control of the speech-recognition engine and provides the user with full access to engine-specific controls.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the General dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Depending on the engine, the General dialog box may allow users to perform these tasks:

· Inform the engine of the speaker's gender or other speaker-specific information.

· Control how stringent the engine is about recognition.

· Control the maximum amount of disk space allotted for speaker information.

· Control optimizations that reduce processing.

· Export and import speaker information to and from a disk so that this information can be transferred between systems.


IVDctDialogs::LexiconDlg

HRESULT LexiconDlg(HWND hWndParent, PTSTR pszTitle);

Displays a Lexicon dialog box, which allows the user to display and edit his or her pronunciation lexicon. For example, the speaker can edit the phonetics of mispronounced words and load or unload personal exception dictionaries.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Lexicon dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVDctDialogs::TrainGeneralDlg

HRESULT TrainGeneralDlg(HWND hWndParent, PTSTR pszTitle);

Displays a General Training dialog box, which trains a speech-recognition engine by having the user speak a preselected set of words. After the user speaks these words and phrases, recognition accuracy should be better for that particular user.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the General Training dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


IVDctDialogs::TrainMicDlg

HRESULT TrainMicDlg(HWND hWndParent, PTSTR pszTitle);

Displays a Microphone Training dialog box, which trains the speech-recognition engine with training for a microphone. This dialog box should be displayed at the request of the engine or whenever the speaker wants to display it.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Microphone Training dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Training information will be associated with both the speaker and the microphone. Ideally, TrainMicDlg will train the engine only for the microphone and any surrounding sounds, but it is also likely to train for the speaker's voice.

If the engine does not supply this dialog box, the speaker cannot train it for the microphone; however, the application will still work.


IVDctDialogs::TrainTopicDlg

HRESULT TrainTopicDlg(HWND hWndParent, PTSTR pszTitle);

Displays a topic training dialog box, which trains the speech-recognition engine for words specially used in the topic that may need imrpovement. This dialog box should be displayed at the request of the engine or whenever the speaker wants to display it.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDMODE
VCMDERR_INVALIDWINDOW
VCMDERR_NOTSUPPORTED
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Topic Training dialog box or VCMDERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

If the engine does not supply this dialog box, the speaker cannot train it for the topic; however, the application will still work.


IVDctGlossary

IVDctGlossary allows applications to add, remove, and rename glossary entries that are available during dictation.

Member function | Description
Add | Adds one or more glossary entries to voice dictation.
Get | Gets information about one or more glossary entries in voice dictation.
Num | Indicates the total number of glossary entries in voice dictation.
Remove | Removes the specified glossary entries from voice dictation.
Set | Sets information for one or more glossary entries in voice dictation.


IVDctGlossary::Add

HRESULT Add(BOOL fGlobal, DWORD dwCmdNum, SDATA dData, DWORD *pdwCmdStart);

Adds one or more glossary entries to for dictation to support during the current dictation session. The added glossary entries are appended to any existing glossary entries.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the glossary entries are added to the global glossary entrie set that is used in every application; Global glossary entries are saved across invocations. If this is FALSE it only provides glossary entries for the application. In general, applications should not modify the global glossary entry set.

dwCmdNum

[in] Number of glossary entries to add.

dData

[in] SDATA structure containing a list of VCMDCOMMAND structures that describe the voice glossary entries to be added. Although they vary in size depending on the glossary data, the structures are contiguous within the list. The text that the glossary entries enters into the document is in the description field of the command.

pdwCmdStart

[out] Address of a variable that receives the number of the first glossary entry added to the menu.

Glossary entries are numbered sequentially from 1 to n. New glossary entries are added to the end of the menu, so the first glossary entry added is numbered n+1. Unlike Voice Commands, Voice Dictation glossary entries do not support lists.


IVDctGlossary::Get

HRESULT Get(BOOL fGlobal, DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag,
PSDATA pdData, DWORD *pdwCmdNum);

Retrieves information about one or more glossary entries.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the glossary entries are read from the global glossary entry set that is used in every application. If this is FALSE it accesses the glossary entries for the application. In general, applications should not modify the global glossary entry set.

dwCmdStart

[in] Number of the first glossary entry to retrieve. Glossary entries are numbered sequentially from 1 to n. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Either the number of glossary entries to retrieve or the identifier of the glossary entries, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the total number of glossary entries in the menu, the function returns as many glossary entries as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

pdData

[out] Address of an SDATA structure that receives the address and size of a buffer. The buffer contains a list of VCMDCOMMAND structures that describe the glossary entries retrieved. Although they vary in size depending on the glossary entry data, the structures are contiguous within the list. The text that the glossary entries enters into the document is in the description field of the command.

pdwCmdNum

[out] Address of a variable that receives the number of glossary entries actually copied to the buffer.

The calling application allocates the SDATA structure and passes its address to Get. Get allocates memory (using the OLE task allocator) for the returned data and sets the pData member of SDATA to point to the memory. If the allocation fails, pData is set to NULL and the dwSize member is set to zero. The calling application must free the memory pointed to by pData as well as the SDATA structure itself.

The calling application must free the memory allocated by the member function by using the CoTaskMemFree function.


IVDctGlossary::Num

HRESULT Num(BOOL fGlobal, DWORD *pdwNumCmd);

Retrieves the total number of glossary entries in the global or application specific glossary entry set.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the glossary entries are read from the global glossary entry set that is used in every application. If this is FALSE it accesses the glossary entries for the application. In general, applications should not modify the global glossary entry set.

pdwNumCmd

[out] Address of a variable that receives the number of glossary entries.


IVDctGlossary::Remove

HRESULT Remove(BOOL fGlobal, DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag);

Removes the specified glossary entries from either the global or application specific glossary entries.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the glossary entries are removed from the global glossary entry set that is used in every application; Global glossary entries are saved across invocations. If this is FALSE it only removes glossary entries for the application. In general, applications should not modify the global glossary entry set.

dwCmdStart

[in] Number of the first glossary entry in the menu to remove. Glossary entry positions are sequential, starting with 1 for the first glossary entry on the menu. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Number of glossary entries to remove or the identifier of the glossary entries, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the total number of glossary entries in the menu, the function removes as many glossary entries as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION


IVDctGlossary::Set

HRESULT Set(BOOL fGlobal, DWORD dwCmdStart, DWORD dwCmdNum, DWORD dwFlag,
SDATA dData);

Sets information for one or more glossary entries in either the global or application-specific glossary entry set.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
VCMDERR_INVALIDCHAR
VCMDERR_MENUTOOCOMPLEX
VCMDERR_OUTOFDISK
VCMDERR_OUTOFMEM
VCMDERR_VALUEOUTOFRANGE

fGlobal

[in] If this is TRUE, the glossary entries are set in the global glossary entry set that is used in every application; Global glossary entries are saved across invocations. If this is FALSE it only changes glossary entries for the application. In general, applications should not modify the global glossary entry set.

dwCmdStart

[in] Number of the first glossary entry to set. Glossary entry positions are sequential, starting with 1 for the first glossary entry on the menu. If dwFlag is the VCMD_BY_IDENTIFIER value, this parameter is ignored.

dwCmdNum

[in] Either the number of glossary entries to set or the identifier of the glossary entries, depending on the value of dwFlag. If the sum of dwCmdStart and dwCmdNum exceeds the number of glossary entries in the menu, the function sets as many glossary entries as possible.

dwFlag

[in] Flag that identifies the nature of dwCmdNum. This parameter can be one of these values:

VCMD_BY_IDENTIFIER
VCMD_BY_POSITION

dData

[in] SDATA structure that contains a pointer to a data buffer and the size of the buffer. The data buffer contains a list of VCMDCOMMAND structures that describe the voice glossary entries to set. Although they vary in size depending on the glossary entry data, the structures are contiguous within the list. The text that the glossary entries enters into the document is in the description field of the command.


IVDctGUI

The IVDctGUI application allows the application to control the Correction Window supplied by voice commands.

Member function | Description
FlagsGet | Get control flags for the correction window.
FlagsSet | Set control flags for the correction window.
ReplaceUI | Replaces the correction window user interface with the application's own.
SetSelRect | Sets the selection and insertion rectangle.


IVDctGUI::FlagsGet

HRESULT FlagsGet(DWORD *pdwFlags);

pdwFlags is filled in with a set of flags indicating states and settings of the correction window.

pdwFlags

[out] Address of a variable that receives the flags. The flags can be a combination of the VDCTGUIF_XXX flags.


IVDctGUI::FlagsSet

HRESULT FlagsSet(DWORD dwFlags);

Changes the state of flags controlling the correction window.

dwFlags

[in] Flags that indicate the state. The flags can be a combination of the VDCTGUIF_XXX flags.


IVDctGUI::ReplaceUI

HRESULT ReplaceUI(BOOL fReplace);

An application that wishes to replace the standard Correction Window user interface with it's own. When ReplaceUI is TRUE, the standard correction window will be automatically hidden (although calls to FlagsGet will not indicate so) when the window handle specified during IVoiceDictation::Activate has focus.

fReplace

[in] TRUE to replace the standard user interface. FALSE to use the standard user interface.


IVDctGUI::SetSelRect

HRESULT SetSelRect(RECT *pRect);

SetSelRect sets the rectangle in screen coordinates for where the selection or insertion point currently is. An application should call this whenever the window is moved or the insertion/selection coordinates change. If the user has not locked the correction window in place then the correction window will position itself near the selection rectangle. Applications will often call this in response to a selection change notification.

pRect

[in] Pointer to the rectangle containing screen coordinates of the caret or selection.


IVDctInvTextNorm

The IVDctInvTextNorm interface allows applications to control the inverse text normalization performed by the voice dictation object. "Inverse text normalization" is a feature in the voice dictation module which automatically formats text spacing, capitalization, and converts some phrases to a more symbolic form. Example: Inverse text normalization is what converts the spoken phrase, "the time is eight oh five p m period" to "The time is 8:05 PM."

Member function | Description
OptionsEnum | Enumerates the options for inverse text normalization.
OptionsGet | Indicates if an inverse text normalization switch is on or off.
OptionsSet | Turns an inverse text normalization switch on or off.


IVDctInvTextNorm::OptionsEnum

HRESULT OptionsEnum(PSTR *ppOptions, DWORD *pdwSize);

OptionsEnum returns a buffer containing the names and descriptions of all the inverse text normalization options available in this topic.

ppOptions

[out] Filled with a pointer to memory that the calling application must free with CoTaskMemFree(). The data is filled with an alternating sequence of NULL terminated strings. The first string is the name of the first option, followed by the description of the first option, followed by the name of the second option, description of the second option, etc.

pdwSize

[out] Filled in with the number of bytes of data in *ppOptions.


IVDctInvTextNorm::OptionsGet

HRESULT OptionsGet(PCSTR pszOptionName, BOOL *pfOn);

OptionsGet returns the state of the option, whether the switch is on or off. For example, there might be a switch for "Two spaces between sentences" that will put two spaces between sentence is it's on, or only one if it's off.

pszOptionName

[in] Option name. A list of option names can be gotten from OptionsEnum.

pfOn

[out] Filled in with TRUE if the option switch is on, FALSE if it's off.


IVDctInvTextNorm::OptionsSet

HRESULT OptionsSet(PCSTR pszOptionName, BOOL fOn);

OptionsSet changes the state of the option,.

pszOptionName

[in] Option name. A list of option names can be gotten from OptionsEnum.

fOn

[in] TRUE to turn the option on, FALSE to turn it off.


IVDctNotifySink

The IVDctNotifySink interface is used by a voice-dictation object to return status information to an application. This interface is passed to the voice-dictation object by an application when it calls the IVoiceDictation::Register member function to register itself to use voice dictation.

The IVDctNotifySink interface has these member functions:

Member function | Description
AttribChanged | Indicates that an attribute has been modified and which one.
Command | Notifies the application when a command for the application is spoken.
Dictating | Sent when an application starts/stops dictation.
Interference | Notify what kind of interference with the audio signal is happening.
PhraseFinish | Indicates that a speech recognition phrase is finished.
PhraseHypothesis | Hypothesis for what might be spoken.
PhraseStart | Indicates that a speech recognition phrase is being processed.
TextBookmarkChanged | Called when a bookmark position has changed.
TextChanged | Called when some text in the buffer has changed.
TextSelChanged | Called when the current selection has changed.
Training | Indicates that the speaker needs to train the system.
UtteranceBegin | Speech has started.
UtteranceEnd | Speech has finished.


IVDctNotifySink::AttribChanged

HRESULT AttribChanged(DWORD dwAttribute);

Notifies applications on a voice-dictation site that a site attribute has changed.

· Return value is ignored.

dwAttribute

[in] Site attribute that was changed. This parameter can be one of these values:

Value | Meaning
IVDNSAC_AWAKE | Awake state.
IVDNSAC_AUTOGAINENABLE | Automatic gain.
IVDNSAC_DEVICE | Wave-in audio device.
IVDNSAC_ENABLED | Enabled state.
IVDNSAC_MICROPHONE | Current microphone.
IVDNSAC_ORIGINAPP | The application receiving this notification originated the attribute change.
IVDNSAC_SPEAKER | Name of the current speaker.
IVDNSAC_SRMODE | Speech-recognition mode.
IVDNSAC_THRESHOLD | Confidence threshold.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES or VDCTRF_ALLBUTVUMETER value when the application registered to use voice dictation on the site.

dwAttribute includes the IVDNSAC_ORIGINAPP value only if the application sets an attribute by calling the IVDctAttributes interface's ModeSet member function.


IVDctNotifySink::CommandBuiltIn

HRESULT CommandBuiltIn(PTSTR pszCommand);

Sent when a spoken command was recognized and handled. The commands was for the current dictation session and automatically handled by the Voice Dictation module.

· Return value is ignored.

pszCommand

[in] Address of the command string. If this parameter contains NULL, the command was not recognized.

An application can use the CommandBuiltIn notification to monitor commands and inform the user what was heard. Most applications ignore this notification.

If automatic locking is on then the buffer is locked as soon as this callback is made. The application will have to call IVDctText::UnLock.


IVDctNotifySink::CommandOther

HRESULT CommandOther(PTSTR pszCommand);

Sent when a spoken command was recognized and handled. The commands was for another dictation session.

· Return value is ignored.

pszCommand

[in] Address of the command string. If this parameter contains NULL, the command was not recognized.

An application can use the CommandOther notification to monitor commands and inform the user what was heard. An application should not rely on this notification for information about the recognition of its own commands. Most applications ignore this notification.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES or VDCTRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVDctNotifySink::CommandRecognize

HRESULT CommandRecognize(DWORD dwID, DWORD dwFlags,
DWORD dwActionSize, PVOID pAction, PTSTR pszCommand);

Sent when a spoken phrase was recognized as being from the application's command set. Along with the notification, the application receives the text of the phrase and the action data that was supplied by the application when it originally defined the command.

· Return value is ignored.

dwID

[in] Identifier of the command that was recognized. The command identifier is stored in the dwID member of the VCMDCOMMAND structure that defines the command.

dwFlags

[in] VCMDCMD_VERIFY if the application should request verification from the user or NULL if verification is not required. To request verification, the application should display a dialog box. An application would typically require verification for a destructive or irreversible command such as "Format disk."

dwActionSize

[in] Size of the data in pAction.

pAction

[in] Address of a string that contains action data to accompany the recognized command. The action data is obtained from the VCMDCOMMAND structure for the command.

pszCommand

[in] Address of the command string for the command that was recognized.

You should not use the contents of pszCommand to identify the recognized command. Instead, use the data in pAction or the identifier in dwID to determine which command was recognized. The pszCommand string may not contain the same string that you specified in the VCMDCOMMAND structure because it is possible for the user to edit the text for commands for your application using Microsoft Dictation or another voice-aware application.

The notification is sent to all applications that are registered on the voice-command site, regardless of the settings of the dwFlags parameter of the IVoiceDictation::Register member function when the application registered to use voice dictation.

If automatic locking is on then the buffer is locked as soon as this callback is made. The application will have to call IVDctText::UnLock.


IVDctNotifySink::Interference

HRESULT Interference(DWORD dwType);

Notifies the application that the engine cannot recognize speech properly for a known reason.

· Return value is ignored.

dwType

[in] Type of interference. This parameter can be one of these values:

Value | Meaning
SRMSGINT_AUDIODATA_STARTED | The engine has resumed receiving audio data from the audio source.
SRMSGINT_AUDIODATA_STOPPED | The engine has stopped receiving audio data from the audio source.
SRMSGINT_NOISE | The background noise is too high.
SRMSGINT_NOSIGNAL | The engine cannot detect a signal, possibly because the microphone is off or unplugged.
SRMSGINT_TOOLOUD | The speaker is too loud; recognition results may be degraded.
SRMSGINT_TOOQUIET | The speaker is too quiet; recognition results may be degraded.


IVDctNotifySink::PhraseFinish

HRESULT PhraseFinish(DWORD dwFlags, PSRPHRASE pSRPhrase);

Informs the application that the speaker has finished a phrase and the speech-recognition engine is certain about the words that were spoken.

· Return value is ignored.

dwFlags

[in] This parameter can be one or both of the following values:

Value | Meaning
ISRNOTEFIN_RECOGNIZED | The recognition was above the threshold set with the ISRAttributes::ThresholdSet member function. If this flag is not set, the engine did not have enough confidence in the recognition, and it should be declared as an "unknown utterance."
ISRNOTEFIN_THISGRAMMAR | This grammar had the highest score of all active grammars and claimed the recognition. If this flag is not set, another grammar had a higher score and claimed the recognition.

For an application to accept a recognition as valid, both ISRNOTEFIN_RECOGNIZED and ISRNOTEFIN_THISGRAMMAR should be set.

pSRPhrase

[in] Address of an SRPHRASE structure that contains the engine's best recognition choice for the phrase. If dwFlags does not contain either ISRNOTEFIN_RECOGNIZED or ISRNOTEFIN_THISGRAMMAR, pSRPhrase may be NULL.

An application can use the PhraseFinish notification to monitor what speech recognition is controlling the Voice Dictation object. An application should not rely on this notification for information about the recognition of its own commands. Most applications ignore this notification.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES or VDCTRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVDctNotifySink::PhraseHypothesis

HRESULT PhraseHypothesis(DWORD dwFlags, PSRPHRASE pSRPhrase);

Informs the application that the recognition engine has a hypothesis about a phrase.

· Return value is ignored.

dwFlags

[in] This parameter can be one or both of the following values:

Value | Meaning
ISRNOTEFIN_RECOGNIZED | The recognition was above the threshold set with the ISRAttributes::ThresholdSet member function. If this flag is not set, the engine did not have enough confidence in the recognition, and it should be declared as an "unknown utterance."
ISRNOTEFIN_THISGRAMMAR | This grammar had the highest score of all active grammars and claimed the recognition. If this flag is not set, another grammar had a higher score and claimed the recognition.

For an application to accept a recognition as valid, both ISRNOTEFIN_RECOGNIZED and ISRNOTEFIN_THISGRAMMAR should be set.

pSRPhrase

[in] Address of an SRPHRASE structure that contains the engine's best choice for the phrase.

An application can use the PhraseFinish notification to monitor what speech recognition is controlling the Voice Dictation object. An application should not rely on this notification for information about the recognition of its own commands. Most applications ignore this notification.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES or VDCTRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVDctNotifySink::PhraseStart

HRESULT PhraseStart();

Notifies the application that the engine has started processing audio data for a recognition result.

· Return value is ignored.

After sending PhraseStart, the engine may send one or more PhraseHypothesis notifications, but it must always conclude with a PhraseFinish.

An application can use the PhraseStart notification to monitor what speech recognition is controlling the Voice Dictation object. An application should not rely on this notification for information about the recognition of its own commands. Most applications ignore this notification.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES or VDCTRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVDctNotifySink::TextChanged

HRESULT TextChanged(DWORD dwReason);

Notifies the application that text has been added, removed, or changes have been made to the data. The application should call IVDctText::Lock followed by IVDctText::GetChanges to find out what text has changed.

· Return value is ignored.

dwReason

[in] This parameter can be a combination of one or more of the following values:

Value | Meaning
VDCT_TEXTADDED | Text was added.
VDCT_TEXTREMOVED | Text was removed.
VDCT_TEXTREPLACED | Text was replaced.
VDCT_TEXTMOVED | Text was moved.


IVDctNotifySink::TextSelChanged

HRESULT TextSelChanged();

The voice dictation object calls the application to inform the application that the selection has changed. The application should lock the text, find the new position and then update the IVDctGUI::SelSetRect. Some built in commands might cause the selection to change.

· Return value is ignored.


IVDctNotifySink::Training

HRESULT Interference(DWORD dwTrain);

Indicates to the application that the engine requires training for the current speaker, the current grammar, or the current microphone.

· Return value is ignored.

dwTrain

[in] Flag that indicates the recommended type of training. This parameter can be one of these values:

Value | Meaning
SRGNSTTRAIN_MICROPHONE | Training for the current microphone.
SRGNSTTRAIN_GRAMMAR | Training for the current grammar.
SRGNSTTRAIN_GENERAL | General training.

The application must call the appropriate training dialog box for the engine.


IVDctNotifySink::UtteranceBegin

HRESULT UtteranceBegin();

Notifies the application that the engine has detected the beginning of an utterance or sound.

· Return value is ignored.

Detecting the beginning of an utterance doesn't necessarily have any bearing on the recognition of words or the completion of a phrase. Because a call to UtteranceBegin implicitly signals the end of the silence, an application can use this function to detect long periods of silence and react accordingly.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES or VDCTRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVDctNotifySink::UtteranceEnd

HRESULT UtteranceEnd();

Notifies the application that an utterance has ended.

· Return value is ignored.

The engine sends this notification 0.25 to 0.5 seconds after speech stops. This doesn't necessarily have any bearing on the recognition of words or completion of a phrase. Because a call to UtteranceEnd implicitly signals the beginning of silence, an application can use this function to detect long periods of silence and react accordingly.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES or VDCTRF_ALLBUTVUMETER value when the application registered to use voice commands on the site.


IVDctNotifySink::VU Meter

HRESULT VUMeter(WORD wLevel);

Notifies the application of the current VU level, which indicates the loudness of the digital-audio stream.

· Return value is ignored.

wLevel

[in] Value from 0 to 0xFFFF that indicates a linear level. 0xFFFF is the loudest volume that the VU meter can register.

The VU level returned by this function is only a rough estimate used to display a visual indicator on the screen. The engine sends this notification about eight times per second.

The notification is sent only to applications that set the dwFlags parameter of the IVoiceDictation::Register member function to the VDCTRF_ALLMESSAGES value when the application registered to use voice commands on the site.


IVDctText

The IVDctText interface allows applications to query and set the contents of the virtual text edit box that all dictated words are entered into. Applications need to keep the virtual text field synchronized with the one they display to end users.

Almost all of the functions in IVDctText will fail until IVDctText::Lock is called. The Lock call prevents the dictation and inverse text normalization features from modifying the text while the application looks at it. When the application is finished it must call IVDctText::UnLock.

Member function | Description
AutoLockGet | Gets the state of automatic locking.
AutoLockSet | Turns automatic locking on/off.
BookmarkAdd | Places a bookmark in the text.
BookmarkEnum | Enumerates the bookmarks within a section of text.
BookmarkMove | Moves a bookmark within the text.
BookmarkQuery | Finds a bookmark's position.
BookmarkRemove | Removes a bookmark from the text.
FX | Performs built in effects on the text.
GetChanges | Returns what changes occurred since the last call to UnLock().
Hint | Sends a hint down to the speech recognizer.
Lock | Locks the text object so only the application can modify the text.
ResultsGet | Returns the results object associated with a word.
TextGet | Gets text from the internal buffer.
TextMove | Moves text within the internal buffer.
TextRemove | Removes text from the internal buffer.
TextSelGet | Gets the current text selection.
TextSelSet | Sets the current text selection.
TextSet | Writes text to the internal buffer.
UnLock | Unlocks the text object after it was locked.
Words | Tells the recognizer to expect specific words.


IVDctText::AutoLockGet

HRESULT AutoLockGet(BOOL *pfLock);

Gets the value of the automatic locking feature.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pfLock

[out] Returns TRUE if the automatic lock feature is on, FALSE if it is off.


IVDctText::AutoLockSet

HRESULT AutoLockSet(BOOL fLock);

Sets the value of the automatic locking feature.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

fLock

[in] Turns the automatic locking feature on/off.

If automatic locking is on, then after command notifications the text buffer will be locked so that the engine pauses recognition until the buffer is unlocked. This allows the application to do accurate synchronization on machines that are behaving slowly. For example, the sequences of utterances "this" "app-command" "is" will make sure that "app-command" will be processed before "is" is recognized.


IVDctText::BookmarkAdd

HRESULT BookmarkAdd(VDCTBOOKMARK *pBookMark);

This adds an application-specific bookmark into the text. The meaning of the bookmark is left solely up to the application. Bookmarks move with the text so applications can place them to insure that insertions and deletions do not put font formatting out of sync.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pBookMark

[in] Characteristics of the bookmark.


IVDctText::BookmarkEnum

HRESULT BookmarkEnum(DWORD dwStart, DWORD dwNumChars,

PVDCTBOOKMARK *ppBookMark, DWORD *pdwNumMarks);

This examines a range of the text and returns all of the bookmarks contained within it. It returns full information on the bookmark. The application must free up the memory with CoTaskMemFree.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwStart

[in] Starting position in characters. Bookmarks positioned from dwStart to dwStart+dwNumChars, inclusive, will be returned.

dwNumChars

[in] Number of characters. Bookmarks positioned from dwStart to dwStart+dwNumChars, inclusive, will be returned.

ppBookMark

[out] Filled with a pointer to memory containing an array of bookmarks. If none were found this is set to NULL. The memory must be freed by the application with CoTaskMemAlloc.

pdwNumMarks

[out] Filled with the number of bookmarks in ppBookMark.


IVDctText::BookmarkQuery

HRESULT BookmarkQuery(DWORD dwID, VDCTBOOKMARK *pBookMark);

Returns bookmark information for a bookmark.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwID

[in] ID of the bookmark.

pBookMark

[out] Filled in with the bookmark information if successful.


IVDctText::BookmarkRemove

HRESULT BookmarkRemove(VDCTBOOKMARK *pBookMark);

Removes the bookmark.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pBookMark

[in] Bookmark to remove.


IVDctText::FX

HRESULT FX(DWORD dwFX);

Performs an action on the text defined by dwFX.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwFX

[in] Effect to perform on the currently selected text. If a "word" is partially out of the selection it is affected. If the selection is 0 characters (an insertion point), then a word immediately to the left of the insertion point will be affected. If there is no word immediately to the left, then the word immediately to the right. It can be one of the following:

Value | Description
VDCTFX_CAPFIRST | Capitalizes the first letter of the words in the selection.
VDCTFX_LOWERFIRST | Lower cases the first letter of the words in the selection if only the first letter of the word is capitalized.
VDCTFX_TOGGLEFIRST | If the first letter is lower case then the first letter is capitalized and all other letters are lower cased. If the first letter is upper case then the whole word is lower cased.
VDCTFX_CAPALL | Capitalizes all of the letters in the words.
VDCTFX_LOWERALL | Lower-cases all of the letters in the words.


IVDctText::GetChanges

HRESULT GetChanged(DWORD *pdwNewStart, DWORD *pdwNewEnd,

DWORD *pdwOldStart, DWORD *pdwOldEnd);

This informs the applications of the changes that have occurred since the last time the application called IVDctText::Unlock. It returns where the changes started and ended in the old text, and where they now start and end in the new text. Thus, if text was not inserted/deleted then pdwXXXStart == pdwXXXEnd. If text was deleted then pdwOldStart < pdwOldEnd. Likewise if text was inserted, pdwNewStart < pdwNewEnd. If several changes occurred since the last call to UnLock then they will all be wrapped up into one larger change.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pdwNewStart

[out] Filled with the beginning of the new region that was changed.

pdwNewEnd

[out] Filled with the end of the new region that was changed.

pdwOldStart

[out] Filled with the beginning of the old region that was changed.

pdwOldEnd

[out] Filled with the end of the old region that was changed.

Example: If the text was "This is a test" the last time the application called UnLock, and is now changed to "This was a test" then, the following values would be returned:

*pdwNewStart = 5

*pdwNewEnd = 8

*pdwOldStart = 5

*pdwOldEnd = 7


IVDctText::Hint

HRESULT Hint(PTSTR pszHint);

Gives the speech-recognition engine information about the subcontext in which a document is being used. This calls right into the low-level speech recognition API, ISRGramDictation::Hint, and has the same functionality.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

pszHint

[in] Address of a null-terminated string that indicates the kind of text that is being dictated. This parameter can be one of these strings:

String | Description
"Address" | An address
"Date" | A date
"Text" | Free-form text


IVDctText::Lock

HRESULT Lock();

Locks the text object so that nothing except the application can modify it. An application shouldn't keep the text object locked for long. None of the other text functions will work until Lock() is called. Calling lock will call ISRCentral::Lock and prevent the engine from doing any further processing.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY


IVDctText::ResultsGet

HRESULT ResultsGet(DWORD dwStart, DWORD dwNumChars,

DWORD *pdwPhraseStart, DWORD *pdwPhraseNumChars,

PIUNKNOWN *ppIUnkPhraseResults);

This gets the results object for a specific selection so that an application can get more information from it, such as the digital audio. The results objects are documented in the section, "Low-Level Speech Recognition API."

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwStart

[in] Starting character. The results object will be the left-most one that intersects with the region between dwStart and dwStart+dwNumChars.

dwNumChars

[in] Number of characters to search.

pdwPhraseStart

[out] Filled with the starting character of the results object text.

pdwPhraseNumChars

[out] Filled with the number of character of the results object text.

ppIUnkPhraseResults

[out] Filled in with the IUnkown for the results object, or NULL if no results object exists. The application will be responsible for releasing the object.


IVDctText::TextGet

HRESULT TextGet (DWORD dwStart, DWORD dwNumChars, PSDATA pData);

Gets the text from the internal buffer. It returns memory allocated by CoTaskMemAlloc. If the ANSI version of the function is called then the buffer size might vary on DBCS languages.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwStart

[in] Starting character. If this exceeds the size of the buffer then an error is returned.

dwNumChars

[in] Number of characters to get.

pData

[out] The SDATA structure is filled in with a pointer to the memory and the size of the memory. The application must free the memory with CoTaskMemFree.


IVDctText::TextMove

HRESULT TextMove (DWORD dwStart, DWORD dwNumChars,

DWORD dwMoveTo);

This moves text from one section of the virtual document to another. This takes a starting point, number of characters , destination, and reason for the move. If any training data is attached, then that is moved also. Bookmarks are also moved.

The selection (see TextSelGet TextSelSet) is moved or resized by calling this. If the selection is entirely after the move location then it is moved forward/backwards by the number of inserted characters. If the text changed is within the selection then the selection is shrunk/enlarged by the number of characters removed/inserted. If the selection and the text modifications overlap the selection is not changed.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwStart

[in] Starting character. If this exceeds the size of the buffer then an error is returned.

dwNumChars

[in] Number of characters to move.

dwMoveTo

[in] Position to move the starting text to.


IVDctText::TextRemove

HRESULT TextRemove (DWORD dwStart, DWORD dwNumChars,

DWORD dwReason);

This removes text from the buffer. Results objects and bookmarks are removed.

The selection (see TextSelGet TextSelSet) is moved or resized by calling this. If the selection is entirely after the move location then it is moved forward/backwards by the number of inserted characters. If the text changed is within the selection then the selection is shrunk/enlarged by the number of characters removed/inserted. If the selection and the text modifications overlap the selection is not changed.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwStart

[in] Starting character. If this exceeds the size of the buffer then an error is returned.

dwNumChars

[in] Number of characters to remove.

dwReason

[in] Reason for removing the text. It is one of the following values.

Value | Description
VDCT_TEXTCLEAN | Clears the data from the buffer without making any assumption whether or not the text was correct.
VDCT_TEXTKEEPRESULTS | Removes the text from the buffer. The results objects are all called with ISRResCorrection::Validate.


IVDctText::TextSelGet

HRESULT TetSelGet (DWORD *pdwStart, DWORD *pdwNumChars);

Returns the position of the text selection within the virtual text field. Applications should keep this in sync with the text they display to the user. If the selection's screen coordinates change the application should call IVDctGUI::SetSelRect.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pdwStart

[out] Filled with the starting character of the selection.

pdwNumChars

[out] Filled with the number of characters of selection. If pdwNumChars is set to 0 then the selection is actually an insertion point.


IVDctText::TextSelSet

HRESULT TetSelSet (DWORD dwStart, DWORD dwNumChars);

Sets the position of the selection. Applications should keep this in sync with the text they display to the user. If the selection's screen coordinates change the application should call IVDctGUI::SetSelRect.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwStart

[in] Starting character of the selection.

dwNumChars

[in] Number of characters of selection. If dwNumChars is 0 then the selection is actually an insertion point.


IVDctText::TextSet

HRESULT TextSet (DWORD dwStart, DWORD dwNumChars,

PCTSTR pszBuf, DWORD dwReason);

TextSet replaces a text selection defined by the beginning and number of characters. The text is replaced with the new text.

The selection (see TextSelGet TextSelSet) is moved or resized by calling this. If the selection is entirely after the move location then it is moved forward/backwards by the number of inserted characters. If the text changed is within the selection then the selection is shrunk/enlarged by the number of characters removed/inserted. If the selection and the text modifications overlap the selection is not changed.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwStart

[in] Starting character. If this exceeds the size of the buffer then an error is returned.

dwNumChars

[in] Number of characters to replace.

pszBuf

[in] NULL-terminated text string to replace the old text with.

dwReason

[in] Reason for removing the old text. It is one of the following values.

Value | Description
VDCT_TEXTCLEAN | Clears the data from the buffer without making any assumption whether or not the text was correct.
VDCT_TEXTKEEPRESULTS | Removes the text from the buffer. The results objects are all called with ISRResCorrection::Validate.


IVDctText::UnLock

HRESULT UnLock();

Unlocks the text object so that dictation and inverse text normalization can happen.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY


IVDctText::Words

HRESULT Words(PTSTR pszWords);

Informs the speech-recognition engine that a particular set of words is likely to be spoken. The engine can ignore the set of words. This calls right into the low-level speech recognition API, ISRGramDictation::Words, and has the same functionality.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

pszWords

[in] Address of a list of words (separated by spaces) that indicate words likely to be spoken during dictation. The list is terminated by a null character.


IVoiceDictation

The IVoiceDictation interface allows an application to register with a voice-dictation object and to control recognition of speech. The application must call the IVoiceDictation::Register member function before it can call certain other functions, such as those in IVDctDialogs.

The IVoiceDictation interface is supported by the voice-dictation object. It has these member functions:

Member function | Description
Activate | Starts dictation listening.
Deactivate | Stops dictation listening.
Register | Registers a notification sink with the dictation object.
SessionSerialize | Writes a dictation session to disk.
SiteInfoGet | Gets the dictation information pertaining to a site.
SiteInfoSet | Sets the dictation information pertaining to a site.
TopicAddGrammar | Adds a new topic. Takes a grammar object to define the topic.
TopicAddString | Adds a new topic. Parameters are a several strings such as the topic name and language.
TopicDeserialize | Reads all of the topic data to a file.
TopicEnum | Enumerates the topics stored on the PC's hard disk.
TopicRemove | Removes a topic from disk.
TopicSerialize | Writes all of the topic data to a file.


IVoiceDictation::Activate

HRESULT Activate(HWND hWnd);

Activates a dictation session. The application must have called register before calling Activtate. Speech recognition will not necessarily start listening right away after an Activate call. The dictation mode in ISRAttributes::ModeSet must be set to a mode that enables dictation. The session will be listening if and only if Activate has been called AND the foreground window is hWnd AND the site's mode (ISRAttributes::ModeSet/Get) supports dictation. Calling Deactivate will stop dictation for the session. If an application releases the voice dictation object then Deactivate is automatically called.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

hWnd

[in] Specifies the top-level window that must be active for dictation to happen. If the window handle is NULL the recognition will be for all windows, unless preempted by another dictation session specific to a window. Only one application can activate dictation with a NULL handle.


IVoiceDictation::CFGSet

HRESULT CFGSet(LANGID lang, PCSTR pszTopic, PCSTR pszCFG);

CFGSet passes down the text for a Voice Dictation Context Free Grammar to the Voice Dictation module. The grammar is compiled and used for the topic. The CFG is persistent, so it will continue to be used by the topic until it's changed.

lang

[in] Language that the inverse text normalization is for.

pszTopic

[in] Topic which the context free grammar is specific to. If pszTopic is NULL, the new CFG will be used as the default CFG for any topics that do not have a topic-specific CFG.

pszCFG

[in] Context free grammar text. See "Voice Dictation Context Free Grammars" for a description of the text.


IVoiceDictation::Deactivate

HRESULT Deactivate(HWND hWnd);

Deactivates a dictation session and stops it listening. See IVoiceDictation::Activate for more details.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY


IVoiceDictation::Register

HRESULT Register(PCTSTR pszApplication, PCTSTR pszTopic,

PISTORAGE pISessionData, PCTSTR pszSite,

PVOID pNotifyInterface, IID IIDNotifyInterface,

DWORD dwFlags);

Registers necessary information with the dictation object. Most functions and interfaces in the Voice Dictation object will not work until register is called. An application can only call register once per object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszApplication

[in] Application's name. The text string might be displayed to the end user.

pszTopic

[in] Topic to use for the session. If the parameter is NULL or an empty string then a general purpose topic, "General", will be used.

pISessionsData

[in] Interface to an IStorage that contains session data from a previous session. All of the text, results objects, and bookmarks will be loaded in from the previous session. If this is NULL the session will start out with empty text. Alternatively, an application could pass a NULL parameter and then call SessionDeserialize.

pszSite

[in] Site name. If this parameter is NULL or an empty string the default site, "Local PC" will be used. If a site has not been used/created before it will be filled in with default settings.

pNotifyInterface

[in] Pointer to the notification sink interface that will receive all of the notification messages for the dictation session.

IIDNotifyInterface

[in] GUID for the interface of the notification. This can be IID_IVDctNotifySinkA or IID_IVDctNotifySinkA. The application can use IID_IVDctNotifySink, which will map to either the ANSI or UNICODE version depending upon the compile flags.

dwFlags

[in] Flag that indicates whether the application is to receive all notifications. This parameter can be one of these values:

Value | Meaning
VDCTRF_ALLMESSAGES | Sends all notifications to pNotifyInterface.
VDCTRF_ALLBUTVUMETER | Sends all but VUMeter notifications to pNotifyInterface.
VDCTRF_VUMETER | Sends VUMeter notifications to pNotifyInterface.
VDCTRF_NOMESSAGES | Does not send notifications.


IVoiceDictation::SessionDeserialize

HRESULT SessionDeserialize(PISTORAGE pISessionData);

SessionDeserialize loads a serialized session. It replaces all of the text, results object, bookmarks, and selection position information with the state of the information that existed when IVoiceDictation::SessionSerialize was called.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pISessionData

[in] Storage to read all of the session data from.


IVoiceDictation::SessionSerialize

HRESULT SessionSerialize(PISTORAGE pISessionData);

SessionSerialize saves the current state of dictation to IStorage provided. The information includes all of the text in the buffer, results objects associated with the text, and bookmarks. An application can re-load a session during the Register call or by calling SessionSerialize.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pISessionData

[in] Storage to write all of the session data in.


IVoiceDictation::SiteInfoGet

HRESULT SiteInfoGet(PCTSTR pszSite, PVDCTSITEINFO pSiteInfo);

Given the name of a site, this returns properties about the site.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSite

[in] Site name. If this is NULL or an empty string the default site, "Local PC", will be used. If the site does not exist an error will be returned.

pSiteInfo

[out] Filled in with the site information.


IVoiceDictation::SiteInfoSet

HRESULT SiteInfoSet(PCTSTR pszSite, PVDCTSITEINFO pSiteInfo);

Given the name of a site, this sets properties about the site. If any applications are using the site then the properties of the site won't be changed until all applications are closed.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSite

[in] Site name. If this is NULL or an empty string the default site, "Local PC", will be used. If the site does not exist it will be created.

pSiteInfo

[in] New site information.


IVoiceDictation::TopicAddGrammar

HRESULT TopicAddGrammar(SRGRMFMT eFormat, PCTSTR pszTopic, SDATA sData);

Adds a new topic.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

eFormat

[in] Grammar format to be added.

pszTopic

[in] Topic name. If this is NULL or an empty string the general topic, "General", will be used. If the topic exists then an error is returned.

sData

[in] Contains data that will be passed directly to ISRCentral::GrammarLoad. For more information see the section, "Low-Level Speech Recognition API."


IVoiceDictation::TopicAddString

HRESULT TopicAddString(PCTSTR pszTopic, LANGUAGE *pLanguage);

Adds a new topic.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszTopic

[in] Topic name. If this is NULL or an empty string the general topic, "General", will be used. If the topic exists then an error is returned.

pLanguage

[in] Language to use for the topic.

The language and topic name will be compiled into a grammar format to be passed to the speech recognition engine. It will be the standard grammar format for dictation.


IVoiceDictation::TopicDeserialize

HRESULT TopicDeserialize(PCTSTR pszTopic, PISTORAGE pITopicData);

Reads information stored about a topic from a storage and writes it onto the local hard disk. Applications can call TopicSerialize and TopicDeserialize to move information learned in topics from one machine to another.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszTopic

[in] Topic name. If this is NULL or an empty string the general topic, "General", will be used. If the topic exists then an error is returned.

pITopicData

[in] IStorage that was used to serialize the topic.


IVoiceDictation::TopicEnum

HRESULT TopicEnum(PSDATA pData);

Enumerates all of the topics stored on the user's hard disk.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pData

[out] The memory in pData is filled with a doubly NULL-terminated list of topic names.


IVoiceDictation::TopicRemove

HRESULT TopicRemove(PCTSTR pszTopic);

Removes a topic from the set stored on the user's hard disk.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszTopic

[in] Topic name. If this is NULL or an empty string the general topic, "General", will be used. If the topic does not exist then an error is returned.


IVoiceDictation::TopicSerialize

HRESULT TopicSerialize(PCTSTR pszTopic, PISTORAGE pITopicData);

Writes information stored about a topic on the local hard disk to a storage. Applications can call TopicSerialize and TopicDeserialize to move information learned in topics from one machine to another. Much of the information in a serialized session is engine dependent, so if the session is opened on a PC using a different dictation engine than some information will be lost.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszTopic

[in] Topic name. If this is NULL or an empty string the general topic, "General", will be used. If the topic does not exist then an error is returned.

pITopicData

[in] IStorage where the information will be serialized.


Structures

This section describes, in alphabetical order, the structures used by voice-dictation functions.


VDCTBOOKMARK

typedef struct {

DWORD dwID;

DWORD dwPosn;

} VDCTBOOKMARK, * PVDCTBOOKMARK;

Information about a bookmark.

dwID

Identifier for the bookmark. The application to set this to any value it likes and interpret it however it wishes.

dwPosn

Position of the bookmark from the beginning of the virtual text buffer. In characters.


VDCTMEMORY

typedef struct {

DWORD dwMaxRAM;

DWORD dwMaxTime;

DWORD dwMaxWords;

BOOL fKeepAudio;

BOOL fKeepCorrection;

BOOL fKeepEval;

} VDCTMEMORY, *PVDCTMEMORY;

Provides information about memory restrictions for a voice dictation session.

dwMaxRAM

Maximum RAM, in bytes, to be used for storing the recognition results (including digital audio recordings, N-best list, etc.). If this is set to 0 when IVDctAttributes::MemorySet() is called then the current value isn't changed.

dwMaxTime

Maximum time, in seconds, for which the recognition results (including digital audio recordings, N-best list, etc.) will be maintained. If this is set to 0 when IVDctAttributes::MemorySet() is called then the current value isn't changed.

dwMaxWords

Maximum words for which the recognition results (including digital audio recordings, N-best list, etc.) will be maintained. If this is set to 0 when IVDctAttributes::MemorySet() is called then the current value isn't changed.

fKeepAudio

If TRUE then Voice Dictation will keep the ISRResAudio data around for new utterances. If FALSE then it discards the audio right away.

fKeepCorrection

If TRUE then Voice Dictation will keep the ISRResCorrection data around for new utterances. If FALSE then it discards the correction information right away.

fKeepEval

If TRUE then Voice Dictation will keep the ISRResEval data around for new utterances. If FALSE then it discards the evaluation information right away.


VDCTSITEINFO

typedef struct {

DWORD dwAutoGainEnable;

DWORD dwThreshold;

DWORD dwDevice;

DWORD dwMode;

TCHAR szMicrophone[VDCT_MICLEN];

TCHAR szSpeaker[VDCT_SPEAKERLEN];

GUID gModeID;

} VDSITEINFO, * PVDSITEINFO;

Provides information about a site used for Voice Dictation.

dwAutoGainEnable

Automatic gain setting used for IVDctAttributres::AutoGainEnableSet.

dwThreshold

Value from 0 to 100 that indicates the recognition threshold for the speech-recognition engine to be used by the site.

dwDevice

Device identifier of the wave-in audio device to be used by the site. The device identifier can be obtained by calling the waveInGetNumDevs and waveInGetDevCaps multimedia functions.

szMicrophone

Name of the current microphone for the audio source to be used by the site.

szSpeaker

Name of the current speaker for the site.

gModeID

GUID that uniquely identifies the speech-recognition mode to be used by the site. The GUID for a speech-recognition mode can be obtained by using a speech-recognition enumerator. For more information about speech-recognition enumerators, see section, "Low-Level Speech Recognition API."


Constants


VSRMODE_XXX Values

The following flags are used with the member functions of the IVCmdAttributes interface to identify the current mode.:

Value | Meaning
VSRMODE_DISABLED | Dictation and command and control are turned off globally on the system. The user does not want to use them.
VSRMODE_OFF | Speech recognition is temporarily off for the system.
VSRMODE_CMDPAUSED | Speech recognition is paused (listening for a wake-up phrase such as "computer"). The command and control recognition is handling the paused.
VSRMODE_CMDONLY | Command and control recognition is listening. Dictation is off.
VSRMODE_DCTONLY | Dictation is listening. Command and control is off.
VSRMODE_CMDANDDCT | Both command and control, and dictation are on and listening at the same time. This can only be set if the engine can handle simultaneous command and control, and dictation.


VDICTGUIF_XXX Values

The following flags are used with the member functions of the IVCmdGUI interface to identify the state of the correction window.:

Value | Meaning
VDCTGUIF_VISIBLE | The correction window is visible.
VDCTGUIF_DONTMOVE | The correction window has been locked down so it doesn't move with the text when IVCmdGUI::SetSelRect() is called.
VDCTGUIF_ADDWORD | Shows a modeless dialog that allows users to add words.


Interaction with Voice Commands

Voice Dictation will use the speech recognition sharing object. It will share a speech recognition engine with voice-commands if the engine can support both dictation and continuous CFG's. If the engine supports this, users will be able to have dictation and command and control listening at the same time. If two separate engines are used then voice dictation and voice commands will be mutually exclusive, so turning on one will turn off the other.


Tips and Tricks for Applications


Reducing memory footprint

The Voice Dictation object maintains a mirror image of the text that is in the document. If a user loads in a large document, this might get unwieldy, especially if audio recordings are maintained for each of the words. Audio data is inherently large. Here are some ways to alleviate some of the memory problems:

The application can call IVDctAttributes::MemorySet() to specify how much memory should be used by storing the audio and other speech recognition correction information. Of course, if hundreds of pages are being mirrored, there's a minimum amount of memory that will be consumed.

Alternatively, an application might only keep the last page or paragraph that the user has dictated in the Voice Dictation buffer. This places a cap on the size of the data and may work well for the user. However, if the user scrolls back a few pages and wishes to hear what he/she said then they might be disappointed because the information won't be available.


Built In Features


Inverse Text Normalization Features

Voice Dictation provides the following "inverse text normalization" features to modify the text. Some of them can be turned on/off.

Automatic Spacing. Place spaces between words and sentences.

Automatic Capitalization of Words. Learn the capitalization corrections from users and use that capitalization until the user changes it again.

Automatic Capitalization of Sentences. Automatically capitalize words at the beginning of a sentence.

Number to Digits. Converts "six hundred forty two" to "642". It also handles decimals.

Years. Converts "nineteen ninety four" to "1994".


Built In Commands

Voice Dictation provides the following built in commands. More commands may be available, depending upon the language.

"Capitalize". Capitalizes the next word.

"Capitalize-That". Capitalizes the previous word.

"Choose-one" through "Choose-ten". If the correction window is visible this replaces the last word with the Nth entry in the list.

"Cross-Out". Deletes the previous word.

"Go-Back-<1..9>". Moves the cursor back one through nine words.

"Go-Forward-<1..9>". Moves the cursor forward one through nine words.

"Go-To-The-End". Moves the cursor to the end of the document.

"Hide-Window". Hides the correction window.

"Show-Window". Shows the correction window.

"Stop-Dictating". Stops dictation and turns on speech recognition. Start Dictating will be supplied by Voice Commands.

"Today's-date". Type in today's date.


Built In Glossary Entries

Voice Dictation provides the following built in glossary entries. Different glossary entries might be built in, depending upon the language.

"Indent-Paragraph". Inserts a tab character.

Miscellaneous. Miscellaneous conversions like "et-cetera" to "etc."

"New-Line". Inserts a new line.

"New-Paragraph". Inserts two lines.

"One-Space" to "Ten-Spaces". Inserts several spaces.

"Paragraph". Inserts two lines.

Punctuation. Inserts various punctuation marks.


Voice Dictation Context Free Grammars

The Voice Dictation CFG is used by Voice Dictation to perform inverse text normalization. The data is a text string terminated by a NULL. It contains several sections, each section detonated by a word between brackets. The sections are:

[Space]

[Capital]

[Rules]

[Switch]


[Space]

The [space] section defines spacing rules for the topic, determining how many spaces are inserted between words. The space section contains a series of lines (separated by "\n" or "\n\r"). Each line represents a "rule" that determines if a space is used between two words (or symbols.) The parser analyzes each rule in priority from first to last.

Each line contains three "strings", optionally followed by some commands in curly braces. The first string indicates what text occurs to the left of the possible space, the second string indicates what characters are to the right, and the last string contains the number of spaces to insert. Note that the left string is in reverse order.

Each "string" is a quote, followed by a series of characters, ended by a quote, just like a C/C++ string. A backslash followed by another backslash is interpreted as just a backslash, while a backslash followed by another character is interpreted as a control sequence. A capital "A" represents any non-space character.

The commands on the lines can be:

{<number>} and {!<number>} - Indicate that the rule is to be used based upon the state of an options switch. For example, "{5}" indicates that the rule is used if switch 5 is on, while {!5} indicates the rule is only used if switch 5 is turned off.

{E<one character in quotes} and {O<one character in quotes>} - Indicates that the rules is turned on if there are an even (or odd) number of the specified character in the current line. A line is denoted by a carriage-return or line-feed character. Example: {E"\""} looks for an even number of quotes in the current line.

Here are subset of the spacing rules used for Voice Dictation English:

[Space]

"" "." ""

"" "," ""

"" "\\" ""

"" ":" ""

"" ";" ""

"" "%" ""

"" "?" ""

"" "!" ""

"\" " "" ""

"" "\"" " " {E"\""}

"" "\"" "" {O"\""}

"* " "" ""

"" "*" " " {E"*"}

"" "*" "" {O"*"}

.\period "" " " {1}

.\period "" " " {!1}

"?" "" " " {1}

"?" "" " " {!1}

"!" "" " " {1}

"!" "" " " {!1}

"" "/" ""

"/A" "" ""

"-" "-" ""

"--" "" " "

"" "--" " "

"-A" "" ""

"" "-" ""

"(" "" ""

"A" ")" ""

"[" "" ""

"A" "]" ""

"{" "" ""

"A" "}" ""

"<" "" ""

"A" ">" ""

"\n" "" ""

"\r" "" ""

"\t" "" ""

"" "\n" ""

"" "\r" ""

"" "\t" ""

Since the rules are evaluated from first to last, the first rule evaluated is:

"" "." ""

The first string, containing empty quotes, means that it doesn't matter what text is to the left. The next string with just a period indicates that the rule is used if a period is to the right of the possible space insertion. The third string, empty quotes, means that no text is inserted. Hence, the first rule says, don't insert spaces if there's a period to the right of it. This causes periods to "snuggle" up to text to the left.

Further on down the list you see:

.\period "" " " {1}

.\period "" " " {!1}

"?" "" " " {1}

"?" "" " " {!1}

"!" "" " " {1}

"!" "" " " {!1}

This set of rules either places one or two spaces after sentences. If option switch 1 is TRUE then two spaces are inserted between sentences. If it's FALSE then only one space appears between sentences. The first string indicates that a sentence ends with a period, question mark, or exclamation point. The second string indicates that it doesn't matter what's to the right. The third string has one or twos space in it.

Another interesting set of rules is:

"\" " "" ""

"" "\"" " " {E"\""}

"" "\"" "" {O"\""}

This pair of rules causes quotes to work. The first string indicates that a word after a quote snuggles up to the quote. The next two lines cause quotes to alternate between a word before the quote snuggling or not. Without this rule, the phrase, "he said quote hello quote" becomes "He said " hello"". With it, "He said "hello"".


[Capital]

The capitalization rules tell the system when to capitalize the first letter of a word automatically, basically to capitalize at the beginning of the sentence.

Each line is a new rule, and rules are evaluated from first to last. Each rule contains one string (inverted) or a word, and an optional switch to turn it on/off.

Here are subset of the capitalization rules used for Voice Dictation English:

[Capital]

.\period

!\exclamation-point

?\question-mark

Mr.

Mrs.

Ms.

Dr.

Mme.

"\n"

"\r"

"\t"

"\"\n"

"\"\r"

"\"\t"

"\"."

"\"?"

"\"!"

The first few rules are words, which indicate that the next word dictated after those words is to be capitalized.

The "\n", "\r", and "\t" rules cause capitalization to occur on new lines and after tabs.


[Rules]

The [rules] section contains rules for the context free grammar. Each line can be one of the following:

A comment, starting with two slashes, "//".

The definition of a rule.

A rule definition is:

<rule name> = {series of symbols}

The rule name can not have a space in it. It is cases insensitive. If the name is new then a new rule is created. If the name is a duplicate of another rule, then it as added as an alternative to the other rule. The main rule is <Start>, so every rule must be referenced directly or indirectly by <Start> for it to be used.

The symbols can be selected from the following:

<rule name> - Another rule.

"string" - A string that is entered into the text if the rule proves valid.

word - Word that must be spoken for the rule to be valid.

[opt] - Indicates that the next symbol is optional.

[1+] - Indicates that the next symbol must be repeated one or more times.

{<number>} - Requires an option switch to be TRUE in order for the rule to be valid.

{!<number>} - Requires an option switch to be FALSE in order for the rule to be valid.

Here some example rules:

<Start> = {2} <Acronym>

<Start> = {3} <Digits>

<Acronym> = <Letter> [1+] <Letter>

<Letter> = A. "A" <PeriodAfter>

<Letter> = B. "B" <PeriodAfter>

<Letter> = C. "C" <PeriodAfter>

<Letter> = D. "D" <PeriodAfter>

// etc.

<PeriodAfter> = {4} "."

<PeriodAfter> = {!4} ""

<Digits> = <Digit> [1+] <Digit>

<Digit> = zero "0"

<Digit> = oh "0"

<Digit> = one "1"

<Digit> = two "2"

// etc.

The <Start> rule links all of the other rules. It allows a user to speak acronyms or digits. The {2} and {3} are option switches that allow the user to turn on/off the acronym and digit inverse text normalization.

The acronym rule requires that the user speaks at least two letters in a row before it turns on so that if the user dictates "fred z smith" it becomes "Fred Z. Smith" and not "Fred Z Smith". The <Acronym> rule references the <Letter> rule, which contains one entry per letter.

<Letter> = B. "B" <PeriodAfter>

The "B." indicates that the user speaks that letter. The ""B"" is the text typed in. The <PeriodAfter> rule might insert a period if rule 4 is turned on, or nothing if it is off. If rule 4 is on, the dictated text, "F. E. I. F." becomes "F.E.I.F.", but if it's off, the dictated text becomes "FEIF".

Digits work in a similar manner.


[Switch]

The [switches] section provides human readable descriptions of the option switches, with one switch per line. The format is:

<number> = <1 or 0> "name" "description"

The <1 or 0> sets the default state to on or off.

The switches section for the default Voice Dictation CFG looks something like this:

1 = 0 "Two spaces between sentences" "This puts two spaces between sentences."

2 = 1 "Acronyms" "Removes spaces between a sequence of letters."

3 = 1 "Digits" "Removes spaces between a sequence of digits."

4 = 0 "Period in acronyms" "Puts periods in acronyms, like F.E.I.F."


Localization Issues

For the Voice Dictation object to correctly recognize a specific language it must be supplied with an engine supporting the specific language. Most features of the Voice Dictation object will work without any localization of the object, but some features require localization.

Features that need localization:

Default Global Glossary- If no default table for the global glossary exist for the specific language then nothing will be loaded in. An application or user could add their own in. Punctuation is handled by the glossary.

Built-In Commands - If a translation table for the built-in commands does not exist then the English-named commands will be loaded in. An application or user could rename them into the vernacular.

Inverse Text Normalization - If the localized default CFG is not available then the application will need to set one.


Default Rules

These are the default rules for English.

[Space]

"" "." ""

"" "," ""

"" "/" ""

"" "\\" ""

"" ":" ""

"" ";" ""

"" "%" ""

"" "?" ""

"" "!" ""

"\" " "" ""

"" "\"" " " {E"\""}

"" "\"" "" {O"\""}

"* " "" ""

"" "*" " " {E"*"}

"" "*" "" {O"*"}

.\period "" " " {1}

.\period "" " " {!1}

"?" "" " " {1}

"?" "" " " {!1}

"!" "" " " {1}

"!" "" " " {!1}

"-" "-" ""

"--" "" " "

"" "--" " "

"-A" "" ""

"A" "-" ""

"(" "" ""

"A" ")" ""

"[" "" ""

"A" "]" ""

"{" "" ""

"A" "}" ""

"<" "" ""

"A" ">" ""

"\n" "" ""

"\r" "" ""

"\t" "" ""

[Capital]

.\period

!\exclamation-point

?\question-mark

Mr.

Mrs.

Ms.

Dr.

Mme.

"\n"

"\r"

"\t"

"\"\n"

"\"\r"

"\"\t"

"\"."

"\"?"

"\"!"

[Switch]

1= 1 "Spaces Between Sentences" "Places two spaces between sentences ending with a period, question mark, and exclamation point."

[Rules]

// Start

<Start> = <Integer19>

// Natural Numbers

<Natural> = <Less1000000000000000Not>

<Natural> = <Less1000000000000Not>

<Natural> = <Less1000000000Not>

<Natural> = <Less1000000Not>

<Natural> = <Less1000Not>

<Natural> = <Less100Not>

// Natural Numbers, zero through 19 are written out whole

<Natural19> = <Less1000000000000000Not>

<Natural19> = <Less1000000000000Not>

<Natural19> = <Less1000000000Not>

<Natural19> = <Less1000000Not>

<Natural19> = <Less1000Not>

<Natural19> = <Less100Not19>

// Integer, 0..19 not included

<Integer19> = <Natural19>

<Integer19> = <MinusNegative> "-" <Natural>

// 1..9

<1..9> = one "1"

<1..9> = two "2"

<1..9> = three "3"

<1..9> = four "4"

<1..9> = five "5"

<1..9> = six "6"

<1..9> = seven "7"

<1..9> = eight "8"

<1..9> = nine "9"

// Zero through nine

<0..9> = zero "0"

<0..9> = oh "0"

<0..9> = <1..9>

// 10..19

<10.19> = ten "10"

<10.19> = eleven "11"

<10.19> = twelve "12"

<10.19> = thirteen "13"

<10.19> = fourteen "14"

<10.19> = fifteen "15"

<10.19> = sixteen "16"

<10.19> = seventeen "17"

<10.19> = eighteen "18"

<10.19> = nineteen "19"

// 20..99

<20.99> = twenty "20"

<20.99> = twenty-one "21"

<20.99> = twenty-two "22"

<20.99> = twenty-three "23"

<20.99> = twenty-four "24"

<20.99> = twenty-five "25"

<20.99> = twenty-six "26"

<20.99> = twenty-seven "27"

<20.99> = twenty-eight "28"

<20.99> = twenty-nine "29"

<20.99> = twenty "2" <1..9>

<20.99> = thirty "30"

<20.99> = thirty-one "31"

<20.99> = thirty-two "32"

<20.99> = thirty-three "33"

<20.99> = thirty-four "34"

<20.99> = thirty-five "35"

<20.99> = thirty-six "36"

<20.99> = thirty-seven "37"

<20.99> = thirty-eight "38"

<20.99> = thirty-nine "39"

<20.99> = thirty "3" <1..9>

<20.99> = forty "40"

<20.99> = forty-one "41"

<20.99> = forty-two "42"

<20.99> = forty-three "43"

<20.99> = forty-four "44"

<20.99> = forty-five "45"

<20.99> = forty-six "46"

<20.99> = forty-seven "47"

<20.99> = forty-eight "48"

<20.99> = forty-nine "49"

<20.99> = forty "4" <1..9>

<20.99> = fifty "50"

<20.99> = fifty-one "51"

<20.99> = fifty-two "52"

<20.99> = fifty-three "53"

<20.99> = fifty-four "54"

<20.99> = fifty-five "55"

<20.99> = fifty-six "56"

<20.99> = fifty-seven "57"

<20.99> = fifty-eight "58"

<20.99> = fifty-nine "59"

<20.99> = fifty "5" <1..9>

<20.99> = sixty "60"

<20.99> = sixty-one "61"

<20.99> = sixty-two "62"

<20.99> = sixty-three "63"

<20.99> = sixty-four "64"

<20.99> = sixty-five "65"

<20.99> = sixty-six "66"

<20.99> = sixty-seven "67"

<20.99> = sixty-eight "68"

<20.99> = sixty-nine "69"

<20.99> = sixty "6" <1..9>

<20.99> = seventy "70"

<20.99> = seventy-one "71"

<20.99> = seventy-two "72"

<20.99> = seventy-three "73"

<20.99> = seventy-four "74"

<20.99> = seventy-five "75"

<20.99> = seventy-six "76"

<20.99> = seventy-seven "77"

<20.99> = seventy-eight "78"

<20.99> = seventy-nine "79"

<20.99> = seventy "7" <1..9>

<20.99> = eighty "80"

<20.99> = eighty-one "81"

<20.99> = eighty-two "82"

<20.99> = eighty-three "83"

<20.99> = eighty-four "84"

<20.99> = eighty-five "85"

<20.99> = eighty-six "86"

<20.99> = eighty-seven "87"

<20.99> = eighty-eight "88"

<20.99> = eighty-nine "89"

<20.99> = eighty "8" <1..9>

<20.99> = ninety "90"

<20.99> = ninety-one "91"

<20.99> = ninety-two "92"

<20.99> = ninety-three "93"

<20.99> = ninety-four "94"

<20.99> = ninety-five "95"

<20.99> = ninety-six "96"

<20.99> = ninety-seven "97"

<20.99> = ninety-eight "98"

<20.99> = ninety-nine "99"

<20.99> = ninety "9" <1..9>

// Double digit

<DoubleDigit> = <20.99>

<DoubleDigit> = <10.19>

// Three digit number

<TripleDigit> = hundred "1" <Less100Pad>

<TripleDigit> = <1..9> hundred [opt] and <Less100Pad>

<TripleDigit> = <1..9> hundred "00"

// Less that 100, not padded with zeros

<Less100Not> = <1..9>

<Less100Not> = <DoubleDigit>

<Less100Not> = zero "0"

// Less that 100, not padded with zeros, don't do 0..19

<Less100Not19> = <20.99>

// Less that 100, padded with zeros

<Less100Pad> = "0" <1..9>

<Less100Pad> = <DoubleDigit>

<Less100Pad> = zero "00"

// Less than 1000, not padded with zeros

<Less1000Not> = <TripleDigit>

// Less than 1000, padded with zeros

//<Less1000Pad> = "000"

<Less1000Pad> = "0" <Less100Pad>

<Less1000Pad> = <TripleDigit>

// Less than 1,000,000, not padded with zeros

<Less1000000Not> = <1..9> thousand ",000"

<Less1000000Not> = <1..9> thousand [opt] and "," <Less1000Pad>

<Less1000000Not> = <DoubleDigit> thousand ",000"

<Less1000000Not> = <DoubleDigit> thousand [opt] and "," <Less1000Pad>

<Less1000000Not> = <TripleDigit> thousand ",000"

<Less1000000Not> = <TripleDigit> thousand [opt] and "," <Less1000Pad>

// Less than 1,000,000, padded with zeros

<Less1000000Pad> = "000," <Less1000Pad>

<Less1000000Pad> = thousand [opt] and "001," <Less1000Pad>

<Less1000000Pad> = <Less1000Pad> thousand ",000"

<Less1000000Pad> = <Less1000Pad> thousand [opt] and "," <Less1000Pad>

// Less than 1,000,000,000, not padded with zeros

// Disable cause revamp natural: <Less1000000000Not> = <Less1000000Not>

// "million" by itself is not converted - <Less1000000000Not> = million "1,000,000"

// "million" by itself is not converted - <Less1000000000Not> = million [opt] and "1," <Less1000000Pad>

<Less1000000000Not> = <Less100Not> million ",000,000"

<Less1000000000Not> = <Less100Not> million [opt] and "," <Less1000000Pad>

<Less1000000000Not> = <Less1000Not> million ",000,000"

<Less1000000000Not> = <Less1000Not> million [opt] and "," <Less1000000Pad>

// Less than 1,000,000,000, padded with zeros

<Less1000000000Pad> = "000," <Less1000000Pad>

<Less1000000000Pad> = million [opt] and "001," <Less1000000Pad>

<Less1000000000Pad> = <Less1000Pad> million ",000,000"

<Less1000000000Pad> = <Less1000Pad> million [opt] and "," <Less1000000Pad>

// Less than 1,000,000,000,000, not padded with zeros

// Disable cause revamp natural: <Less1000000000000Not> = <Less1000000000Not>

// "billion" by itself is not converted - <Less1000000000000Not> = billion "1,000,000,000"

// "billion" by itself is not converted - <Less1000000000000Not> = billion [opt] and "1," <Less1000000000Pad>

<Less1000000000000Not> = <Less100Not> billion ",000,000,000"

<Less1000000000000Not> = <Less100Not> billion [opt] and "," <Less1000000000Pad>

<Less1000000000000Not> = <Less1000Not> billion ",000,000,000"

<Less1000000000000Not> = <Less1000Not> billion [opt] and "," <Less1000000000Pad>

// Less than 1,000,000,000,000, padded with zeros

<Less1000000000000Pad> = "000," <Less1000000000Pad>

<Less1000000000000Pad> = billion [opt] and "001," <Less1000000000Pad>

<Less1000000000000Pad> = <Less1000Pad> billion ",000,000,000"

<Less1000000000000Pad> = <Less1000Pad> billion [opt] and "," <Less1000000000Pad>

// Less than 1,000,000,000,000,000, not padded with zeros

// Disable cause revamp natural: <Less1000000000000000Not> = <Less1000000000000Not>

// "trillion" by itself is not converted - <Less1000000000000000Not> = trillion "1,000,000,000,000"

// "trillion" by itself is not converted - <Less1000000000000000Not> = trillion [opt] and "1," <Less1000000000000Pad>

<Less1000000000000000Not> = <Less100Not> trillion ",000,000,000,000"

<Less1000000000000000Not> = <Less100Not> trillion [opt] and "," <Less1000000000000Pad>

<Less1000000000000000Not> = <Less1000Not> trillion ",000,000,000,000"

<Less1000000000000000Not> = <Less1000Not> trillion [opt] and "," <Less1000000000000Pad>

// Less than 1,000,000,000,000,000, padded with zeros

<Less1000000000000000Pad> = "000," <Less1000000000000Pad>

<Less1000000000000000Pad> = trillion [opt] and "001," <Less1000000000000Pad>

<Less1000000000000000Pad> = <Less1000Pad> trillion ",000,000,000,000"

<Less1000000000000000Pad> = <Less1000Pad> trillion [opt] and "," <Less1000000000000Pad>


What is Telephony?

Telephony applications are applications that are accessed via the telephone rather than locally over the PC. A GUI application may also support telephony features although the user interface deign for the two interaction mechanisms are significantly different. Many GUI applications support telephony because of the flexibility that a long-distance connection to the PC provides.

Here are some typical telephony applications:

Voice Mail or Answering Machine Software. Most users are familiar with "Voice mail" or computerized answering machine software. These pieces of software allow users to call into a computer and access audio messages that have been left for them. Voice-mail and answering machine software programs are often extended to E-mail, address books, and other types of data.

Accessing Databases. Large numbers of telephony applications allow users to access databases such as movie listings, stock quotes, or news.

Call Routing. Many of the same telephony applications that provide voice-mail or database access also allows incoming calls to be routed to other phone lines. Because most contemporary call routing systems rely on DTMF (touch-tone) to rout the call they ask for an extension number, but with speech recognition this could just as easily be a name.


Why Should a GUI Application use Telephony?

It's obvious why voice-mail and movie-listings applications should be accessible by the phone. But why GUI applications? GUI applications might provide telephony functionality for the following reason:

Remote Access. Any application that provides a functionality that users might want to access when they are not near a computer might use telephony. An application that can control the user's VCR is a great example; After all, how often have you forgotten to set the VCR to record a show only to remember when you were at work?

Database. Any application that provides a database which users might want to access when they are not near a computer might use telephony. This includes E-mail systems, address books, etc.

Reminders and Alerts. Some applications need to remind the user to take action at specific times, such as a schedule program. These can use the telephone lines to call and remind the user no matter where the user is in the world. A scheduling application could remind all of a meeting's participants by giving them a call before the meeting started. Alternatively, a PC could call a user if something of importance had happened. Perhaps when a hard drive on a server crashes over the weekend.


Why Use Speech in Telephony?

Contemporary telephony applications use recordings to play audio to the users, and wait for DTMF (touch-tone) responses from the user. Speech technology has the following advantages over DTMF and recorded audio:

Speech recognition allows a much larger set of inputs than DTMF. While DTMF can only allow the users 12 choices in any state, speech recognition can provide the user with hundreds or thousands of possible responses. Hence, a DTMF system that wishes to ask, "Who do you want to call?" needs a complicated menu structure to maneuver through thousands of names. Speech recognition only has to hear the full name.

In a DTMF application, frequent users either have to memorize or write down an arcane DTMF key sequences so they don't have to listen to all of the prompts. Users of speech recognition systems don't need to do this. It's much easier for users to remember what words to speak than what numbers to press.

In some places in Europe, only 80% of all telephones are rotary and cannot produce DTMF. In the United States of America, 20% of all telephones are rotary and cannot produce DTMF.

Synthesized text-to-speech allows anything to be spoken, allowing E-mail and names to be spoken to the user. It is also cheaper than hiring a professional talent to record, and localizes easier.


Hardware and Software Requirements

Telephony applications use the same speech recognition engines used for Command and Control speech recognition, and the same text-to-speech engines used on the PC.

These hardware and software requirements should be considered when designing a speech application:

· Processor speed. The speech-recognition and text-to-speech engines currently on the market typically require a 486/66 or faster processor.

· Memory. On the average, the combination of speech recognition and text-to-speech will use 2 megabytes (MB) of random-access memory (RAM) in addition to that required by the running application.

· Telephony card. A number of telephony cards are on the market today. On the low end are cards that use FAX/MODEM chips which have been augmented to handle speech. These are included in almost every new home PC. Higher end cards include DSPs or support for multiple phone lines.

· Operating system. The Microsoft Speech application programming interface (API) requires either Windows 95 or Windows NT version 3.5.

· Speech-recognition and text-to-speech engine. Speech-recognition and text-to-speech software must be installed on the user's system. Many new audio-enabled computers and sound cards are bundled with speech-recognition and text-to-speech engines. As an alternative, many engine vendors offer retail packages for speech recognition or text-to-speech, and some license copies of their engines.

For a list of engine vendors that support the Speech API, see the ENGINE.DOC file included with the Speech Software Development Kit.


Limitations

If you haven't already, you should look at the Voice Commands and Voice Text sections of the Microsoft Speech API documentation. Telephony applications use the same technologies.


Half-duplex Voice Modems

Early low-end "Voice Modems" were basically created by bolting record and playback ability onto an existing FAX/MODEM chip-set. Although this made these voice modems cheap, they were lacking some critical features to make speech recognition viable.

Low-cost voice modems are "half-duplex" and do not support "full-duplex" audio. "Half-duplex" means that an application cannot record and play audio at the same time. Because of this, an important feature, "barge-in" is not possible.

"Barge-in" allows a user to interrupt a prompt being spoken by the telephony application. With barge-in, a user can listen to the list of options and speak the one he wants when he hears it. Without barge-in the user must wait for all of the options to be read before speaking. Having to listen to a long prompt can get annoying.

Applications that expect to run on half-duplex voice modems need to make sure that they design around the problems of half-duplex audio, perhaps even using shorter prompts when barge-in is not available on the machine.


Speaker Independence

Speech recognizers for the local PC use are designed to work well for most speakers, but accept that some users, because of a strange accent, dialect, or voice, will not get good accuracy. On the local PC this is not that large of a problem because the user can always train the speech recognizer to his/her voice, and if that doesn't work he/she can always use the keyboard or mouse.

If a telephony recognizer does not work well for an individual then he/she cannot easily use the application. Often times it is not possible for him/her to train the speech recognizer to his/her voice. Additionally, many cellular phones (and wired phones) have such lousy audio quality that humans can barely understand speech on them, let alone computers. Failure of speech recognition will also mean failure of the application.

Application designers need to be aware that speech recognition will not work for some users, so the users must have a back-off. A back-off can take several forms:

Often users get poor recognition because they are speaking in a manner that causes speech recognition to make mistakes. Give new users tips on how to use speech recognition, such as provide concise responses.

Detect that the user is having difficulty with speech recognition and provide queries that have fewer or more distinct answers to improve speech recognition accuracy.

Allow the user to respond with DTMF.

Detect that the user is having difficulty and forward the call to a real operator.


Design the Application for Speech Recognition

Many applications try to use speech recognition by altering the menus and structure only slightly. Rather than telling the user, "To transfer money press one, to open an account press two, etc.." they say, "To transfer money press or say one, etc." While this makes the addition of speech recognition easy it does not take full advantage of it.

An application using speech recognition should be designed from the start to use speech recognition as a primary input device. DTMF should be available, but only as a back-off for those few users that cannot get speech recognition to work.


Telephones don't have monitors, keyboards, or mice

Telephones don't have monitors, keyboards, or mice. Although this is not a limit of the speech recognition and text-to-speech technologies, it does require that application designers rely heavily on the technologies. Because of this, the technologies must be used even for purposes where they don't perform as well as monitors, keyboards, and mice. The lack of other input and output devices significantly changes the user interface. Application designers should be aware of the following effects:

For many uses, speech recognition is not as good of an input device as the keyboard and mouse. When the user is accessing the application over the phone he/she will be forced to use speech recognition exclusively so special user interface design should be taken so that speech recognition's weaknesses don't preclude the use of the application.

Applications that have a GUI continually provide visual cues about the application's state. They have title bars, text displays, and various buttons and other controls that give users a clue about what they can do. Users accessing the application over the telephone do not get this information, or if they do it is delivered to them at the much slower pace of speech. Users of a telephony application often forget the application's state, so it is helpful to remind them occasionally.

Speech is slower at communicating information that video, and it does not easily allow users to select which information they want detailed. This means that telephony applications need to give users trimmed down slices of information and allow the user to specify which of the pieces he/she wants more information about. For example, an E-mail application designed for a GUI will display a list of hundreds of messages. A telephony E-mail application cannot read out the titles of the hundreds of messages. It must provide user interface that allows the user to focus in on the messages he/she wishes to hear. The E-mail application might first ask the user if he/she wants to hear new messages or ones that were already read. From there it could organize messages by priority, etc.

In any particular state a user might have hundreds of options. A GUI can visually display all of the options on the screen, but a telephony application cannot read them all out. When users enter a new state telephony applications should read out an abridged list of options and allow users to ask for more options or more detailed information about an option.

When a user types a number or word into a field on an application they can see the results. Telephony applications cannot display the results so they must provide audio feedback to indicate that they heard the correct information. Because speech recognition often makes mistakes, telephony applications must also provide an easy mechanism for users to correct the mistakes.

Because it is not always obvious when the computer has stopped talking or that the computer has heard the user, the application should give audio feedback. The most effective audio feedback seems to be short beeps. Play one short beep at the end of a question to indicate that the user is expected to speak, another one when a response is recognized, and a third when speech is heard but unrecognized. This not only reassures the user that he/she is being heard, but it also hints to users that they're talking to a computer since real people don't beep.


Telephony Users Often Have Different Motivations

A typical PC users has experience using the computer and the software. They either have personally purchased the software or need to use it to get their job done.

Many telephony applications (like those that provide movie listings) are used by people that have completely different motivations than PC users. Telephony users don't want to talk to a computer; They want to talk to a human to get their task completed. The telephony application is merely a substitute for the real thing, and almost always inferior to the human. However, it's an annoyance they have to live with. Because of this telephony users are less forgiving of poor user interface design and are quick to hang up.

Furthermore, telephony users are a more diverse population. They are from all age groups and education levels. Many of them don't even know that they're talking to a computer and just think it' a very dense operator on the other side of the phone. Telephony applications using speech recognition get better "accuracy" if the users know that they're talking to a computer because the users will adjust their speaking style.


Tweaking the prompts

Current speech recognition technology has the flaw that users must say one of the phrases that the computer expects or accuracy will fall. If the user speaks something that the speech recognizer isn't expecting then either the speech recognizer will return an "unrecognized" response, or worse, it will think it heard another command and do something completely different than what the user wanted.

An application designer should pay close attention to the wording of questions since the phrasing and vocabulary will significantly effect whether or not the user is likely to give one of the expected responses. For example, if the movie application wants to know what time the user wishes to see a movie, it could ask, "What time do you want to see the movie?" However, this can produce responses ranging from "This evening" to "7:00" to "Sometime tomorrow." If the question is reworded to, "Do you want to see an afternoon showing, evening showing, or late night showing?" then the user's response will be more limited.

An application should anticipate synonymous responses. Users will tend to use the same phrasing that they hear from the prompts. If the prompts don't hint at any vocabulary or phrasing then the responses will be varied. In the case of the movie time, the application should expect responses like "In the afternoon," "afternoon", and "afternoon showing". Protypes will show what kind of responses are likely.

Word spotting might work well in some prompts if the recognizer is only looking for a key-word like "afternoon", "evening", or "late night". If more than a few keywords are possible then accuracy decreases.


Verification and Undo

Because speech recognition makes mistakes its important for the application to verify the data with the user. The level of verification can vary. An application might explicitly ask if it heard a response correctly if a misrecognition would cause the user significant problems.

If a misrecognition would only cause minor problems the application should mention the recognition result in passing and allow the user to go back and correct the response. For example, if after asking the time the movie application asked what theater, it could phrase the theater question to include the time. "Where do you want to see the movie, 2001, this evening?"


Skipping prompts/menus

Telephony applications that use DTMF can only provide twelve options at each state (0 though 9, * and #). Speech recognition can provide hundreds and even thousands of options at a state. Even though the speech recognition can handle hundreds of responses, it's not possible to list all of the responses to the user because of the slowness of speech playback.

DTMF menus play a game of "20 questions" with the user. A financial telephony application might first ask the user if they want to get their balance or transfer. If they transfer it then asks the user from what account, and then to which account, followed by how much. Each menu has a limited number of responses.

Speech recognition menus need to provide the same architecture so that novice users are guided by the menus and know what to do. As a result, novice users will maneuver through a speech enabled telephony application at about the same speed as one that provides only DTMF. However, experienced users will learn shortcuts.

Once the user has completed a task the slow way, a telephony application might give the user a hint and tell him how to accomplish the task quicker the next time. For example, the financial application might say, "Next time you call, you can just say, 'Transfer $500 from checkings to savings." The user will be able to bypass several menus the next time, saving him/her time.


Detect the User's Experience Level

If possible, an application should determine the user's experience with the system. If the user logs on the application can keep statistics about individual him/her over several sessions. Alternatively, the system can notice if the user is having difficulty maneuvering through prompts or is whizzing through them. As an application determines the user's experience level it can:

Shorten or elongate prompts depending upon the user's experience.

Emphasize DTMF if the user is having problems with speech recognition.

Provide more or less help.

Send the user to a real operator if they're having a lot of difficulty.


Usability Studies

Traditional GUI applications are intended to be used often and for long periods of time by their users. Any confusing points in the user interface will eventually be overcome through repeated use, and trail and error. The user might even have a manual or help file around to help him/her.

Often, telephony applications are used only occasionally by users, and only for short periods of time. It is likely that large portions of the calls will be from first time users. After all, how many people check the movie listings more than once a week or for more than five minutes at a time?

Because users on average will not have much experience with the application user interface must be as simple and self explanatory as possible. To insure this, the application must be put through a lot of usability studies.

The usability work in a telephony application will typically precede as follows:

Application designers figure out the tasks that a user wishes to accomplish with the application.

A prototype is coded up and a small group of users is given hypothetical tasks to see if they can accomplish them.

The application designers use the feedback from the prototype to modify the application design.

The real application is coded with logging ability. The logging ability keeps track of statistics (listed below) to figure out how successful a call was.

If an existing service is being replaced the a new speech-enabled one, a small percentage of calls are diverted from the existing service into the speech-enabled application. Detailed logging information is kept.

Application designers review the statistics and implement changes to improve performance.

Repeat steps five and six, gradually expanding the number of calls, until the speech-enabled application is handling all of the phone calls.

During the initial stages of an application extensive statistics on performance should be kept. The statistics logging might be disabled when the application goes into full usage, or might be continued so that continual improvement can be made. Some statistics and data to keep are:

How many and which users are actually successful at completing their tasks? In many applications it's obvious when a user has completed his task. In a movie listing application the caller will hang up after he has heard the movie time if it is successful, or hang up before the movie time is played if it is not successful. Some applications are more difficult to test and it may be necessary to poll a small sample of users their satisfaction level.

How long does it take successful users to complete a task? The less time it takes a user to complete a task the happier they'll be, and fewer telephony servers will be needed.

At each state or prompt, how many and which users speak a valid response? If a prompt verifies the data, how often is it wrong? How often does the speech recognizer produce "unrecognized" for a prompt? How long does it take a user to get through a state/prompt? What are the most common user responses? Prompts that take too long to maneuver through, produce a lot of unrecognized results, or a lot of misrecognitions need to be reworked. Features that aren't used can be hidden or removed. The most common responses should be top in the list of responses spoken to the user.

In the early stages of usability tests, applications should keep recordings of everything the user says to the speech recognizer. Application designers can listen to the responses and use them to adjust the wording of prompts or the responses which are accepted by the recognizer. If an application developer is striving for accuracy at any cost and is willing to pay the money to have custom speech recognition models created, the speech recognition vendor can use the audio from real-life users to improve accuracy.


Multi-Line Applications

Most telephony applications are designed to handle several phone lines coming into the same PC. Multi-line telephony applications need to be designed to handle the multiple input channels in such a way that one channel doesn't slow down or harm another channel.

The easiest multi-line application has one process running at least one thread per phone line. Because each line has its own thread, the lines are independent and (generally) one line will not cause another line to slow down. Multi-threaded lines also allow for improved performance on multi-processor CPUs. However, if one of the lines in a multi-threaded application causes a GP Fault, all of the threads will die.

The most stable multi-line telephony design is to have one process per phone line. This insures that one phone line cannot crash and pull down the other lines. It also parallelizes well. It is more difficult to code.


Getting Digits From the User

Even the best speech recognizer have only 99% accuracy per digit. While this might seem high, the user only has a 90% of getting all ten digits in a row correct. Because of this, any sequence of digits that are entered should be played back to the user for verification. If the speech engine misrecognized any of them the application should break the digits into groupings and have the user speak and verify each of the groups. Because the groups have fewer digits they have a higher accuracy. Almost all long sequences of digits are grouped by convention. Example: Phone numbers in the United States of America are two groups of three digits each followed by a group of four digits.


Third Party Controls

A number of ISVs and engine vendors produce "controls" that make telephony applications easier to write because they handle a lot of the user interface work for the application. Telephony controls are akin to Windows buttons, list-boxes, and menus. They provide an easy-to-implement standard that also provides a common user interface for the user.

As an example, an application might use a telephony control to ask the user a yes/no question. To ask the question the application only has to specify the initial question, "Do you want pepperoni on your pizza?" and tell the control to ask away. The control speaks the text, waits for a response, interprets it, and returns the result. The control even handles users that don't respond in time, unrecognized results, or synonymous results such as, "Yeah sure".


What APIs to use

Microsoft does not have a high level telephony API at this time.

Applications wishing to provide telephony functionality should use the low-level speech recognition and text-to-speech APIs.


Why Use the Low-Level API?

The high level speech API (Voice Commands, Voice Text, and Voice Dictation) exist to make speech easy to use, but in the process prevent full control of text-to-speech. In the low level API:

The application must pay attention to many more details than with the high level API.

The application can do anything that it wants to so long as the engine can support the feature.

If standard low-level API doesn't provide for a feature, then engines can add onto API.

The engine is in application's process space.

Multiple instances of engines can be run at once.

Audio comes from or goes to any media that the application desire.


Overview of Text-to-Speech Objects

Text-to-speech objects provide fairly detailed control of text-to-speech. Together, these objects serve as a translation layer for an application, allowing it to use any of the text-to-speech engines available on the system and to send text as audio data to multimedia wave-out devices or other audio destinations. There are the following text-to-speech objects:

· The engine enumerator enumerates the text-to-speech modes for a particular engine. This object is supplied by the engine vendor.

· The text-to-speech enumerator enumerates the text-to-speech modes for all engines available to the application. This object is supplied by Microsoft.

· The engine object provides a text-to-speech mode for the application to use. This object is supplied by the engine vendor.

· The multimedia audio-destination object represents an audio destination and is based on a multimedia device driver. This object is supplied by Microsoft.

To perform text-to-speech translation, an application needs an engine object to convert the text into speech and a multimedia audio-destination object to play speech output.

The sole purpose of the text-to-speech enumerator and engine enumerator is to locate a text-to-speech mode and create an engine object. As soon as the application has the engines it needs, it can release the objects.

The following illustration shows the sequence in which text-to-speech objects are typically created and gives the instructions used to create them.

After an application has created the objects it needs, it can use them to play text as speech. When the text-to-speech engine receives text from the application, the engine subdivides the text into buffers and processes one buffer at a time.

The following illustration shows how an application interacts with the engine and audio-destination objects to play a buffer of text.


Audio-Destination Object

Before an application can use text-to-speech, it must determine where the speech audio should be played. Typical audio destinations include the user's computer, a telephone line, or a file.

To play audio on a wave-out device, the application can create an instance of the multimedia audio-destination object, which is an OLE Component Object Model (COM) object that supports audio communication interfaces in common with the engine. To send audio to a different destination, such as a file, or to manipulate the audio before it reaches the destination device, an application can use a custom audio-destination object that the application developer provides or use one provided by a third-party vendor.


Multimedia Audio-Destination Object

Most applications need to play audio using a multimedia wave-out driver. Microsoft supplies a component object called the multimedia audio-destination object that provides access to the multimedia wave-out driver. To use the multimedia audio-destination object, an application calls the CoCreateInstance function with the CLSID_MMAudioDest class identifier. The function returns the address of the IUnknown interface supported by the audio-destination object. The application needs to specify this address when creating an engine object (described later in this section).

By default, the multimedia audio-destination object uses WAVE_MAPPER as the audio device. To use a different device, the application can call the IUnknown::QueryInterface member function with the IID_IAudioMultiMediaDevice interface identifier. QueryInterface returns the address of the IAudioMultimediaDevice interface. The application can then call the IAudioMultimediaDevice::DeviceNumSet member function to set the identifier of the multimedia device to use as the audio destination.

An application can retrieve a list of wave-out audio devices available in the system by using the WaveOutGetNumDevs and WaveOutGetDevCaps multimedia functions. For more information about these functions, see the Microsoft Win32 Software Development Kit (SDK).

Before an application closes, it must call the IAudioMultimediaDevice interface's Release function to release the audio-destination object.


Custom Audio-Destination Object

A custom audio destination can allow an application to perform the following tasks:

· Send speech output to an audio file for processing at a later time.

· Mix the outgoing audio with another signal, such as sound effects or audio from another text-to-speech engine. (Currently, multimedia does not support the sharing of a wave device.)

· Play audio on hardware that does not support a multimedia driver.

· Apply special effects to the text-to-speech voice, such as reverberation, echo, or distortion.

· "Fool" the text-to-speech engine into thinking that the audio is playing several milliseconds before it is actually being played to give animated graphics time to draw.

To write an audio-destination object, you must create an OLE COM object that supports these three interfaces: IUnknown, IAudio, and IAudioDest. You might also define a custom interface that allows the application to control the audio destination, such as an IAudioFile interface to specify which file to send audio output to.

When an engine first receives the address of the IUnknown interface for the audio destination (for more information, see "Engine Selection" later in this section), it calls the QueryInterface member function for a variety of interfaces, such as those that control specific hardware. If the engine cannot find any of these custom interfaces, it eventually tries the IAudio and IAudioDest interfaces. The engine then calls the IAudio::WaveFormatSet member function with several different wave formats, starting with custom formats and working its way down to pulse code modulation (PCM) if none of the earlier formats are supported. (The application must determine which formats are acceptable.) If the engine cannot find any interfaces or formats that it can use, it fails to create an engine object.

After the engine selects a wave format, it generates the engine object and returns from the call to the ITTSFind::Select or ITTSEnum::Select member function. As the application calls member functions on the engine object, that object calls member functions, such as the IAudio::Start, IAudioSource::DataGet, and so on, on the audio-destination object to get the needed digital audio.

Some suggestions for implementing a custom audio-destination object follow:

· Getting notifications before the audio is actually played. Pass all calls to the custom audio-destination object on to the multimedia audio-destination object, unchanged with this exception: decrement the QWORD time passed with the IAudioDest::BookMark member function by enough time — for example, a few milliseconds — so that the engine sets the bookmark that much earlier than the audio is actually played.

· Mixing audio. Design the audio-destination object to accept only those wave formats that have the same sampling rate as that of the other audio channels. The audio-destination object merely caches the data in a circular buffer that is read by the wave-mix code whenever it needs more data to output to the hardware device.


Engine Object

After an application has created an audio-destination object, its next task is to find a suitable engine and text-to-speech mode and to create an instance of the engine object to represent the mode. An engine typically provides an assortment of text-to-speech modes that it can use to play text in different voices or audio sampling rates.

An application finds a suitable engine and text-to-speech mode by creating a text-to-speech enumerator and using it to find a mode and engine that meet the application's criteria. An application can also enumerate all available modes and select an engine and mode from the list.


Text-to-Speech Enumerator

To create a text-to-speech enumerator, an application calls the CoCreateInstance function with the CLSID_TTSEnumerator class identifier and the IID_ITTSFind interface identifier. This creates a text-to-speech enumeration object with an ITTSFind interface that the application can use to find an engine.

After creating the text-to-speech enumerator and getting the address of its ITTSFind interface, the application fills a TTSMODEINFO structure with values that indicate the kind of engine the application is looking for, such as the language, the dialect, the engine features needed, the engine name, and so on. Any members of the TTSMODEINFO structure that are not relevant to the application can be set to NULL. The application then calls the ITTSFind::Find member function.

If Find succeeds, it fills a second TTSMODEINFO structure with information about the closest matching text-to-speech capabilities available. Because this is only the closest match, an application should check the mode to see if it is acceptable. If it is not, the application can enumerate all modes supported by all engines in the system in an attempt to find an acceptable mode, or the application can fail.

An application can rank certain mode characteristics as more or less important than the engine's default rankings by filling a TTSMODEINFORANK structure that indicates the priority of each TTSMODEINFO member. The application specifies the address of the TTSMODEINFORANK structure along with the TTSMODEINFO structure when using Find.


Engine Enumerator

If an application is not satisfied with the result of the ITTSFind::Find member function, or if the application requires a list of the available text-to-speech modes, it should retrieve the address of the ITTSEnum interface on the text-to-speech enumerator object.

If the application has already created the text-to-speech enumerator and has the address of the ITTSFind interface, it can call the QueryInterface member function of ITTSFind with the IID_ITTSEnum interface identifier.

If the application has not yet created the text-to-speech enumerator, it can call the CoCreateInstance function with the CLSID_TTSEnumerator class identifier and the IID_ITTSEnum interface identifier. This returns the address of the enumerator's ITTSEnum interface, which includes the standard IEnumX member functions Next, Clone, Skip, and Reset. The application can use these member functions to examine the TTSMODEINFO structures for all of the text-to-speech engines and modes installed on the system.


Engine Selection

After an application has identified an acceptable text-to-speech mode, it calls the Select member function to select the mode and create an engine object to represent the mode in the application. Both the ITTSFind and ITTSEnum interfaces include the Select member function, and the syntax is the same for both.

When calling Select, the application specifies a globally unique identifier (GUID) that uniquely identifies the mode to use, the address of a variable that receives the address of the ITTSCentral interface on the engine object, and the address of the IUnknown interface on the audio-destination object created earlier (for more information, see "Audio-Destination Object" earlier in this section).

When Select is called, it ultimately creates an engine object. However, as the engine object initializes, it first attempts to communicate with the audio-destination object to see if they share an acceptable format for digital audio. If they do, the engine finishes initializing and Select returns the address of the ITTSCentral interface on the engine object. If they do not, the engine object frees itself and Select returns an error.

If Select is successful, it returns the address of the ITTSCentral interface on the newly created engine object. After obtaining the address of the ITTSCentral interface on the engine object, the text-to-speech enumerator is no longer needed and the application should release it by using the ITTSFind::Release or ITTSEnum::Release member function.

The ITTSCentral interface provides centralized control of a text-to-speech engine, such as sending text to the engine, injecting speech-inflection tags (control tags) into text as it is played, controlling the playing of the audio output, registering or releasing a notification interface, and so on. The engine uses the notification interface to notify the application of speech-related events. For more information about the notification interface, see "Engine Notifications" later in this section.

The ITTSCentral::ModeGet member function fills a TTSMODEINFO structure with information about the current text-to-speech mode.

The ITTSCentral::Phoneme member function converts text to its phonemic representation, which is the intermediate stage between Unicode text and digital-audio data. An application can use Phoneme to adjust the phonetic representation of text, before it is spoken by the engine, to correct mispronounced words.

An application can use the ITTSCentral::PosnGet member function to retrieve the exact byte in the audio stream that is currently being played, and then use this information to synchronize actions with events. For example, if an action should occur a certain interval after a particular byte is played, the application can call PosnGet to get the time that byte was played, add the interval to the time, and synchronize the action with the result.

The ITTSCentral::AudioPause member function pauses the engine so it does not play any more audio until the application calls the ITTSCentral interface's AudioResume or AudioReset member function, which stops all speech, releases all pending text buffers, and empties the text-to-speech engine's speaking queue.

The ITTSCentral interface includes several other member functions, which are described in subsequent sections of this section.


Text Buffers

Before an engine can begin speaking text, an application must pass the text to the engine. To do this, the application adds text buffers to the engine's speaking queue.

To add a text buffer to the queue, an application calls the ITTSCentral::TextData member function. When calling TextData, the application specifies an SDATA structure that contains the address of the text buffer to add to the speaking queue and a value indicating the number of bytes in the buffer, including the terminating null character. The application also specifies a value indicating the character set that the engine should use to translate the text in the buffer and a flag indicating whether the text contains control tags. Because TextData copies the text buffer into the engine's speaking queue, the application can free the buffer as soon as the member function returns.

Another option is for an application to specify the address of an ITTSBufNotifySink interface when calling TextData. The engine object uses the interface to notify the application about the status of the buffer it placed into the speaking queue. For more information about buffer-related notifications, see "Buffer Notifications" later in this section.

When the engine starts playing the buffer, the application receives an ITTSNotifySink::AudioStart notification. The application receives the ITTSNotifySink::AudioStop notification when the audio finishes playing. If the engine supports visual feedback, the ITTSNotifySink::Visual notification should be called periodically.

To play more buffers, the application calls TextData with the additional buffers. If a previous buffer is still playing, the new buffers are added to the end of the queue. When the engine has finished processing the last text-to-speech buffer, the engine notifies the application by calling the ITTSBufNotifySink::TextDataDone notification. The engine does not wait for the audio object to finish speaking the audio before sending the notification.


Buffer Notifications

An application can receive notifications about the status of the buffers that it places in the engine's speaking queue. For example, the engine can notify the application when a particular buffer starts playing or when the audio for certain words is played.

To receive buffer notifications from the engine, the application must create a COM object that supports the ITTSBufNotifySink interface and specify the address of the interface when calling the ITTSCentral::TextData member function. When a buffer-related event occurs, the engine calls the member function of the ITTSBufNotifySink interface that corresponds to the event. The application developer determines how the member function responds to the notification.

When the engine starts playing a text buffer, it calls the ITTSBufNotifySink::TextDataStarted member function, which receives a QWORD value indicating the time when the engine started reading text from the buffer. The engine calls the ITTSBufNotifySink::TextDataDone member function when it finishes processing the last text-to-speech buffer. The engine does not wait for the audio object to finish speaking the audio before sending the notification. The function receives a QWORD value that indicates the time when the engine finished sending text to the audio-destination object and a flag indicating the reason that the engine finished sending the text. The flag is the TTSBNS_ABORTED value if the application called the ITTSCentral::AudioReset member function.

Some text-to-speech engines notify the application as each word is being spoken by calling the ITTSBufNotifySink::WordPosition member function. The notification includes a QWORD value that indicates the time when the engine started playing the word and the offset from the beginning of the buffer to the word that is currently being played. An application can use this information to highlight the word being played.

An application can use the ITTSCentral::ToFileTime member function to convert QWORD values to Win32 FILETIME structures.

The engine calls the ITTSBufNotifySink::Bookmark member function when it encounters a bookmark in the text. An application can tag text with bookmarks to receive Bookmark notifications when the engine plays the text. These bookmarks can be used as cues for animation or other actions.


Control Tags

An application can embed control tags in text buffers. By using control tags, the application provides clues about the text, so the engine can do a better job of speaking it, perhaps even changing the speaking voice. An application can use control tags to perform the following tasks:

· Change the voice, language, or speaking style. For example, an application might use a different voice for different data types or conduct a conversation between different voices.

· Insert bookmarks in text. Bookmarks notify an application when the audio for the given text has been reached so that the application can switch to a new page in a book, do some animation, or do any other synchronized event.

· Specify how a sentence should be read. An application can instruct the engine to read a sentence quickly, with excitement, or in a monotone.

If text contains tags, the application should set the TTSDATAFLAG_TAGGED value when it calls the ITTSCentral::TextData member function to place a buffer in the engine's speaking queue. As an alternative, the application can call the ITTSCentral::Inject member function to insert control tags into text as the engine plays it.

For more information about about text-to-speech control tags, see Appendix A, "Text-to-Speech Control Tags."


Engine Notifications

Many engines provide notifications that are independent of the text buffer that is being spoken, although the notifications are sent only when at least one buffer is in the queue. These notifications include such things as the time when the audio starts or finishes playing, and visual cues that correspond to the phonemes that are spoken.

To receive engine notifications, the application must create a COM object that supports the ITTSNotifySink interface. Before the engine object can use the interface, the application must register it with the engine by calling the ITTSCentral::Register member function. When calling Register, the application specifies the address of its ITTSNotifySink interface and the IID_ITTSNotifySink interface identifier. If Register succeeds, the engine returns a notification identifier.

When the application no longer needs to receive engine notifications, it can call the ITTSCentral::UnRegister member function, specifying the notification identifier returned by Register. Calling UnRegister is optional, because the system automatically releases all of an application's notification interfaces when the application releases the engine object. However, the system may not release the notification interfaces immediately after the engine is released.

While the application has an ITTSNotifySink interface registered with the engine, it can receive several types of notifications. The engine calls the ITTSNotifySink::AudioStart member function as soon as it starts playing the audio data, and the ITTSNotifySink::AudioStop member function as soon as it stops.

The engine calls the ITTSNotifySink::Visual member function to indicate the phoneme that is currently being spoken and (if the engine supports it) mouth-position cues. An application can use this information to display animated faces. The engine calls Visual at the exact time that the mouth position is reached. If the application cannot render the animation frame quickly enough (within 1/30th of a second), it should implement a custom audio destination that forwards the notification to the application ahead of time. Not all engines call Visual.


Engine Attributes

A text-to-speech engine has a set of attributes that affect various aspects and parameters of text-to-speech translation. Initially, the attributes are set to optimal values as defined by the engine. The application can maintain these values, or use the engine object's ITTSAttributes interface to query and set them. To get the address of ITTSAttributes, call the ITTSCentral::QueryInterface member function with the IID_ITTSAttributes interface identifier.

Whenever an engine attribute has changed, the engine notifies an application by calling the application's ITTSNotifySink::AttribChanged member function. The member function includes a flag that identifies the attribute that changed. In the current version of the speech API, the AttribChanged notification is not very useful. The notification will be more useful in future versions because multiple processes or applications will be able to share the same text-to-speech engine through a text-to-speech sharing object (similar to the speech-recognition sharing object); an application will need to know when another application changes an attribute of the shared engine.

Pitch

An application can set the baseline pitch (in hertz) of the speaking voice for a mode by using the ITTSAttributes::PitchSet member function. It can query for the current pitch by using the ITTSAttributes::PitchGet member function.

If the text being spoken contains text-to-speech control tags, the actual pitch of the speaking voice typically fluctuates above the baseline pitch, but usually does not go below it. The Pit control tag changes the baseline pitch setting for the mode, the same as when PitchSet is called.

Real-Time Setting

The real-time setting is the percentage of processor time that the engine expects to use during constant text-to-speech translation. For example, if the real-time setting is 100, the engine takes one full minute of processor time to process one minute of speech. If the real-time setting is 50, the engine takes 30 seconds of processor time to process the same minute of speech. This value is difficult to compute precisely, so it should be regarded as an estimate.

The real-time setting can be more than 100 for non-real-time applications (for example, applications that translate text from a file). For most engines, the amount of processor time required diminishes markedly during periods of silence.

If an application changes the real-time setting for an engine by calling the ITTSAttributes::RealTimeSet member function, the engine does its best to meet the new real-time expectation. However, a real-time setting of one is not possible for today's personal computers.

An application retrieves the current real-time setting for an engine by using the ITTSAttributes::RealTimeGet member function.

Average Speaking Speed

The text-to-speech engine can speak text at various speeds. An application can set the engine's average speaking speed by using the ITTSAttributes::SpeedSet member function and can retrieve the current average speed by using the ITTSAttributes::SpeedGet member function. The average talking speed is measured in words per minute.

If the engine plays text that contains text-to-speech control tags, the actual speed of the speaking voice fluctuates above and below the average. The Spd control tag changes the baseline speed setting for the mode, the same as when SpeedSet is called.

Speaking Volume

An application can use the ITTSAttributes::VolumeSet member function to set the baseline speaking volume for a text-to-speech mode. To set the volume, the application specifies a value in the range 0x0000 for absolute silence to 0xFFFF for the maximum volume for the mode.

The application specifies the volume for the left and right channels; the level for the left channel is in the low-order word, and the level for the right channel is in the high-order word. The text-to-speech engine or the audio object may not support independent control of the left and right channels. In this case, the engine typically uses the left channel value for the baseline volume.

If the text being played contains control tags, the actual volume of the speaking voice fluctuates above and below this baseline. The Vol control tag changes the baseline volume setting for the mode, the same as when VolumeSet is called.

An application can retrieve the current baseline speaking volume for a mode by using the ITTSAttributes::VolumeGet member function.


Dialog Boxes

A text-to-speech engine provides a set of dialog boxes that enable the user to configure the engine. By using the ITTSDialogs interface provided by a text-to-speech engine object, an application can direct the engine to display one of its dialog boxes. The controls and capabilities in the dialog boxes are determined solely by the engine. An application retrieves the address of the ITTSDialogs interface by calling the IUnknown::QueryInterface member function on the engine object with the IID_ITTSDialogs interface identifier.

When an application calls a member function of the ITTSDialogs interface to display a dialog box, it specifies the handle of the parent window for the dialog box. By specifying NULL instead of a window handle, the application can determine whether the engine supports the given dialog box. The engine returns the NOERROR value if it supports the dialog box or the TTSERR_NOTSUPPORTED error value if it does not.

The application also specifies the title of the dialog box by specifying the address of a title string. If the application specifies NULL instead of a title string address, the engine uses the default title.

About Dialog Box

An engine's About dialog box contains information that identifies the text-to-speech engine and its manufacturer and contains the manufacturer's copyright notice. An application displays the About dialog box by calling the ITTSDialogs::AboutDlg member function.

General Dialog Box

The General dialog box gives the user control of the general features of the text-to-speech engine. An application displays the General dialog box by using the ITTSDialogs::GeneralDlg member function. Depending on the engine, the General dialog box may allow the user to perform the following tasks:

· Set the maximum amount of disk space that the text-to-speech engine can use. The amount of disk space affects the playing of concatenated wave files.

· Set the speaker preferences supported by the engine.

· Provide the user with access to any other dialog boxes provided by the engine.

Lexicon Dialog Box

A text-to-speech engine maintains a database of pronunciations called a pronunciation lexicon, and an engine may allow an application to collect new or corrected pronunciations from the user. The Lexicon dialog box displays the user's pronunciation lexicon and allows the user to edit it. For example, the user can edit the phonetics of mispronounced words and load or unload personal exception dictionaries. An application uses the ITTSDialogs::LexiconDlg member function to display the Lexicon dialog box.

For more information about pronunciation lexicons, see the section, "Common Elements of the Speech Application Programming Interface."

Translation Dialog Box

The ITTSDialogs::TranslateDlg member function causes the engine to display a dialog box that enables the user to control how the engine speaks symbols, currencies, and abbreviations. The user can also control number-translation techniques, such as whether the engine should speak "34" as "thirty-four," "three four," or the old English "four and thirty."


Releasing an Engine Object and Shutting Down

When an application finishes using text-to-speech, it should release all of the outstanding interfaces that it has on text-to-speech objects. This includes those from the engine object. Once all of the objects are free, the engine begins shutting down. The engine releases all notification sink objects from the application.

The application should not shut down until all of its notification sinks (for both the ITTSBufNotifySink and ITTSNotifySink interfaces) have been released. This is important because text-to-speech is an asynchronous process, possibly running in a different process space.


Using Text-to-Speech

This section contains examples demonstrating how to perform the following tasks:

· Initialize an application for using text-to-speech translation. This includes creating the multimedia audio-destination object, the text-to-speech enumerator, the engine object, and the application's buffer-notification object.

· Add a text buffer to an engine's speaking queue.

· Process buffer-related notifications.


Initializing an Application for Text-to-Speech

To implement text-to-speech features, you must initialize the OLE libraries and then create instances of the audio-destination object and the text-to-speech engine object. The engine object translates a text buffer into audio output and passes it to the audio-destination object, which plays the speech output on the audio-output device. In addition, you can use the text-to-speech enumerator and engine enumerator to find an engine and mode that meets your application's requirements.

The following example demonstrates how to initialize an application for text-to-speech. It shows how to create the necessary objects for implementing text-to-speech translation. The example accomplishes the following tasks:

· Uses the CoInitialize function to initialize the OLE libraries.

· Creates an instance of the multimedia audio-destination object, which allows the engine to send speech output to the multimedia wave-out driver.

· Uses the text-to-speech enumerator to find an engine and mode that most closely matches the application's requirements.

· Creates an instance of an application-defined class that implements the ITTSBufNotifySink interface. The engine calls this interface to notify the application of buffer-related events.

// FindAndSelectEngine - creates a multimedia audio-destination object,

// finds and selects a text-to-speech engine and mode, and creates

// the buffer notification object.

// Returns TRUE if successful or FALSE if an error occurs.

//

// Global variables:

// g_pIAMM - address of IAudioMultiMediaDevice interface

// g_TTSModeInfo - address of TTSMODEINFO structure that contains

// information about the selected mode

// g_pIBufNotifySink - address of the application's

// ITTSBufNotifySink interface

BOOL FindAndSelectEngine(void)

{

HRESULT hRes; // result code

TTSMODEINFO ttsmiTemp; // information about the desired mode

PITTSFIND pITTSFind; // address of ITTSFind interface

// Initialize OLE.

if (FAILED(CoInitialize(NULL)))

return FALSE;

// Create the multimedia audio-destination object and retrieve the

// address of the object's IAudioMultiMediaDevice interface.

hRes = CoCreateInstance(CLSID_MMAudioDest, NULL, CLSCTX_ALL,

IID_IAudioMultiMediaDevice, (void**) &g_pIAMM);

if (hRes)

return FALSE;

// Set the device identifier for the multimedia wave-out device.

g_pIAMM->DeviceNumSet(WAVE_MAPPER);

// Create the text-to-speech enumerator and retrieve the address of

// the object's ITTSFind interface.

hRes = CoCreateInstance(CLSID_TTSEnumerator, NULL, CLSCTX_ALL,

IID_ITTSFind, (void**) &pITTSFind);

if (FAILED(hRes)) {

g_pIAMM->Release;

return FALSE;

}

// Define the characteristics of the desired text-to-speech mode.

ZeroMemory(&ttsmiTemp, sizeof(ttsmiTemp));

ttsmiTemp.language.LanguageID =

MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US);

ttsmiTemp.wGender = GENDER_FEMALE;

ttsmiTemp.wAge = TTSAGE_ADULT;

lstrcpy(ttsmiTemp.szStyle, "Business");

// Find the engine and mode that most closely matches the desired

// characteristics.

hRes = pITTSFind->Find(&ttsmiTemp, NULL, &g_TTSModeInfo);

if (hRes) {

g_pIAMM->Release;

pITTSFind->Release();

return FALSE;

}

// Select the text-to-speech mode and create an engine object to

// represent the mode. Also, retrieve the address of the engine's

// ITTSCentral interface.

hRes = pITTSFind->Select(g_TTSModeInfo.gModeID, &g_pITTSCentral,

(LPUNKNOWN) g_pIAMM);

if (hRes) {

g_pIAMM->Release;

pITTSFind->Release();

return FALSE;

};

// Release the text-to-speech enumerator; it is no longer needed.

pITTSFind->Release();

// Create an instance of the buffer notification object based on the

// application-defined CBufNotify class.

if ((g_pIBufNotifySink = new CBufNotify) == NULL)

return FALSE;

return TRUE;

}


Adding a Text Buffer to the Speaking Queue

An application causes a text-to-speech engine to speak text by placing text buffers in the engine's speaking queue. To place a buffer in the queue, an application fills an SDATA structure with the address of the text buffer and the amount of text in the buffer. The application passes the buffer to the engine by calling the ITTSCentral::TextData member function. When calling TextData, the application also specifies the character set that the engine should use to translate the text in the buffer, a flag that indicates whether the buffer contains control tags and, optionally, the address of the application's ITTSBufNotifySink interface. The engine uses this interface to notify the application of buffer-related events.

The following example shows how to add a text buffer to the speaking queue.

// SpeakTheText - puts a text buffer into the text-to-speech

// engine's speaking queue.

// Returns TRUE if successful or FALSE if an error occurs.

// pszTextBuffer - address of the text buffer

//

// Global variables:

// g_pIBufNotifySink - address of the ITTSBufNotifySink interfaceBOOL SpeakTheText(LPSTR pszTextBuffer)

{

HRESULT hRes;

SDATA data;

data.dwSize = lstrlen(pszTextBuffer);

data.pData = pszTextBuffer;

// Put the text buffer into the engine's speaking queue, and give

// the engine the address of the ITTSBufNotifySink interface for

// buffer-related notifications.

hRes = g_pITTSCentral->TextData(CHARSET_TEXT, 0, data,

g_pIBufNotifySink, IID_ITTSBufNotifySink);

if (FAILED(hRes))

return FALSE;

return TRUE;

}


Processing Buffer Notifications

The text-to-speech engine object calls an application's ITTSBufNotifySink interface to inform the application of buffer-related events so that the application can respond to them. To receive notifications, the application must create a COM object that supports the ITTSBufNotifySink interface and must pass the address of the interface to the engine object when calling the ITTSCentral::TextData member function.

The ITTSBufNotifySink interface consists of a set of member functions that correspond to buffer-related events. When an event occurs, the engine object calls the member function that corresponds to the event.

The following example shows how to define an object class that implements the ITTSBufNotifySink interface.

class CBufNotify : public ITTSBufNotifySink {

private:

public:

CBufNotify(void);

~CBufNotify(void);

// IUnknown members

STDMETHODIMP QueryInterface (REFIID, LPVOID FAR *);

STDMETHODIMP_(ULONG) AddRef(void);

STDMETHODIMP_(ULONG) Release(void);

// ITTSNotifySink

STDMETHOD (BookMark) (QWORD, DWORD);

STDMETHOD (TextDataDone) (QWORD, DWORD);

STDMETHOD (TextDataStarted) (QWORD);

STDMETHOD (WordPosition) (QWORD, DWORD);

};

typedef CBufNotify * PCBufNotify;


Low-Level Text-to-Speech Quick-Look


ILexPronounce

This interface is supported by the text-to-speech engine.

Member function | Description
Add | Adds the pronunciation and part of speech of a word to the engine's pronunciation lexicon.
Get | Gets the pronunciation and part of speech of a given word from the lexicon.
Remove | Removes the pronunciation and part of speech of the given word from the lexicon.


ITTSAttributes

This interface is supported by the text-to-speech engine.

Member function | Description
PitchGet | Gets the current baseline pitch for a text-to-speech mode.
PitchSet | Sets the baseline pitch for a text-to-speech mode.
RealTimeGet | Gets the real-time setting for the engine.
RealTimeSet | Sets the new percentage of processor time that the engine expects to use during constant speech.
SpeedGet | Gets the current baseline average talking speed for a text-to-speech mode, in words per minute.
SpeedSet | Sets the baseline average talking speed for a text-to-speech mode, in words per minute.
VolumeGet | Gets the current baseline speaking volume for the text-to-speech mode.
VolumeSet | Sets the baseline speaking volume for a text-to-speech mode.


ITTSBufNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
Bookmark | Notifies the application that the engine has encountered a bookmark in the text being played.
TextDataDone | Notifies the application that the text in a particular text-to-speech buffer has been sent to the audio-destination object or that the application called the ITTSCentral:AudioReset member function.
TextDataStarted | Notifies the application that a particular text-to-speech buffer has reached the beginning of the text-to-speech buffer queue.
WordPosition | Notifies the application which text is currently being played.


ITTSCentral

This interface is supported by the text-to-speech engine.

Member function | Description
AudioPause | Immediately pauses text-to-speech output for an engine object.
AudioReset | Resets the digital-audio stream.
AudioResume | Resumes text-to-speech output that has been paused.
Inject | Injects text-to-speech control tags into the text currently being spoken.
ModeGet | Gets information specific to the current text-to-speech mode, such as certain qualities of the voice.
Phoneme | Converts text to a phonemic representation, which is an intermediate stage between Unicode text and digital-audio data.
PosnGet | Gets the exact byte in the audio stream that is currently being played.
Register | Registers an ITTSNotifySink interface for the engine to use to notify the application of buffer-independent text-to-speech events.
TextData | Starts the process of converting text into audio data to be spoken.
ToFileTime | Converts a QWORD representing the time in bytes to a Win32 FILETIME structure.
UnRegister | Allows an application to release a notification interface through which an engine sent text-to-speech notifications to an application.


ITTSDialogs

This interface is supported by the text-to-speech engine.

Member function | Description
AboutDlg | Displays an About dialog box that identifies the text-to-speech engine and contains the copyright notice.
GeneralDlg | Allows general control of the text-to-speech engine and should invoke some of the other dialog boxes.
LexiconDlg | Allows the speaker to display and edit his or her pronunciation lexicon.
TranslateDlg | Allows the end-user to control symbols, currencies, abbreviations, and number-translation techniques, as well as translation of abbreviations and case sensitivity, and how both affect the prosody of spoken text.


ITTSEnum

This interface is supported by the text-to-speech enumerator and the engine enumerator.

Member function | Description
Clone | Returns another enumerator containing the same enumeration state as the current one.
Next | Retrieves the specified number of items in the enumeration sequence.
Reset | Resets the enumeration sequence back to the beginning.
Select | Selects a text-to-speech mode and creates an engine object to represent the selected mode.
Skip | Skips a specified number of elements in the enumeration sequence.


ITTSFind

This interface is supported by the text-to-speech enumerator and the engine enumerator.

Member function | Description
Find | Finds the text-to-speech mode that most closely matches the desired characteristics.
Select | Selects a text-to-speech mode and creates an engine object to represent the selected mode.


ITTSNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
AttribChanged | Notifies an application that an engine attribute has changed.
AudioStart | Notifies an application when audio data starts playing.
AudioStop | Notifies an application when audio data stops playing.
Visual | Notifies an application which phoneme is being used in the current digital-audio stream.


Objects

This section describes the OLE objects that an application can use to implement text-to-speech.


Engine Object


Purpose

Represents a text-to-speech mode of an engine, which processes text into audible speech. An engine typically provides an assortment of text-to-speech modes that can be used to play text in different voices or audio sampling rates. The engine object is supplied by the engine vendor.


Created by

To create an engine object, you first use a text-to-speech enumerator to locate the mode to use and then call the ITTSEnum::Select member function with the GUID of the mode and the address of the IUnknown interface of an audio-destination object.

As the engine object initializes, it calls the IAudio interface on the audio-destination object to determine whether or not it can use that audio destination. If it can use it, the engine object calls the IAudio::PassNotify member function with an IAudioDestNotifySink notification interface, so the audio-destination object can notify the engine when audio data is played. If the engine cannot use the specified audio destination, the Select member function returns an error and the engine object releases itself.

If Select is successful, it returns the address of the ITTSCentral interface on the newly created engine object.


Interacts with

The engine object receives notifications from the audio-destination object about the status of audio being played, such as whether the audio-destination object has space available for more text.

The engine object can notify the application about events that pertain to text-to-speech, such the time that audio starts or stops playing. To receive notifications, the application calls the ITTSCentral::Register member function on the engine object and passes the address of the notification interface. The application must create a COM object that supports the ITTSNotifySink interface. The application developer determines how the application responds to notifications.


Interfaces

The engine object should support the ITTSAttributes, ITTSCentral, and ITTSDialogs interfaces, and it may also support the ILexPronounce interface. An engine object may support other interfaces defined by the engine vendor to provide additional functionality.

There are the following interfaces.

Interface | Description
ILexPronounce | (Optional) Allows an application to query and control the pronunciation lexicon for a speech-recognition or text-to-speech engine.
ITTSAttributes | Controls the attributes of a text-to-speech engine. Member functions allow an application to adjust the pitch, speed, and volume of the voice and the engine's share of the processor.
ITTSBufNotifySink | Notifies an application of changes to the buffer that contains the text being spoken.
ITTSCentral | Controls an engine object. Member functions allow an application to send text to the engine; inject speech-inflection tags (control tags) into text as it is played; convert Unicode text to a phonemic representation; pause, resume, and reset the audio output; obtain information about the text-to-speech mode; register or release a notification interface; retrieve the time that a byte in the audio stream was played; and convert the time to a Win32 FILETIME structure value.
ITTSDialogs | Displays Windows dialog boxes that allow the user to configure the text-to-speech engine, such as controlling how symbols and currencies are pronounced.
ITTSNotifySink | Notifies an application of engine-specific events related to processing text into speech, such as a change of attributes, the time that audio starts or stops playing, and hints for synchronizing animation with the phoneme that is being spoken.


Engine Enumerator


Purpose

Enumerates the text-to-speech modes supported by a particular engine. An application does not interact with the engine enumerator directly; instead, it uses the text-to-speech enumerator, which passes member function calls to the engine enumerator. The engine enumerator is supplied by the engine vendor.


Created by

The engine enumerator is created when the application calls the ITTSFind::Find member function on a text-to-speech enumerator. The application passes the address of a TTSMODEINFO structure that indicates the desired characteristics of the mode and the address of a TTSMODEINFORANK structure that indicates the relative importance of each characteristic. As an alternative, the application can call ITTSEnum member functions to enumerate the modes.

The text-to-speech enumerator creates an engine enumerator for each engine registered on the system and calls each engine enumerator.


Interacts with

When an engine enumerator receives a ITTSFind::Find call from the text-to-speech enumerator, the engine enumerator searches its list of text-to-speech modes for the closest match to the characteristics specified by Find. (If the engine enumerator does not support the ITTSFind interface, it uses the ITTSEnum interface to scan the list.)

If the engine enumerator locates an appropriate text-to-speech mode, it returns the address of an TTSMODEINFO structure that contains information about the mode, including its GUID. The application then calls the ITTSFind::Select member function with this GUID to select the mode and create an engine object.

Once an engine object exists, the engine enumerator is no longer needed. The engine enumerator is released automatically when the application releases the text-to-speech enumerator.


Interfaces

The engine enumerator should support the ITTSEnum interface and may support the ITTSFind interface. The engine enumerator may support other interfaces defined by the engine vendor to provide additional functionality.

Interface | Description
ITTSEnum | Enumerates and selects a text-to-speech mode.
ITTSFind | (Optional) Finds and selects the text-to-speech mode that most closely matches the requested characteristics.

For more information about common enumeration functions, see the section, "Common Elements of the Speech Application Programming Interface."


Text-to-Speech Enumerator


Purpose

Enumerates the text-to-speech modes provided by all of the engines available to the application. The application uses the text-to-speech enumerator to select a text-to-speech mode and create an engine object to represent that mode within the application. The text-to-speech enumerator is supplied by Microsoft.


Created by

To create a text-to-speech enumerator, call the CoCreateInstance function with the CLSID_TTSEnumerator class identifier and the IID_ITTSFind or IID_ITTSEnum interface identifier, depending on the interface desired.

CoCreateInstance returns the address of the requested interface on the newly created text-to-speech enumerator.


Interacts with

When a text-to-speech enumerator is created, it loads the registry entries for each engine installed on the system and creates an engine enumerator object to represent each engine.

The application calls the ITTSFind::Find member function on the text-to-speech enumerator with the address of a TTSMODEINFO structure that indicates the desired characteristics of the mode and the address of a TTSMODEINFORANK structure that indicates the relative importance of each characteristic. As an alternative, the application can call ITTSEnum member functions to enumerate the modes.

The text-to-speech enumerator passes these calls to each engine enumerator until one of them returns a mode that fits the requested characteristics.

After the application has created an engine object, the text-to-speech enumerator is no longer needed and the application can release it.


Interfaces

The text-to-speech enumerator supports the following interfaces.

Interface | Description
ITTSEnum | Enumerates and selects a text-to-speech mode.
ITTSFind | Finds and selects the text-to-speech mode that most closely matches the requested characteristics.

For more information about common enumeration functions, see the section, "Common Elements of the Speech Application Programming Interface."


Interfaces and Member Functions

Text-to-speech interfaces allow an application to play text over an audio destination using a text-to-speech engine. There are the following interfaces.

Interface | Description
ITTSAttributes | Controls the attributes of a text-to-speech engine, such as the pitch, talking speed, and the volume of a voice.
ITTSBufNotifySink | Used by a text-to-speech engine to notify an application of changes to the buffer that contains the text being spoken.
ITTSCentral | Controls the playback of text by a text-to-speech engine, including sending text to the engine, pausing and resuming playback, getting information about the engine or the text being played, and registering a notification sink.
ITTSDialogs | Displays Windows dialog boxes that allow the user to configure the text-to-speech engine.
ITTSEnum | Enumerates and selects a text-to-speech mode.
ITTSFind | Finds and selects the text-to-speech mode that most closely matches the requested characteristics.
ITTSNotifySink | Used by a text-to-speech engine to notify an application that the audio has started or stopped or to notify an application of the phoneme being spoken.

In addition to the interfaces described in this section, each object supports IUnknown, which is a common interface supported by all OLE component objects. A text-to-speech engine may also support ILexPronounce, an optional interface that allows the user to correct words in the engine's pronunciation lexicon. For more information about IUnknown and ILexPronounce, see the section, "Common Elements of the Speech Application Programming Interface."


ITTSAttributes Interface

The ITTSAttributes interface allows the user of an engine object to control various attributes and parameters of text-to-speech. Initially, attribute values are set to an optimal level as defined by the engine. The application can change these values or maintain them.

The ITTSAttributes interface is supported by the text-to-speech engine. It has these member functions:

PitchGet | SpeedGet
PitchSet | SpeedSet
RealTimeGet | VolumeGet
RealTimeSet | VolumeSet


ITTSAttributes::PitchGet

HRESULT PitchGet(WORD *pwPitch);

Retrieves the current baseline pitch for a text-to-speech mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

pwPitch

[out] Address of a variable that receives the current baseline pitch, in hertz, for a text-to-speech mode.

If the text being spoken contains text-to-speech control tags, the actual pitch of the speaking voice typically fluctuates above this baseline. The Pit tag changes the baseline pitch setting for the mode, the same as when the PitchSet member function is called. The default baseline pitch is the optimal pitch for the mode.


ITTSAttributes::PitchSet

HRESULT PitchSet(WORD wPitch);

Sets the baseline pitch for a text-to-speech mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

wPitch

[in] New baseline pitch for the voice, in hertz. If the value is out of range for the engine, an error is returned and the pitch is not changed. An application can specify TTSATTR_MINPITCH or TTSATTR_MAXPITCH for minimum or maximum allowable values.

The actual pitch of the voice typically fluctuates above this baseline. It usually does not go below it.


ITTSAttributes::RealTimeGet

HRESULT RealTimeGet(DWORD *pdwRealTime);

Retrieves the real-time setting for the engine, which is the percentage of processor time that the engine expects to use during constant speech.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

pdwRealTime

[out] Address of a variable that receives the real-time setting. This value can be over 100 for non-real-time applications.

For most engines, the amount of processor time required diminishes markedly during periods of silence. For example, if pdwRealTime is 100, the engine takes 1 full minute of processor time to process one minute of speech. If pdwRealTime is 50, the engine takes 30 seconds of processor time to process the same minute of speech. This value is difficult to compute precisely, so it should be regarded as an estimate.


ITTSAttributes::RealTimeSet

HRESULT RealTimeSet (DWORD dwRealTime);

Sets the real-time setting for the engine. The real-time setting is the percentage of processor time that the engine expects to use during constant speech.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

dwRealTime

[in] New real-time setting. An application can specify TTSATTR_MINREALTIME or TTSATTR_MAXREALTIME for minimum or maximum allowable values. If the value is out of range for the engine, the member function returns an error, and the real-time setting is not changed.

The engine tries to meet the real-time expectation, but the engine will not be able to consume exactly dwRealTime percentage of the CPU.


ITTSAttributes::SpeedGet

HRESULT SpeedGet(DWORD *pdwSpeed);

Retrieves the current baseline average speed for a text-to-speech mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

pdwSpeed

[out] Address of a variable that receives the baseline average speed for the voice, in words per minute.

If the text being played contains text-to-speech control tags, the actual speed of the speaking voice fluctuates above and below this baseline. The Spd tag changes the baseline speed setting for the mode, the same as when the SpeedSet member function is called. The default baseline speed is the optimal speed for the mode.


ITTSAttributes::SpeedSet

HRESULT SpeedSet(DWORD dwSpeed);

Sets the baseline average speed for a text-to-speech mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

dwSpeed

[in] New baseline average speed, in words per minute. An application can specify TTSATTR_MINSPEED or TTSATTR_MAXSPEED for minimum or maximum allowable values. If the value is out of range for the engine, an error is returned and the speed is not changed.


ITTSAttributes::VolumeGet

HRESULT VolumeGet(DWORD *pdwVolume);

Retrieves the current baseline speaking volume for the text-to-speech mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

pdwVolume

[out] Address of a variable that receives the volume level for the voice. The volume level is a linear range from 0x0000 for absolute silence to 0xFFFF for the maximum volume for the mode. The level for the left channel is in the low-order word, and the level for the right channel is in the high-order word.

If the text being played contains text-to-speech control tags, the actual volume of the speaking voice fluctuates above and below this baseline. The Vol tag changes the baseline volume setting for the mode, the same as when the VolumeSet member function is called.


ITTSAttributes::VolumeSet

HRESULT VolumeSet(DWORD dwVolume);

Sets the baseline speaking volume for a text-to-speech mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

dwVolume

[in] New baseline speaking volume. The low-order word specifies the volume for the left channel, and the high-order word specifies the volume for the right channel. An application can specify TTSATTR_MINVOLUME or TTSATTR_MAXVOLUME for minimum or maximum allowable values. If the values are out of range for the engine, the function returns an error and the speaking volume is not changed.

The text-to-speech engine or the audio object may not support independent control of the left and right channels. In this case, the engine typically uses the left channel value for the baseline volume.


ITTSBufNotifySink Interface

The ITTSBufNotifySink interface is used by a text-to-speech engine to notify an application of changes to the buffer that contains the text being spoken. This interface is passed to the engine when the ITTSCentral::TextData function is called. The WordPosition member function is optional and may or may not be supported by an engine. An application does not have to supply an ITTSBufNotifySink interface; it is optional.

The ITTSBufNotifySink interface is supported by an application. It has these member functions:

BookMark

TextDataDone

TextDataStarted

WordPosition


ITTSBufNotifySink::BookMark

HRESULT BookMark(QWORD qTimeStamp, DWORD dwMarkNum);

Notifies the application that the engine has encountered a bookmark in the text being played.

· Return value is ignored.

qTimeStamp

[in] Time, in bytes, when the bookmark was set. An application can ignore this parameter.

dwMarkNum

[in] Bookmark identifier.

Bookmarks come from tags in tagged text and can be placed in text by embedding the Mrk tag before playing the text or by using the ITTSCentral::Inject member function to insert the tag on the fly. BookMark notifications are not sent if the ITTSCentral::AudioReset member function is called.


ITTSBufNotifySink::TextDataDone

HRESULT TextDataDone(QWORD qTimeStamp, DWORD dwFlags);

Notifies the application that the text in a particular text-to-speech buffer has been sent to the audio-destination object or the application called the ITTSCentral::AudioReset member function.

· Return value is ignored.

qTimeStamp

[in] Time, in bytes, when the engine finished sending all of the text in a text-to-speech buffer to an audio-destination object. An application can ignore this parameter.

dwFlags

[in] Reason that the text data is finished. If this parameter is NULL, all of the text in the text-to-speech buffer has been sent to the audio-destination object. If this parameter is the TTSBNS_ABORTED value, the application called the ITTSCentral::AudioReset member function.

This notification is sent when the text-to-speech buffer is emptied, either by sending its data to the audio-destination object or by the application calling the ITTSCentral::AudioReset member function to stop sending the text and empty the buffer. The ITTSBufNotifySink interface is released by the engine after an pending BookMark or WordPosition notifications are sent.

Because the notification is sent after the audio finishes processing, the application may receive the notification before the audio in the buffer actually finishes playing.


ITTSBufNotifySink::TextDataStarted

HRESULT TextDataStarted(QWORD qTimeStamp);

Notifies the application that a particular text-to-speech buffer has reached the beginning of the text-to-speech buffer queue.

· Return value is ignored.

qTimeStamp

[in] Time, in bytes, when the engine started reading text from a text-to-speech buffer. An application can ignore this parameter.


ITTSBufNotifySink::WordPosition

HRESULT WordPosition(QWORD qTimeStamp, DWORD dwByteOffset);

Notifies the application of the word that is currently being played.

· Return value is ignored.

qTimeStamp

[in] Time, in bytes, when the engine started playing the word. An application can ignore this parameter.

dwByteOffset

[in] Offset, in bytes, from the beginning of the text-to-speech buffer to the word that is currently being played.

An application can use the information returned by WordPosition to highlight the word being played.


ITTSCentral Interface

The ITTSCentral interface provides centralized control of a text-to-speech engine, allowing for actions such as text entry and conversion of phonemes.

The ITTSCentral interface is supported by the text-to-speech engine object. It has these member functions:

AudioPause | PosnGet
AudioReset | Register
AudioResume | TextData
Inject | ToFileTime
ModeGet | UnRegister
Phoneme |


ITTSCentral::AudioPause

HRESULT AudioPause(void);

Immediately pauses text-to-speech output for an engine object.

· Returns NOERROR if successful, or the <E_OUTOFMEMORY> error value otherwise.

Calling AudioPause affects output for the current engine object only. When the engine object is first created, text-to-speech output is not paused.

Unless the engine is paused, it immediately plays any text sent to it using the ITTSCentral::TextData member function. To defer playing of text, you should call AudioPause before calling TextData. Any text that is sent to the engine is not played until the application calls the ITTSCentral::AudioResume member function.


ITTSCentral::AudioReset

HRESULT AudioReset(void);

Resets the digital-audio stream.

· Returns NOERROR if successful, or the <E_OUTOFMEMORY> error value otherwise.

When AudioReset is called, the engine stops playing the current text and releases any pending text-to-speech buffers, emptying the text-to-speech speaking queue. If the engine is in the paused state (as a result of a call to AudioPause), the engine returns to an unpaused state. BookMark notifications are not sent, even if the text contains bookmarks. When the queue is empty, the engine calls the ITTSBufNotifySink::TextDataDone notification and then the ITTSBufNotifySink::Release notification.


ITTSCentral::AudioResume

HRESULT AudioResume(void);

Resumes text-to-speech output that has been paused.

· Returns NOERROR if successful, or the <E_OUTOFMEMORY> error value otherwise.

Calling AudioResume affects output for the current engine object only.


ITTSCentral::Inject

HRESULT Inject( TCHAR *pszTag);

Injects text-to-speech control tags into the text currently being spoken.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDCHAR>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

pszTag

[out] Address of a string that contains the tag to inject. A string can contain multiple tags, separated by spaces. The tags in the string should be in the same format as tags embedded in text — for example, "\Spd=360\ \Vce=Gender="Female"\".

Do not use Inject to inject bookmark tags (Mrk).

An application can use this member function to control speaking parameters on the fly without having to reconstruct the text-to-speech buffers. If the engine understands a tag, it should act on the tag as soon as possible after receiving it.


ITTSCentral::ModeGet

HRESULT ModeGet(PTTSMODEINFO pttsInfo);

Retrieves information about the current text-to-speech mode, such as certain qualities of the voice.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>

pttsInfo

[out] Address of a TTSMODEINFO structure that receives information about the current text-to-speech mode.


ITTSCentral::Phoneme

HRESULT Phoneme(VOICECHARSET eCharacterSet, DWORD dwFlags,
SDATA dText, PSDATA pdPhoneme);

Converts text to a phonemic representation, which is an intermediate stage between Unicode text and digital-audio data.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDCHAR>
<TTSERR_NOTSUPPORTED>

eCharacterSet

[in] Character set to use to convert the text. This parameter can be one of these values:

Value | Meaning
CHARSET_ENGINEPHONETIC | An engine-specific phonetic character set
CHARSET_IPAPHONETIC | The International Phonetic Alphabet (IPA) character set

CHARSET_TEXT cannot be used as the character set for converted text.

dwFlags

[in] TTSDATAFLAG_TAGGED if the text contains text-to-speech control tags that alter inflection, speed, and so on, or NULL if the text does not contain tags. Note that an engine-specific phonetic character set may use a different tagging protocol than that used by the IPA character set. This is left for future expansion.

dText

[in] SDATA structure that contains the address of the null-terminated text string to be converted. If the engine supports the IPA character set, tags can be used to intersperse words written in IPA phonemes with words in the language's character set. The application should not change this memory until the function returns.

pdPhoneme

[out] Address of an SDATA structure that receives a pointer to the phonemic representation of the text using the specified character set. The text string is null-terminated. The character set is wide or multibyte, depending upon the interface used (ITTSCentralW or ITTSCentralA). Engine-specific character sets must follow the wide/multibyte convention.

The calling application allocates the SDATA structure and passes its address to Phoneme. Phoneme allocates memory (using the OLE task allocator) for the returned data and sets the pData member of SDATA to point to the memory. If the allocation fails, the pData member is set to NULL and the dwSize member is set to zero. The calling application must free the memory pointed to by pData, as well as the SDATA structure itself.

This function can be used to adjust the phonetic representation of text before it is spoken by the engine, such as correcting mispronounced words.

Phoneme is synchronous; that is, the member function does not return until the conversion is complete. Depending on the amount of text passed, this process can take some time to complete, so an application should call Phoneme with a background thread.

If the CHARSET_IPAPHONETIC value is used in the ANSI version of Phoneme, the function returns an error.

It is up to the calling application to free the memory allocated by the function, using the CoTaskMemAlloc and CoTaskMemFree functions.


ITTSCentral::PosnGet

HRESULT PosnGet(QWORD *pqwTimeStamp);

Retrieves the exact byte in the audio stream that is currently being played (or as exact as possible given the audio device).

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>

pqwTimeStamp

[out] Address of a variable that receives the byte in the audio stream that is currently being played.

The PosnGet function works only for real-time audio destinations, such as audio speakers; it does not work for non-real-time destinations, such as files.

An application can use PosnGet to synchronize actions with events. For example, if an action should occur a certain interval after a particular byte is played, the application can call PosnGet to get the time that byte was played, add the interval to the time, and synchronize the action with the result.


ITTSCentral::Register

HRESULT Register(PVOID pNotifyInterface, IID IIDNotifyInterface,
DWORD *pdwKey);

Registers an ITTSNotifySink interface for the engine to use to notify the application of buffer-independent text-to-speech events.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDINTERFACE>

pNotifyInterface

[in] Address of the notification interface through which the engine notifies the application about buffer-independent text-to-speech information. The interface identifier is specified by IIDNotifyInterface.

Because passing the address to the engine does not transfer ownership of the notification interface, the engine must call the AddRef member function of the notification interface before returning from the call to Register. The engine must also call the Release member function of the notification interface when the ITTSCentral::UnRegister member function is called for the given key. The calling application must release any reference counts it holds on the notification interface after calling Register, unless it needs the notification object to be valid after calling UnRegister.

IIDNotifyInterface

[in] Interface identifier of the interface used for notification. In the current implementation, the identifier must be IID_ITTSNotifySinkA (for ANSI) or IID_ITTSNotifySinkW (for Unicode). An engine can define additional values to support custom interfaces. An application can use the IID_ITTSNotifySink identifier to get ANSI or Unicode, depending on the default for the application.

pdwKey

[out] Address of a variable that receives a key for the notification interface. The application must supply this key with the UnRegister member function to release the interface.

An application can call Register multiple times. The member function returns a different key each time it is called.

An application can use IID_ITTSNotifySink to get ANSI or Unicode, depending on the default for the application. For more information about ANSI and Unicode versions of interfaces and structures, see the section, "Common Elements of the Speech Application Programming Interface."


ITTSCentral::TextData

HRESULT TextData(VOICECHARSET eCharacterSet, DWORD dwFlags, SDATA dText,
PVOID pNotifyInterface, IID IIDNotifyInterface);

Starts the process of converting text into audio data to be spoken.

· Returns NOERROR if successful, or one of these error values:

<E_FAIL>
<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDCHAR>
<TTSERR_QUEUEFULL>
<TTSERR_WAVEDEVICEBUSY>

eCharacterSet

[in] Character set to be used to convert the text. This parameter can be one of these values:

Value | Meaning
CHARSET_ENGINEPHONETIC | An engine-specific phonetic character set
CHARSET_IPAPHONETIC | The IPA character set
CHARSET_TEXT | The text character set

If the engine cannot handle the character set, it returns an error.

dwFlags

[in] TTSDATAFLAG_TAGGED if the text contains text-to-speech control tags that alter inflection, speed, and so on, or zero if the text does not contain tags. Note that an engine-specific phonetic character set may use a different tagging protocol than that used by the IPA character set. This is left for future expansion.

dText

[in] SDATA structure. The pData member contains the null-terminated string for the text to be spoken, and the dwSize member specifies the length of the string, including the terminating NULL. Because TextData copies this text into one or more text-to-speech buffers, the application can free the source buffer as soon as the function returns. When the engine finishes processing the last text-to-speech buffer, the engine notifies the application by calling the ITTSBufNotifySink::TextDataDone member function. The engine does not wait for the audio object to finish speaking the audio before calling TextDataDone. The size of a text-to-speech buffer is determined by the engine. If the engine cannot allocate enough memory to store the text-to-speech buffers, it should return E_OUTOFMEMORY.

If dText contains an empty (NULL) string, the engine returns NOERROR and sends both the ITTSBufNotifySink::TextDataStarted and ITTSBufNotifySink::TextDataDone notifications.

pNotifyInterface

[in] Address of the notification interface through which the engine notifies the application about the status of the text-to-speech buffer. The interface identifier is specified by IIDNotifyInterface. If the application does not require notifications, this parameter can be NULL.

Because passing the address does not transfer ownership of the notification interface, the engine must call the AddRef member function of the notification interface before returning from the call to TextData. The engine must also call the Release member function of the notification interface when it closes. The calling application must release any reference counts it holds on the notification interface after calling TextData, unless it needs the notification object to be valid when the engine releases it.

IIDNotifyInterface

[in] Interface identifier of the interface used for notification. In the current implementation, the identifier must be IID_ITTSBufNotifySinkA (for ANSI) or IID_ITTSBufNotifySinkW (for Unicode). An engine can define additional values to support custom interfaces.

TextData is asynchronous — that is, the member function returns before all of the text is played.

TextData passes the text to the engine, which performs the actual conversion of text to audio; however, this function may initiate the conversion in a different thread. The engine can use a background thread to convert the text.

Text is loaded into a series of buffers. As each buffer is loaded, the engine converts the text to audio. If the engine receives the next buffer while it is still processing the previous one, the next buffer is appended to the previous buffer's text. If the engine cannot allocate enough memory to store the text-to-speech buffers, it should return E_OUTOFMEMORY.

Each text-to-speech buffer is assumed to be a continuation of the previous buffer and is not autonomous. To make the buffer autonomous, send an Rst (reset) tag. If an engine reaches the end of a buffer and finds no more buffers in the text-to-speech buffer queue, the engine can assume that the end of the buffer completes a statement. For best results, the SDATA structure specified by dText should contain the address of a complete sentence or paragraph. If the structure contains the address of a complete sentence or paragraph, the text-to-speech buffer that TextData passes to the engine for conversation to audio data will be aligned on a sentence or paragraph boundary.

If TextData is called while the engine is processing the previous buffer, the engine concatenates the new buffer with the previous one. For example, if the previous buffer contains "<a bunch of words> wal" and TextData adds another buffer containing "ked <a bunch more words>", the engine should speak, "<a bunch of words> walked <a bunch more words>." However, if the engine has already finished processing the previous buffer (because it processes a sentence or paragraph ahead), it speaks, "<a bunch of words> wal. ked <a bunch more words>."

Initially, an engine object plays any text that is sent to it as soon as possible. To change this state, you should call the ITTSCentral::AudioPause member function. Once AudioPause is called, no audio is output until the ITTSCentral::AudioResume member function is called.


ITTSCentral::ToFileTime

HRESULT ToFileTime(PQWORD pqTimeStamp, FILETIME *pFT);

Converts a QWORD representing the time in bytes to a Win32 FILETIME structure.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_VALUEOUTOFRANGE>

pqTimeStamp

[in] Time, in bytes, returned by the engine.

pFT

[out] Address of a variable that receives the Win32 FILETIME value of pqTimeStamp.

An application can use the value returned by ToFileTime to correlate real-time events with a time value returned by the engine. The value returned by the engine is an estimate that should be very accurate within a few seconds of the present, but becomes less accurate farther away from the present, due to time variations in the audio card's clock.


ITTSCentral::UnRegister

HRESULT UnRegister(DWORD dwKey);

Allows an application to release a notification interface through which an engine sent text-to-speech notifications to an application.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDKEY>

dwKey

[in] Key value returned by the Register function for the notification interface when the application registered the interface.

An application must call UnRegister before closing the engine object.


ITTSDialogs Interface

The ITTSDialogs interface allows an application to display Windows dialog boxes to configure a text-to-speech engine. The controls and capabilities in these dialog boxes are determined by the particular text-to-speech engine that is being used. All dialog boxes supported by the engine are modal, and calls to the ITTSDialogs interface are synchronous. All of the ITTSDialogs member functions use the same syntax.

The ITTSDialogs interface is supported by the text-to-speech engine object. It has these member functions:

AboutDlg

GeneralDlg

LexiconDlg

TranslateDlg


ITTSDialogs::AboutDlg

HRESULT AboutDlg(HWND hWndParent, PWSTR pszTitle);

Displays an About dialog box that identifies the text-to-speech engine and contains the copyright notice.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>
<TTSERR_INVALIDWINDOW>

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the About dialog box, or TTSERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


ITTSDialogs::GeneralDlg

HRESULT GeneralDlg(HWND hWndParent, PWSTR pszTitle);

Displays a General dialog box that gives the user general control of the text-to-speech engine and gives the user full access to engine-specific controls.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDWINDOW>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the General dialog box or TTSERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Depending on the engine, GeneralDlg may allow the user to perform these tasks:

· Set the maximum amount of disk space allotted for text-to-speech. This is relevant for playing concatenated wave files.

· Set the speaker preferences supported by the engine.

· Access all of the standard dialog boxes.


ITTSDialogs::LexiconDlg

HRESULT LexiconDlg(HWND hWndParent, PWSTR pszTitle);

Displays a Lexicon dialog box that allows the speaker to view and edit his or her pronunciation lexicon. For example, the speaker can edit the phonetics of mispronounced words and load or unload personal exception dictionaries.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDWINDOW>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Lexicon dialog box or TTSERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


ITTSDialogs::TranslateDlg

HRESULT TranslateDlg(HWND hWndParent, PWSTR pszTitle);

Displays a Translation dialog box that lets the user control symbols, currencies, abbreviations, and number-translation techniques, such as whether to speak "34" as "thirty-four," "three four," or the old English "four and thirty." This dialog box also allows the user to control the translation of abbreviations and case sensitivity and how both affect the prosody of spoken text.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDWINDOW>
<TTSERR_NOTSUPPORTED>
<TTSERR_VALUEOUTOFRANGE>

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the Translation dialog box or TTSERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


ITTSEnum Interface

The ITTSEnum interface allows an application to enumerate the different text-to-speech modes and to select a mode for which to create an engine object.

Calling ITTSEnum on an engine enumerator allows an application to recurrently traverse through its list of known text-to-speech modes. Calling ITTSEnum on a text-to-speech enumerator calls engine enumerators for all of the text-to-speech engines on the system.

The ITTSEnum interface is supported by the text-to-speech enumerator and the engine enumerator. It has these member functions:

Clone

Next

Reset

Select

Skip

All OLE enumeration interfaces contain the functions Clone, Next, Reset, and Skip. In the ITTSEnum interface, these functions use TTSMODEINFO structures as the elements in the enumeration sequence. For more information about Clone, Next, Reset, and Skip, see the section, "Common Elements of the Speech Application Programming Interface."


ITTSEnum::Select

HRESULT Select(GUID gModeID, PITTSCENTRAL *ppiTTSCentral,
PIUNKNOWN pIUnknownForAudio);

Selects a text-to-speech mode and creates an engine object to represent the selected mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDMODE>
<TTSERR_OUTOFDISK>
<TTSERR_WAVEFORMATNOTSUPPORTED>

gModeID

[in] Globally unique identifier (GUID) from the TTSMODEINFO structure for the currently selected text-to-speech mode. The function creates an engine object to represent this mode.

ppiTTSCentral

[out] Address of a variable that receives a pointer to the ITTSCentral interface for the newly created engine object.

pIUnknownForAudio

[out] Address of the IUnknown interface for the audio-destination object that provides digital-audio data for the engine object. The engine will AddRef the interface, and release the interface when the engine is destroyed. Because of this the application will need to call Release on the audio object after the select call or audio objects will be leaked.

After the application enumerates the text-to-speech modes and determines which one to use, it calls Select to create the engine object.

As the engine object initializes, it calls the IAudio interface of the audio-destination object specified by pIUnknownForAudio to determine whether it can use that audio destination. If it can, the function returns an ITTSCentral interface for the newly created engine object. If it cannot, the engine object releases itself and the Select function returns an error.

After the object is created, it calls the IAudio::PassNotify member function to pass its own notification sink to the audio-destination object. Passing the notification sink to the audio-destination object allows the object to notify the engine when audio data is played. When all references to the engine object are released, the engine object calls the IAudio::Release member function.

The multimedia audio-destination object can be used to send audio to a wave-out device. The Select function passes the IUnknown interface rather than the IAudioDest interface because, although all engines should support IAudioDest, some engines may use custom interfaces for dealing with special hardware or drivers and IUnknown can be used to obtain such interfaces. For more information about the multimedia audio-destination object, see the section, "Multimedia Audio Objects ."

The pIUnknownForAudio parameter is obtained by using the CoCreateInstance function. As such, the reference count for the IUnknown interface begins at 1, and the interface should be released after Select has returned.

The negotiation with the audio-destination object occurs within the call to Select, so if Select returns without an error, the negotiation succeeded.


ITTSFind Interface

The ITTSFind interface allows an application to search through text-to-speech modes to select a mode that most closely matches the desired characteristics and to create an engine object that represents the selected mode.

Because the ITTSFind interface selects a text-to-speech mode based on the relative importance of various characteristics, it may not always select the optimal mode (just as the Windows font-matching function does not always select the optimal font). The ITTSFind interface is best used to select a "default" text-to-speech mode. An application should also allow the user to select a mode and should store its GUID so the mode can be retrieved later.

The ITTSFind interface for the text-to-speech enumerator uses ITTSFind for all of the registered text-to-speech engines and then selects from the closest result. A text-to-speech engine may also supply an ITTSFind interface. However, if it does not, the ITTSFind interface of the text-to-speech enumerator uses the engine's ITTSEnum interface to simulate this functionality.

The ITTSFind interface is supported by the text-to-speech enumerator and the engine enumerator. It has these member functions:

Find

Select


ITTSFind::Find

HRESULT Find(PTTSMODEINFO pttsInfo, PTTSMODEINFORANK pttsInfoRank,
PTTSMODEINFO pttsInfoFound);

Finds the text-to-speech mode that most closely matches the desired characteristics.

· Returns NOERROR if successful, or one of these error values:

<E_FAIL>
<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDMODE>

pttsInfo

[in] Address of a TTSMODEINFO structure that contains the desired characteristics.

pttsInfoRank

[in] Address of a TTSMODEINFORANK structure that contains the relative importance of each characteristic. If this parameter is NULL, the object uses the default ranking provided by the engine.

pttsInfoFound

[out] Address of a TTSMODEINFO structure that receives information about the text-to-speech mode that was found. It is valid only if an error value is not returned.

If no engines can be found or the system runs out of memory, an error is returned.


ITTSFind::Select

HRESULT Select(GUID gModeID, PITTSCENTRAL *ppiTTSCentral,
PIUNKNOWN pIUnknownForAudio);

Selects a text-to-speech mode and creates an engine object to represent the selected mode.

· Returns NOERROR if successful, or one of these error values:

<E_INVALIDARG>
<E_OUTOFMEMORY>
<TTSERR_INVALIDMODE>
<TTSERR_OUTOFDISK>
<TTSERR_WAVEFORMATNOTSUPPORTED>

gModeID

[in] GUID from TTSMODEINFO for the currently selected text-to-speech mode. The function creates an engine object to represent the mode.

ppiTTSCentral

[out] Address of a variable that receives a pointer to the ITTSCentral interface for the newly created engine object.

pIUnknownForAudio

[out] Address of a variable that receives a pointer to the IUnknown interface for the audio-destination object that plays the digital-audio data sent by the engine object. The engine will AddRef the interface, and release the interface when the engine is destroyed. Because of this the application will need to call Release on the audio object after the select call or audio objects will be leaked.

The pIUnknownForAudio parameter is obtained by using the CoCreateInstance function. As such, its reference count begins as 1 and should be released after the Select operation has been performed.

For more information about the parameters of the ITTS_Find::Select member function, see the description of the ITTSEnum::Select member function.


ITTSNotifySink Interface

The ITTSNotifySink interface is used by an engine to return status information to the application. This interface is passed to the engine by a text-to-speech enumerator when it registers the engine. The Visual member function is optional and may or may not be supported by an engine.

The ITTSNotifySink interface is supported by an application. It has these member functions:

AttribChanged

AudioStart

AudioStop

Visual


ITTSNotifySink::AttribChanged

HRESULT AttribChanged(DWORD dwAttribute);

Notifies an application that an engine attribute has changed. This notification is sent because in future versions, multiple processes or applications may be using the same text-to-speech engine through a text-to-speech sharing object (similar to the speech-recognition sharing object).

· Return value is ignored.

dwAttribute

[in] Attribute that was changed. This parameter can be one of these values:

Value | Meaning
TTSNSAC_PITCH | The baseline pitch for the voice
TTSNSAC_REALTIME | The percentage of processor time that the engine expects to use during constant speech
TTSNSAC_SPEED | The baseline average speed for the voice
TTSNSAC_VOLUME | The baseline volume for the voice


ITTSNotifySink::AudioStart

HRESULT AudioStart(QWORD qTimeStamp);

Notifies an application when audio data starts playing. Audio may not start playing until several milliseconds after a buffer is placed while the engine computes the first fraction of a second of audio.

· Return value is ignored.

qTimeStamp

[in] Time, in bytes, when the audio data starts playing. An application can ignore this parameter.

If the engine does not actually send out audio information (for example, if the ITTSCentral::TextData member function is called with only a tag and no other characters or spaces), the ITTSNotifySink interface's AudioStart and AudioStop member functions are not called. This allows an application to change the state of an engine without sending output. The exception to this behavior is the \pau=number\ tag, which actually sends out silence.


ITTSNotifySink::AudioStop

HRESULT AudioStop(QWORD qTimeStamp);

Notifies an application when audio data stops playing.

· Return value is ignored.

qTimeStamp

[in] Time, in bytes, when the audio data stops playing. An application can ignore this parameter.

If the engine does not actually send out audio information, AudioStop is not called. For more information, see the description of the ITTSNotifySink::AudioStart member function.


ITTSNotifySink::Visual

HRESULT Visual(QWORD qTimeStamp, TCHAR cIPAPhoneme,
TCHAR cEnginePhoneme, DWORD dwHints, PTTSMOUTH pTTSMouth);

Notifies an application which phoneme is being used in the current digital-audio stream.

· Return value is ignored.

qTimeStamp

[in] Time, in bytes, when the phoneme starts. An application can ignore this parameter.

cIPAPhoneme

[in] IPA phoneme that is being played. To get the IPA phoneme character, you must use the Unicode ('W') version. This parameter is always NULL for the ANSI ('A') version of this function.

cEnginePhoneme

[in] Engine-specific phoneme that is being played.

dwHints

[in] Kind of sentence in which the phoneme is being played. This parameter can be one of these values:

Value | Meaning
TTSNSHINT_COMMAND | The phoneme is part of a command.
TTSNSHINT_EMPHASIS | The phoneme is part of a word being emphasized.
TTSNSHINT_EXCLAMATION | The phoneme is part of an exclamation.
TTSNSHINT_QUESTION | The phoneme is part of a question.
TTSNSHINT_STATEMENT | The phoneme is part of a statement.

pTTSMouth

[in] Address of a TTSMOUTH structure that contains information about the mouth position corresponding to the current phoneme. If the mouth position is unknown or not supported, this parameter is NULL.

An application can use the information returned by Visual to synchronize character animation with spoken text.


Reference

This section describes, in alphabetical order, the OLE component objects that support text-to-speech, the interfaces and member functions provided by the objects, and the structures used with the interfaces and member functions.


Structures

Some text-to-speech member functions use structures at run time to store and retrieve information about a grammar or text-to-speech mode. This section describes, in alphabetical order, the structures used by text-to-speech member functions.


TTSMODEINFO

typedef struct { // ttsmi
GUID gEngineID;
TCHAR szMfgName[TTSI_NAMELEN];
TCHAR szProductName[TTSI_NAMELEN];
GUID gModeID;
TCHAR szModeName[TTSI_NAMELEN];
LANGUAGE language;
TCHAR szSpeaker[TTSI_NAMELEN];
TCHAR szStyle[TTSI_STYLELEN];
WORD wGender;
WORD wAge;
DWORD dwFeatures;
DWORD dwInterfaces;
DWORD dwEngineFeatures;
} TTSMODEINFO, *PTTSMODEINFO;

Provides detailed information about a text-to-speech mode supported by a particular engine.

gEngineID

Globally unique identifier (GUID) that uniquely identifies the engine. The GUID can be generated randomly using GUIDGEN.EXE, which is provided with the OLE Software Development Kit (SDK). It is possible for more than one mode to have the same engine identifier. In this case, the engine is capable of speaking the different languages or voices of the modes.

szMfgName

Name of the manufacturer — for example, "Acme Corporation."

szProductName

Name of the product — for example, "Acme's Talk-O-Matic."

gModeID

GUID that uniquely identifies the mode. These numbers are guaranteed to be unique, so it is not necessary to monitor and record them. In addition, the GUID is not stored or used directly by OLE, so it is not necessary to load it into the registry.

szModeName

Name of the text-to-speech mode — for example, "English Talk-O-Matic" or "Cartoon Voice."

language

LANGUAGE structure that contains the name of the language and dialect.

szSpeaker

Name of the voice, or NULL if the name is not important.

szStyle

Personality of the voice — for example:

· "Business"

· "Casual"

· "Computer"

· "Excited"

· "Singsong"

wGender

Gender of the voice. This member can be one of these values:

Value | Meaning
GENDER_FEMALE | Female
GENDER_MALE | Male
GENDER_NEUTRAL | Not perceptibly male or female

wAge

Age of the voice. This member can be one of these values:

Value | Meaning
TTSAGE_BABY | About 1 year old
TTSAGE_TODDLER | About 3 years old
TTSAGE_CHILD | About 6 years old
TTSAGE_ADOLESCENT | About 14 years old
TTSAGE_ADULT | Between 20 and 60 years old
TTSAGE_ELDERLY | Over 60 years old

dwFeatures

Text-to-speech features that are available. This member can be a combination of these values:

Value | Meaning
TTSFEATURE_ANYWORD | The engine can attempt to read any word.
TTSFEATURE_IPAUNICODE | The engine's ITTSCentral::Phoneme function supports the CHARSET_IPAPHONETIC voice character set and will return phonemic representations using the International Phonetic Alphabet (IPA) in Unicode. Note that the Unicode version of Phoneme must be called, even if this flag is set.
TTSFEATURE_PCOPTIMIZED | The voice is optimized to work over the computer, which provides better sound quality.
TTSFEATURE_PHONEOPTIMIZED | The voice is designed to work over the telephone and uses an 8-kilohertz (kHz) sampling rate.
TTSFEATURE_PITCH | The engine can adjust the pitch of the voice while it is speaking.
TTSFEATURE_SPEED | The engine can adjust the speed of the voice while it is speaking.
TTSFEATURE_TAGGED | The engine can interpret tagged text to control the voice.
TTSFEATURE_VISUAL | The engine can provide mouth position information for the voice.
TTSFEATURE_VOLUME | The engine can adjust the volume of the voice.
TTSFEATURE_WORDPOSITION | The engine sends notification of word position.

dwInterfaces

Interfaces supported by the engine. This member can be a combination of these values:

Value | Meaning
TTSI_ILEXPRONOUNCE | ILexPronounce interface
TTSI_ITTSATTRIBUTES | ITTSAttributes interface
TTSI_ITTSCENTRAL | ITTSCentral interface
TTSI_ITTSDIALOGS | ITTSDialogs interface

dwEngineFeatures

Features specific to a particular engine. These values are defined by the engine author. For more information about engine features, see the documentation for the engine.

The features provided by a given engine are defined by the engine author. An application can use the ITTSEnum interface to enumerate the engines to identify which engine is being used in order to determine the features it provides.


TTSMODEINFORANK

typedef struct { // ttsmir
DWORD dwEngineID;
DWORD dwMfgName;
DWORD dwProductName;
DWORD dwModeID;
DWORD dwModeName;
DWORD dwLanguage;
DWORD dwDialect;
DWORD dwSpeaker;
DWORD dwStyle;
DWORD dwGender;
DWORD dwAge;
DWORD dwFeatures;
DWORD dwInterfaces;
DWORD dwEngineFeatures;
} TTSMODEINFORANK, *PTTSMODEINFORANK;

Ranks the members of the TTSMODEINFO structure in order of importance.

dwEngineID

Relative importance of a matching engine identifier.

dwMfgName

Relative importance of the specified manufacturer name.

dwProductName

Relative importance of the specified product name.

dwModeID

Relative importance of a matching mode identifier.

dwModeName

Relative importance of the specified mode name.

dwLanguage

Relative importance of the specified language.

dwDialect

Relative importance of the specified dialect. This value is stored in the LANGUAGE structure of TTSMODEINFO.

dwSpeaker

Relative importance of the requested speaker.

dwStyle

Relative importance of the requested speaking style.

dwGender

Relative importance of the requested gender.

dwAge

Relative importance of the requested age. The text-to-speech mode that is closest to the requested age is selected.

dwFeatures

Relative importance of the requested features. Modes that lack the features requested are eliminated from consideration.

dwInterfaces

Relative importance of the requested interfaces. Modes that lack the interfaces requested are eliminated from consideration.

dwEngineFeatures

Relative importance of the requested engine features. Modes that lack the features requested are eliminated from consideration.

Each member of the TTSMODEINFORANK structure contains a priority value for the corresponding feature. Values are relative to each other — that is, a higher value indicates a higher priority, but the value itself is not meaningful.

The ITTSFind::Find member function searches for the text-to-speech mode that most closely matches the attributes specified in TTSMODEINFO, using the rankings of TTSMODEINFORANK to select the mode with the most desirable features.


TTSMOUTH

typedef struct { // ttsm
BYTE bMouthHeight;
BYTE bMouthWidth;
BYTE bMouthUpturn;
BYTE bJawOpen;
BYTE bTeethUpperVisible;
BYTE bTeethLowerVisible;
BYTE bTonguePosn;
BYTE bLipTension;
} TTSMOUTH, *PTTSMOUTH;

Indicates the mouth position that corresponds to a phoneme.

bMouthHeight

Height of the mouth or lips. This is a linear range from 0xFF for the maximum possible height for the mouth to 0x00 for the minimum height (that is, the mouth or lips are closed).

bMouthWidth

Width of the mouth or lips. This is a linear range from 0xFF for the maximum possible width for the face to 0x00 for the minimum width (that is, the mouth or lips are puckered).

bMouthUpturn

Extent to which the mouth turns up at the corners (that is, how much it smiles). This is a linear range from 0xFF for the maximum upturn (that is, the mouth is fully smiling) to 0x00 if the corners of the mouth turn down. If this member is 0x80, the mouth is neutral.

bJawOpen

Angle to which the jaw is open. This is a linear range from 0xFF for completely open to 0x00 for completely closed.

bTeethUpperVisible

Extent to which the upper teeth are visible. This is a linear range from 0xFF for the maximum extent (that is, the upper teeth and gums are completely exposed) to 0x00 for the minimum (the upper teeth are completely hidden). If this member is 0x80, only the teeth are visible.

bTeethLowerVisible

Extent to which the lower teeth are visible. This is a linear range from 0xFF for the maximum extent (that is, the lower teeth and gums are completely exposed) to 0x00 for the minimum (the lower teeth are completely hidden.) If this member is 0x80, only the teeth are visible.

bTonguePosn

Tongue position. This a linear range from 0xFF if the tongue is against the upper teeth, to 0x00 if it is relaxed. If this member is 0x80, the tongue is visible.

bLipTension

Lip tension. This is a linear range from 0xFF if the lips are very tense to 0x00 if they are completely relaxed.

TTSMOUTH is passed by the IVTxtNotifySink::Visual and ITTSNotifySink::Visual member functions along with other information about the phoneme. An application can use this information for graphical animation of spoken text.


Text-to-Speech Error Values

The following errors may be returned by member functions of the text-to-speech objects. If no error occurs, the member functions return NOERROR.

Error value | Description
<<TTSERR_INVALIDCHAR> | The text contains an invalid character.
<<TTSERR_INVALIDINTERFACE> | The interface is invalid.
<<TTSERR_INVALIDKEY> | The registration key is invalid.
<<TTSERR_INVALIDMODE> | The speech mode is invalid.
<<TTSERR_INVALIDWINDOW> | The window is invalid.
<<TTSERR_NOTSUPPORTED> | The function is not supported.
<<TTSERR_OUTOFDISK> | There is no more disk space.
<<TTSERR_QUEUEFULL> | The text queue is full. This error is returned when the application fails to wait for a ITTSBufNotifySink::TextDataDone notification before calling TextData, and the engine's queue is full.
<<TTSERR_VALUEOUTOFRANGE> | The value is out of range.
<<TTSERR_WAVEDEVICEBUSY> | The wave device is busy.
<<TTSERR_WAVEFORMATNOTSUPPORTED>
 | No wave formats are supported.

In addition to the error values described in the preceding list, member functions can return error values generated by lower-level component objects or calls to the Win32 API. For example, a member function of the ITTSEnum interface might return an AUDERR_ value generated by the audio-destination object. For more information, see "Object Levels and Error Values."


Text-to-Speech Control Tags

A text-to-speech engine can usually translate individual words to speech successfully. However, as soon as the engine speaks a sentence, the perceived quality of its translation decreases because the engine cannot correctly synthesize human prosody — the inflection, accent, and timing of human speech.

The prosody of translated speech can be improved by using text-to-speech control tags to better simulate human speech. Control tags can be embedded in the source text passed to an engine with the ITTSCentral::TextData member function, or they can be inserted into the text that is currently playing by calling the ITTSCentral::Inject member function. This allows an application to alter prosody of text as it is spoken, without having to reconstruct the text-to-speech buffers.

This appendix describes control tags that you can use to alter the prosody of text translated into speech. All tags are optional except the bookmark (Mrk) tag, which must be supported.


Syntax

Text-to-speech control tags follow these general rules of syntax:

· All tags begin and end with a backslash character (\).

· The backslash character is not allowed within a tag.

· An odd number of backslash characters in tagged text produce undefined behavior in the engine.

· Tags are case-insensitive. For example, \vce\ is the same as \VCE\.

· Tags are white-space – dependent. For example, \Rst\ is not the same as \ Rst \.

To include a backslash character in tagged text, but outside a tag, use a double backslash (\\).

When the text-to-speech engine encounters a tag it does not understand, the tag should be ignored.

Tags are persistent from one call to the ITTSCentral::TextData member function to another. For example, if the \ctx="e-mail"\ tag is passed to an engine that supports that tag, the engine stays in the "e-mail" context until another tag changes the context.


Conventions

This appendix uses the following typographic conventions.

Example | Description
Chr | Bold type indicates speech-inflection keywords.
string | Italic indicates placeholders for information you supply, such as a character or context string.
[[option]] | Double square brackets indicate items that are optional.
[[option...]] | Three dots (an ellipsis) following an item indicate that more items having the same form may appear.
"C" | Quotation marks are required to delimit strings.


Reference

This section describes, in alphabetical order, the text-to-speech control tags that can be embedded in the source text to improve the prosody of text-to-speech translation.


Chr

\Chr=string[[,string...]]\

Sets the character of the voice.

· Example: \Chr="Angry"\

string

String that specifies the characteristics of the voice.

The default characteristic is "Normal," which produces a normal tone of voice. Although the Chr tag is less specific than setting the inflection, stress, attack, and whispering qualities individually, it is easier to use and allows the engine more flexibility and intelligence in its response.

Several characteristics can be specified in the same tag, separated by commas. Depending on its capabilities, an engine may not support all of the characteristics listed here, or it may support additional characteristics. Some common characteristics are the following:

"Angry"

"Business"

"Calm"

"Depressed"

"Excited"

"Falsetto"

"Happy"

"Loud"

"Monotone"

"Perky"

"Quiet"

"Sarcastic"

"Scared"

"Shout"

"Tense"

"Whisper"


Com

\Com=string\

Embeds a comment in the text. Comments are not translated into speech.

· Example: \com="This is a comment."\

string

Text of the comment.


Ctx

\Ctx=string\

Sets the context for the text that follows, which determines how symbols are spoken.

· Example: \ctx="Unknown"\

string

String that specifies the context. This parameter can be one of these strings:

Context string | Description
"Address" | Addresses and/or phone numbers
"C" | Code in the C or C++ programming language
"Document" | Text document
"E-Mail" | Electronic mail
"Numbers" | Numbers, dates, times, and so on
"Spreadsheet" | Spreadsheet document
"Unknown" | Context is unknown (default)
"Normal" | Normal/default speech mode.


Emp

\Emp\

Emphasizes the next word to be spoken.

· Example: "... the \Emp\truth, the \Emp\whole truth, and nothing \Emp\but the truth."


Eng

\Eng;GUID:command\
\Eng:command\

Embeds an engine-specific command that affects only the engine with the specified globally unique identifier (GUID). Subsequent engine-specific tags for that engine can omit the GUID until an engine-specific tag for a different engine is used.

GUID

Globally unique identifier of the engine.

command

Engine-specific command.


Mrk

\Mrk=number\

Indicates a bookmark in the text.

· Example: \Mrk=75000\

number

Number of the bookmark.

When the text-to-speech engine encounters this tag, it notifies the application by calling the ITTSBufNotifySink::BookMark member function.

Bookmarks have a DWORD range (specified in the dwMarkNum parameter of the ITTSBufNotifySink::BookMark member function), and the Mrk tag accepts a decimal representation. Bookmark number zero (\Mrk=0\) is reserved; a ITTSBufNotifySink::BookMark member function is not sent for bookmark number zero.

Bookmark tags are inserted directly into the text sent to the engine when calling the ITTSCentral::TextData member function.


Pau

\Pau=number\

Pauses speech for the specified number of milliseconds.

· Example: \Pau=1000\

number

Number of milliseconds to pause.


Pit

\Pit=number\

Sets the baseline pitch of the text-to-speech mode to the specified value in hertz.

· Example: \Pit=90\

number

Pitch, in hertz.

The actual pitch fluctuates above and below this baseline. Embedding a Pit tag in text is the same as calling the ITTSAttributes::PitchSet member function.


Prn

\Prn=text=pronunciation\

Indicates how to pronounce text by passing the phonetic equivalent to the engine.

· Example: \Prn=tomato=tomaato\

text

Text to pronounce.

pronunciation

Phonetic equivalent for pronouncing the text.

Using the Prn tag without specifying the word pronunciation (\Prn=text\) will undo the changes to the pronunciation; that is, it will return the word to the original pronunciation.

This tag is valid only for engines that support the International Phonetic Alphabet. The engine should continue to use this pronunciation for the current text-to-speech mode and should store the pronunciation in its lexicon for later use. If a lexicon entry already exists for a particular word, the Prn tag should be ignored for that word.


Pro

\Pro=number\

Activates and deactivates prosodic rules, which affect pitch, speaking rate, and volume of words independently of control tags embedded in the text. Prosodic rules are applied by the engine.

· Example: \Pro=0\

number

Number that indicates whether to activate or deactivate prosodic rules. Setting number to 1, the default, activates prosodic rules. Setting number to 0 deactivates prosodic rules.

Prosody does not have a corresponding ITTSAttributes interface as speed and pitch do. If the engine supports control of prosody, use the Pro tag.


Prt

\Prt=string\

Indicates the part of speech of the next word.

· Example: \prt="Abbr"\

string

String that indicates the part of speech. This parameter can be one of these strings:

String | Description | String | Description
"Abbr" | Abbreviation | "N" | Noun
"Adj" | Adjective | "Ord" | Ordinal number
"Adv" | Adverb | "Prep" | Preposition
"Card" | Cardinal number | "Pron" | Pronoun
"Conj" | Conjunction | "Prop" | Proper noun
"Cont" | Contraction | "Punct" | Punctuation
"Det" | Determiner | "Quant" | Quantifier
"Interj" | Interjection | "V" | Verb

For more information about the parts of speech, see the VOICEPARTOFSPEECH enumeration in the section, "Common Elements of the Speech Application Programming Interface."


Rst

\Rst\

Resets all control tags to the engine's default settings.


Spd

\Spd=number\

Sets the baseline average talking speed of the text-to-speech mode to the specified number of words per minute.

· Example: \Spd=120\

number

Baseline average talking speed, in words per minute.

Embedding a Spd tag in text is the same as calling the ITTSAttributes::SpeedSet member function.


Vce

\Vce=charact=value[[,charact=value]]\

Instructs the engine to change its speaking voice to one that has the specified characteristics.

charact

One of the characteristics listed in the table below.

value

String that specifies the characteristic.

Characteristics are specified in the order of importance. The engine selects a voice that most closely matches the characteristics specified at the beginning of the list.

The charact=value argument can be any of the following:

· Language=language. Requests the engine speak in the specified language

· Accent=accent. Requests the engine speak in the specified accent. For example, if Language="English" and Accent="French" the engine will speak English with a French accent.

· Dialect=dialect. Requests the engine speak in the specified dialect

· Gender=gender. Specifies the gender of the voice: "Male", "Female", or "Neutral".

· Speaker=speakername. Specifies the name of the voice, or NULL if the name is unimportant

· Age=age. Specifies the age of the voice, which can be one of the following values:

Age string | Description
"Baby" | About 1 year old
"Toddler" | About 3 years old
"Child" | About 6 years old
"Adolescent" | About 14 years old
"Adult" | Between 20 and 60 years old
"Elderly" | Over 60 years old

· Style=style. The personality of the voice — for example: "Business", "Casual", "Computer", "Excited", or "Singsong".


Vol

\Vol=number\

Sets the baseline speaking volume for the text-to-speech mode.

· Example: \Vol=32768\

number

Baseline speaking volume.

The volume level is a linear range from 0 for absolute silence to 65535 for maximum volume. The default is 65535. Embedding a Vol tag in text is the same as calling the ITTSAttributes::VolumeSet member function.


Why Use the Low-Level API?

The high level speech API (Voice Commands, Voice Text, and Voice Dictation) exist to make speech easy to use, but in the process prevent full control of speech recognition. In the low level API:

The application must pay attention to many more details than with the high level API.

The application can do anything that it wants to so long as the engine can support the feature.

If standard low-level API doesn't provide for a feature, then engines can add onto API.

The engine is in application's process space.

Multiple instances of engines can be run at once.

Audio comes from or goes to any media that the application desire.


Overview of Speech-Recognition Objects

To perform speech recognition, an application needs an audio-source object to provide speech input, an engine object to process the speech, and a grammar object to provide the engine with the lists of words or rules that determine what the engine can recognize. As an alternative to creating its own engine object and audio-source object, an application can use a speech-recognition sharing object to use an engine and an audio source that are shared with other applications.

An application can use the speech-recognition enumerator and engine enumerator to locate a speech-recognition mode and create an engine object. As soon as the application has the engines it needs, it can release these objects.

If the engine provides a speech-recognition results object, an application can use this object to analyze a recognition, reevaluate or correct the result, or show feedback to the user.

The following illustration shows the sequence in which an application typically creates speech-recognition objects. The illustration includes the names of the interfaces and member functions used to create the objects — the interfaces and member functions are described in subsequent sections of this section.

After the application has created the necessary objects, it can use them to recognize speech. The following illustration shows how the application interacts with the engine, grammar, and audio-source objects to recognize a phrase.


Audio-Source Object

Before an application can use speech recognition, it must determine the origin of the audio to be recognized. Typical audio sources include a microphone, a telephone line, and a file.

To recognize audio from a wave-in device, the application can create an instance of the multimedia audio-source object, which is an OLE Component Object Model (COM) object that supports audio communication interfaces in common with the engine. To use audio from a different source (such as a file), or to manipulate the audio before it gets to the recognition engine, an application can use a custom audio-source object that you provide or one provided by a third-party vendor.


Multimedia Audio-Source Object

Most applications recognize audio from a multimedia wave-in driver. Microsoft provides a component object called the multimedia audio-source object that provides access to the multimedia wave-in driver. To use the multimedia audio-source object, an application calls the CoCreateInstance function with the CLSID_MMAudioSource class identifier. The function returns the address of the IUnknown interface supported by the audio-source object. The application needs to specify this address when creating an engine object (described later in this section).

By default, the multimedia audio-source object uses WAVE_MAPPER as the audio device. To use a different device, the application can call the IUnknown::QueryInterface member function with the IID_IAudioMultiMediaDevice interface identifier. QueryInterface returns the address of the IAudioMultimediaDevice interface. The application can then call the IAudioMultimediaDevice::DeviceNumSet member function to set the identifier of the multimedia device to use as the audio source.

An application can retrieve a list of wave-in audio devices available in the system by using the WaveInGetNumDevs and WaveInGetDevCaps functions.

After acquiring the address of the IAudioMultimediaDevice interface, an application must call the Release member function before it closes to release the audio-source object.

As an alternative to using an engine object and multimedia audio-source object, an application can use the speech-recognition sharing object to use an engine and an audio source that are shared with other applications. For more information about the speech-recognition sharing object, see "Speech-Recognition Sharing Object" later in this section.


Custom Audio-Source Object

A custom audio source can allow the application to do the following:

· Recognize speech from an audio file — for example, to transcribe a recording of a meeting or telephone conversation.

· Share incoming audio with another task. (The multimedia audio-source object does not support sharing a waveform-audio device.)

· Recognize audio from hardware that does not support a multimedia driver.

To write an audio-source object, you would code the object's OLE COM framework and support three interfaces: IUnknown, IAudio, and IAudioSource. You might also define a custom interface that allows the application to control the audio source, such as an IAudioFile interface to specify which file to recognize from.

When an engine first receives the IUnknown for the audio source (see "Engine Selection" later in this section), it calls the QueryInterface member function for a variety of interfaces, such as those that control specific hardware. If the engine cannot find any of these custom interfaces, it eventually tries the IAudio and IAudioSource interfaces. The engine then calls the IAudio::WaveFormatSet member function with several different wave formats, starting with custom formats and working its way down to pulse-code modulation (PCM) if none of the earlier formats is supported. (The application must determine which formats are acceptable.) If the engine cannot find any interfaces or formats that it can use, it fails to create an engine object.

After the engine selects a wave format, it generates the engine object and returns from the call to the ISRFind::Select or ISREnum::Select member function. As the application calls functions on the engine object, that object calls functions on the audio source to get the needed digital audio, such as IAudio::Start, IAudioSource::DataGet, and so on.


Engine Object

After an application has created an audio-source object, its next task is to find a suitable engine and speech-recognition mode, and then create an instance of the engine object to represent the mode. An engine typically provides an assortment of speech-recognition modes that can be used to recognize speech in different languages, dialects, or audio sampling rates. The engine object is supplied by the engine vendor.

An application finds a suitable engine and speech-recognition mode by creating a speech-recognition enumerator and using it to find a mode and engine that meet the application's criteria. An application can also enumerate all available modes and select an engine and mode from the list.


Speech-Recognition Enumerator

To create a speech-recognition enumerator, an application calls the CoCreateInstance function with the CLSID_SREnumerator class identifier and the IID_ISRFind interface identifier. This creates a speech-recognition enumeration object with an ISRFind interface that the application can use to find an engine.

After creating the speech-recognition enumerator and getting the address of its ISRFind interface, the application fills an SRMODEINFO structure with values that indicate the kind of engine the application is looking for, such as the language, the dialect, the engine features needed, the engine name, and so on. Any members of the SRMODEINFO structure that are not relevant to the application can be set to NULL. The application then calls the ISRFind::Find member function.

The Find member function, if it succeeds, fills a second SRMODEINFO structure with information about the closest matching speech-recognition mode available. Because this is only the closest match, the application should check the mode to see if it is acceptable. If not, the application can enumerate all of the engines to try again to find an acceptable mode (described in the next section), or the application can fail.

An application can rank certain characteristics more or less important than the engine's default rankings by filling an SRMODEINFORANK structure that indicates the priority of each member of the SRMODEINFO structure. The application specifies the address of the SRMODEINFORANK structure along with the SRMODEINFO structures when calling the Find member function.


Engine Enumerator

If an application is not satisfied with the result of the ISRFind::Find member function, or if the application requires a list of the available speech-recognition modes, it should retrieve the address of the ISREnum interface on the speech-recognition enumerator object.

If the application has already created the speech-recognition enumerator and has the address of the ISRFind interface, it can call the QueryInterface member function of ISRFind with the IID_ISREnum interface identifier.

If the application has not yet created the speech-recognition enumerator, it can call the CoCreateInstance function with the CLSID_SREnumerator class identifier and the IID_ISREnum interface identifier. This returns the address of the enumerator's ISREnum interface, which includes the standard member functions of the IEnumX interface: Next, Clone, Skip, and Reset. The application can use these functions to examine the SRMODEINFO structures for all of the speech-recognition engines and modes installed on the system.


Engine Selection

After an application has identified a speech-recognition mode, it calls the Select member function to select the mode and create an engine object to represent that mode in the application. Both the ISRFind and ISREnum interfaces include the Select member function, and the syntax is the same for both.

When calling Select, the application specifies a globally unique identifier (GUID) that identifies the mode to use, the address of a variable that receives the address of the ISRCentral interface on the engine object, and the address of the IUnknown interface on the audio-source object created earlier (see "Introduction" earlier in this section).

When the Select member function is called, it ultimately creates an engine object. However, as the engine object initializes, it calls the IAudio interface on the audio-source object to determine whether or not the engine can use that audio source. If so, the engine object calls the IAudio::PassNotify member function, specifying the address of an IAudioSourceNotifySink interface so the audio-source object can notify the engine of events that pertain to incoming audio. If the engine cannot use the specified audio source, the ISREnum::Select member function returns an error and the engine object releases itself.

If successful, the Select member function returns the address of the ISRCentral interface on the newly created engine object. The ISRCentral interface provides centralized control of a speech-recognition engine, including loading a grammar, pausing and resuming the engine, getting information about the engine or the speech being received, and registering a notification sink. The engine uses the notification sink to notify the application of speech-related events.

The ISRCentral interface has the following member functions:

Member function | Description
GrammarLoad | Loads a grammar and returns a grammar object.
ModeGet | Retrieves information about the current speech-recognition mode.
Pause | Pauses the speech-recognition engine.
PosnGet | Retrieves the exact byte in the audio stream that the engine is currently receiving.
Register | Registers an ISRNotifySink interface with the engine, which uses the interface to pass status information to an application.
Resume | Resumes the speech-recognition engine.
ToFileTime | Converts a QWORD variable to a Win32 FILETIME structure.
UnRegister | Releases the ISRNotifySink interface that an engine used to pass status information to an application.

After obtaining the address of the ISRCentral interface on the engine object, the speech-recognition enumerator is no longer needed and the application should release it by calling the ISRFind::Release or ISREnum::Release member function. The application does not need to release the audio-source object because the engine automatically releases it when the engine object is released.


Engine Attributes

A speech-recognition engine has a set of attributes that affect the interaction between the engine and the audio source. An application can query and set a site's attributes by using the ISRAttributes interface provided by the engine object. To get the address of ISRAttributes, call the ISRCentral::QueryInterface member function with the IID_ISRAttributes interface identifier.

Automatic Gain

Many speech-recognition engines can automatically adjust the gain of the incoming audio signal (if the audio device supports it). The gain is the increase in signaling power, measured in decibels (dB), that occurs as a signal is boosted by an electronic device.

An application can use the ISRAttributes::AutoGainEnableSet member function to set the speed with which the engine adjusts the signaling power. When calling AutoGainEnableSet, the application specifies a value from 0 to 100. A value of 0 disables automatic gain, and a value of 100 causes the voice-command object to set the gain to the value for the previous utterance so that if the next utterance is spoken at the same level, the gain is set perfectly. A value between 0 and 100 moderates the automatic adjustments on a linear scale. For example, a value of 50 adjusts the gain to 50 percent of the level for the previous utterance.

When automatic gain is enabled, the speech-recognition engine may adjust the level at the end of an utterance and increase or decrease the gain. The engine adjusts the gain by using the IAudio::LevelGet and IAudio::LevelSet member functions to communicate with the audio-source object that represents the incoming audio stream. If the audio source does not support the IAudio::LevelSet member function, the application cannot set the automatic gain.

An application retrieves the current automatic gain value for a site by using the ISRAttributes::AutoGainEnableGet member function.

Echo Canceling

Echo canceling is a method of controlling echoes on an audio signal, in which the sender checks the inbound channel for a slightly delayed duplicate of its own transmission. In echo canceling, the sender adds an appropriately modified, reversed version of its transmission. The result is to erase the echo electronically but leave incoming data intact.

An application can use the ISRAttributes::EchoSet member function to indicate whether or not the engine should treat the incoming audio signal as though it has residual signal on it. In general, if an application indicates that the audio source uses echo canceling, the speech-recognition engine ignores low-level audio signals so that it does not attempt to recognize the residual signal of the echo cancellation.

An application can use the ISRAttributes::EchoGet member function to determine the current state of echo canceling.

Energy Floor

The energy floor (or noise floor) is the noise value in the signal-to-noise ratio (SNR) for a particular environment. In general, the higher the noise value in the ratio, the more sensitive the speech-recognition engine is to background noise. For example, a quiet office would have a high SNR (low floor), whereas a noisy factory would have a low SNR (high floor).

If an application has information about the SNR that it will receive from the incoming audio stream, it can inform the speech-recognition engine by using the ISRAttributes::EnergyFloorSet member function. The engine can use the value internally to adjust its calculated noise floor in expectation of the audio stream's SNR. The energy floor is specified in negative decibels. For example, a value of 30 indicates an energy floor of –30 dB.

An application uses the EnergyFloorGet function to retrieve the current noise floor expected by the engine.

Microphone

If an audio source uses a microphone as the audio input device, the application can use the ISRAttributes::MicrophoneSet member function to set the name of the microphone for that source. An application can use the microphone name to save or retrieve information about the microphone, such as the type of microphone used to train the engine and the conditions in which the microphone is recording. For example, an application could use this information to preserve the original training when the user changes microphones.

An application uses the ISRAttributes::MicrophoneGet member function to retrieve the microphone name for an audio source.

Real-Time Setting

The real-time setting is the percentage of processor time that the engine developer expects the engine to use during constant speech. For example, if the real-time setting is 100, the engine takes one full minute of processor time to process one minute of speech. If the real-time setting is 50, the engine takes 30 seconds of processor time to process the same minute of speech. This value is difficult to compute precisely, so it should be regarded as an estimate.

The real-time setting can be more than 100 for non-real-time applications (for example, applications that transcribe prerecorded speech). For most engines, the amount of processor time required diminishes markedly during periods of silence.

If an application changes the real-time setting for an engine by calling the ISRAttributes::RealTimeSet member function, the engine attempts to meet the new real-time expectation. However, a real-time setting of 1 is not possible for today's personal computers.

An application retrieves the current real-time setting for an engine by using the RealTimeGet function.

Speaker Training

One way an application can improve the recognition accuracy of an engine is to train the engine to take into account the unique qualities of the user's voice. By using the ISRDialogs interface of the engine object, an application can direct the engine to display its training dialog box.

Typically, an engine's training dialog box displays a sequence of words and phrases that the user must speak into the audio input device. The engine processes the user's spoken input and saves information that helps the engine improve its recognition accuracy for that user. The engine saves the user's name (that is, the speaker's name) along with the user's training information, and it uses the name to load the information whenever that user becomes the speaker for an audio source.

An application changes the speaker name for an audio source by using the ISRAttributes::SpeakerSet member function. Changing the speaker name unloads all training for the previous speaker and loads the training for the new speaker. If no training exists for the new speaker, the application starts with the engine's default training. The ISRAttributes::SpeakerGet member function retrieves the name of the current speaker for an audio source.

Threshold Level

An application sets and queries the threshold level of an audio source by using the ISRAttributes::ThresholdSet and ThresholdGet member functions. The threshold level is a value that indicates the point below which the engine rejects an utterance as unrecognized. A threshold level of 0 indicates that the engine should match any utterance to the closest phrase match. A value of 100 indicates that the engine should determine absolutely that an utterance is the recognized phrase or else reject it.

For example, suppose the engine is expecting "What is the time?" If the threshold is 100 and the user mumbles "What'z tha time" or has a cold, the command may not be recognized. However, if the threshold is too low and the user says a similar-sounding phrase that is not being listened for (such as "What is mine?") the engine may recognize it as "What is the time?"

The engine object indicates whether an utterance is above or below the threshold level when it calls the ISRGramNotifySink interface to inform the application of the utterance. For more information about the ISRGramNotifySink interface, see "Grammar-Specific Notifications" later in this section.

Time-out Values

A speech-recognition engine has two time-out values associated with it. The first, called the incomplete-phrase value, is the number of milliseconds that the speech-recognition engine waits before discarding an incomplete phrase because the user has stopped speaking. For example, if the incomplete-phrase value is 2000, a user speaking "Send mail to " could pause for two seconds before the engine would assume that the user has stopped speaking the phrase.

The second time-out value, called the complete-phrase value, is the number of milliseconds that the engine waits before regarding a phrase as complete after the user has stopped speaking. For example, if the complete-phrase value is 500, a user speaking "Send mail to Fred" would see results one-half second after he or she finished the phrase.

An application sets and queries the time-out values by using the ISRAttributes::TimeOutSet and TimeOutGet member functions.


Speaker Profiles

A speech-recognition engine can collect information from a speaker to improve its recognition accuracy for that speaker, and the engine can store the information in an object called a speaker profile. Storing speaker information in a profile allows an application to transfer speaker information from one system to another, saving the speaker from having to perform training again when switching to another system. Using speaker profiles is extremely useful for dictation systems, which often require a lot of training, and speech servers, which need to share training. For command and control applications, the effort of transferring a speaker profile may outweigh the benefits.

A speaker profile includes all of the information that the engine has collected about a particular speaker, including:

· A data header to ensure that the application isn't passing information to the wrong engine

· The languages for which training has been done

· The known patterns of speech and the language model

· The pronunciation of specific words

· The phonetic training for the speaker

· The identifier of the speaker

· The speaker's preferences

· The number of hours of training that the speaker accomplished to create the profile (useful for merging speakers)

Speaker profiles are optional — an engine may or may not support them. An application determines if the engine supports profiles by calling the QueryInterface member function with the IID_ISRSpeaker interface identifier. If the engine supports speaker profiles, QueryInterface returns the address of the engine object's ISRSpeaker interface.

An application uses the ISRSpeaker interface to manipulate speaker profiles. The ISRSpeaker::Read member function retrieves the profile for a given speaker and copies it into a buffer. The Write member function writes a speaker profile from another system into the current system, and the Merge member function merges a speaker profile from another system into the given profile on the current system.

An application can retrieve the name of the current speaker by using the ISRSpeaker::Query member function. To enumerate the names of all speakers known to the speech-recognition engine, and application can use the Enum member function. An application can select a speaker to be the current speaker in the system by using the Select member function.

An application uses the ISRSpeaker::New member function to create a new profile for a speaker. The new profile initially contains only the default training information supplied by the engine. The engine adds additional information as it completes training for the speaker. An application can use the Revert member function to restore the profile to its original state. For example, an application can use Revert to cancel undesirable changes such as those caused by extraneous noise or unrelated conversations during a training session. An application can delete a speaker profile by using the Delete member function.


Dialog Boxes

A speech-recognition engine provides a set of dialog boxes that let the user configure the engine. By using the ISRDialogs interface provided by a speech-recognition engine object, an application can direct the engine to display one of its dialog boxes. The controls and capabilities in the dialog boxes are determined solely by the engine.

When an application calls a member function of the ISRDialogs interface to display a dialog box, it specifies the handle of the parent window for the dialog box. By specifying NULL instead of a window handle, the application can determine whether the engine supports the given dialog box. The engine returns NOERROR if it supports the dialog box, or SRERR_NOTSUPPORTED if not.

The application also specifies the title of the dialog box by specifying the address of a title string. If the application specifies NULL instead of a title string address, the engine uses the default title.

About Dialog Box

An engine's About dialog box contains information that identifies the speech-recognition engine and its manufacturer, and it contains the manufacturer's copyright notice. An application displays the About dialog box by calling the ISRDialogs::AboutDlg member function.

General Dialog Box

The General dialog box gives the user control of the general features of the speech-recognition engine. An application displays the General dialog box by using the ISRDialogs::GeneralDlg member function. Depending on the engine, the General dialog box may allow the user to do the following:

· Inform the engine of the speaker's gender or other speaker-specific information

· Control how stringent the engine is about recognition

· Control the maximum amount of disk space allotted for speaker information

· Control optimizations that affect speech processing time

· Export and import speaker information to and from a disk so that the information can be transferred between systems

Lexicon Dialog Box

A speech-recognition engine maintains a database of pronunciations called a pronunciation lexicon, and an engine may allow an application to collect new or corrected pronunciations from the user. The Lexicon dialog box displays a user's pronunciation lexicon and allows the user to edit it. For example, the user can edit the phonetics of mispronounced words and load or unload personal exception dictionaries. An application uses the ISRDialogs::LexiconDlg member function to display the Lexicon dialog box.

For more information about pronunciation lexicons, see the section, "Common Elements of the Speech Application Programming Interface."

General Training Dialog Box

The ISRDialogs::TrainGeneralDlg member function displays the General Training dialog box, which trains a speech-recognition engine by having the user speak a preselected set of words and phrases. After the user speaks the words and phrases, the recognition accuracy should improve for that speaker.

Microphone Training Dialog Box

The Microphone Training dialog box trains the engine for use with a particular microphone so that recognition accuracy improves with that microphone. An application should display this dialog box at the request of the engine or whenever the speaker wants to display it. Training information is associated with both the speaker and the microphone. If the engine does not supply this dialog box, the speaker cannot train it for the microphone; however, the application will still work. The application uses the ISRDialogs::TrainMicDlg member function to display the Microphone Training dialog box.


Notification Interface

Many engines provide notifications that are independent of the grammars that happen to be active, although the notifications are sent only when at least one grammar is active. These notifications include such things as when an utterance begins or ends, the Volume Units (VU) Meter level, and the background noise conditions.

To receive these messages, the application must create a COM object that supports the ISRNotifySink interface. The application's response to notifications is up to you.

Before the engine object can use the notification sink, the application must register the sink with the engine by calling the ISRCentral::Register member function, specifying the address of the ISRNotifySink interface, the IID_ISRNotifySink interface identifier, and the address of a double-word variable. If the engine does not support the interface, Register returns an error. If Register succeeds, the engine fills the double-word variable with a notification identifier that the application can use to unregister the notification sink.

During the time that the application has a notification sink registered with the engine, it receives the following notifications:

Member function | Description
AttribChanged | Called when an application changes an attribute of the speech-recognition engine.
Interference | Called when the engine cannot properly recognize an utterance for a known reason. The notification includes a flag indicating the reason.
Sound | Called when the engine recognized a sound or a garbled word.
UtteranceBegin | Called as soon as the speech-recognition engine has received any audio input that might be speech. However, the engine may not begin processing the speech for a few more seconds if it is already processing a previous utterance. The notification is tagged with a QWORD that identifies the time the speech started. An application can pass this into the ISRCentral::ToFileTime member function to determine the time (hours, minutes, and seconds) that the event occurred.
UtteranceEnd | Called as soon as the speech-recognition engine senses that the user has stopped speaking. The notification has both a beginning time and ending time associated with it. Although many engines associate an utterance with a recognition result, this may not always be the case. Several utterances can occur before recognition results are returned, or several results can be returned for one utterance.
VUMeter | A linear VU level from 0 to 0xFFFF, sent between 8 and 16 times a second, which indicates the loudness of the digital-audio stream. This information can be used to provide feedback to the user that the engine object is listening. Not all engines call this member function.

An application can unregister a notification sink by calling the ISRCentral::UnRegister member function, specifying the identifier received in the call to the Register member function. Calling UnRegister is optional because if an application releases the engine object, it releases all of the notification sinks it is using. However, the release of the notification sink may not occur immediately after the engine is released.


Releasing an Engine Object and Shutting Down

When an application finishes using speech recognition, it should release all of the outstanding interfaces that it has on speech-recognition objects. This includes those from the engine object, all grammar objects, and all results objects.

Once all of these objects are free, the engine begins shutting down. The engine releases all notification sink objects from the application.

The application should not shut down until all of its notification sinks (for both ISRGramNotifySink and ISRNotifySink) have been released. This is very important because speech recognition is an asynchronous process, possibly running in a different process space.


Grammar Object

Before an engine can begin recognizing speech, the application must inform the engine what it expects the user to speak. To do this, the application uses one of the following kinds of grammars:

· A context-free grammar uses rules that predict the next words that might possibly follow the word just spoken, reducing the number of candidates to evaluate in order to recognize the next word.

· A dictation grammar defines a context for the speaker by identifying the subject of the dictation, the expected style of language, and what dictation has already been done. Not all engines support dictation grammars.

· A limited-domain grammar does not provide strict syntax structures, but does provide a set of words to recognize. A limited-domain grammar is a hybrid between a context-free grammar and a full dictation grammar. Not all engines support limited-domain grammars.

For purposes of the following discussion, the application is using a context-free grammar for command and control.


Generating a Context-Free Grammar

The easiest way to generate a context-free grammar is to use a context-free grammar (CFG) compiler that accepts a text file containing the rules and words used and compiles it into the standard binary format that is sent to the engine. Such compilers can be obtained from engine vendors or third-party suppliers.

If an application does not have access to a CFG compiler, or if the grammar must be constructed programmatically because you cannot predict its contents, the application must deal with the binary format directly.

To use the binary format, the application must determine the following information:

· The words in the grammar and the unique identifiers for each word

· The rules that define the syntax of the grammar

· The rules that are exported so they can be individually activated or linked with other grammars

· If rules refer to lists, the list names and identifiers

· The rules that are imported from other grammars

· The human language for which the grammar is designed

The application constructs this information by filling a block of memory with appropriate structures, such as SRWORD, SRCFGRULE, SRCFGXRULE, SRCFGIMPRULE, SRCFGLIST, and LANGUAGE. An application must combine these structures into a series of "chunks" that identify the data in the grammar, and then write the chunks into a sequential block of memory or to a memory-mapped file.

For more information about grammar formats and structures, see Appendix B, "Standard Grammar Formats."


Grammar Object Creation

After the application loads the grammar description into memory (or into a memory-mapped file), it loads the grammar by using the ISRCentral::GrammarLoad member function. When calling GrammarLoad, the application indicates the type of the grammar (context-free, dictation, or limited-domain) and its format (standard or engine-native). The application also specifies an SDATA structure that contains the address of the beginning of the grammar memory, and a value that indicates the number of bytes in the grammar data.

Optionally, an application can specify the address of an ISRGramNotifySink interface that the grammar object uses to notify the application of recognition events, and to pass information about the event to the application. To receive recognition notifications, an application must create a COM object that supports the ISRGramNotifySink interface. When a recognition event occurs, the grammar object notifies the application by calling the appropriate member function of the ISRGramNotifySink interface. You determine how the application responds to notifications. For more information about the ISRGramNotifySink interface, see "Grammar-Specific Notifications" later in this section.

When the application calls the GrammarLoad member function, the engine compiles the binary format into an internal format that the engine understands. Compiling the grammar may take some time, perhaps several seconds. If the grammar uses any features that the mode cannot support, such as links and lists, the GrammarLoad member function returns an error.

After the engine compiles the grammar into its own format, it creates a grammar object used to control the grammar. The grammar object supports several interfaces, including IUnknown and ISRGramCommon. Grammar objects attached to context-free grammars also support the ISRGramCFG interface.

If the grammar cannot be loaded because it is too complex or too large, or if it requires features not supported by the engine, the GrammarLoad member function returns an error. If the function succeeds, the fifth parameter receives the address of an IUnknown interface on the grammar object. An application can use the IUnknown interface to retrieve addresses of other interfaces supported by the grammar object.


Grammar Activation

For a speech-recognition engine to begin actively recognizing the words from a grammar, the application must activate the grammar by calling the ISRGramCommon::Activate member function, using the following syntax:

pISRGramCommon-Activate(NULL, FALSE, szRule);

This member function works only if the application has previously obtained the address of the ISRGramCommon interface by calling the IUnknown::QueryInterface member function on the grammar object.

In this call, the szRule parameter is the name of the context-free grammar rule to activate, as specified by an SRCFGXRULE structure. If the rule is already active or cannot be found, the Activate member function returns an error. (NULL causes the grammar to be active continuously rather than when a particular window is active, and FALSE ensures that the engine does not pause after this grammar is involved in a recognition.)

When the application calls the Activate member function, the engine starts the audio source if it is not already started and begins recognizing from it. If the engine is currently processing some audio, the grammar may not be activated until the next period of silence. Recognition continues until the application calls the Deactivate member function.

If the application specifies the handle of a window instead of NULL when calling the Activate member function, the engine "listens" for the grammar only when the specified window is the foreground window (that is, when the GetForegroundWindow function returns the window's handle). When a different window takes the foreground, the engine stops listening for the grammar. This saves the application from having to activate and deactivate a grammar whenever the application window gains and loses the foreground.

If the application passes TRUE with the Activate member function instead of FALSE, the engine pauses processing after the ISRGramNotifySink::PhraseFinish member function returns. This provides time for the application to process the recognition result and change the grammar. The application must call the ISRCentral::Resume member function to allow the engine to begin processing again.


Grammar-Specific Notifications

When the user speaks a command or phrase, the engine notifies the application by calling the ISRNotifySink::UtteranceBegin member function. When the user stops speaking, the engine calls the UtteranceEnd member function. The application should not use these notifications to influence recognition.

Unless the sound turns out to be just noise, some time after the engine calls UtteranceBegin, the PhraseStart member functions of all ISRGramNotifySink interfaces on active grammar objects are called, indicating that the engine has begun recognition processing on the audio. The application is notified of the time (in bytes) when the audio began. An application can convert this into hours, minutes, and seconds by calling the ISRCentral::ToFileTime member function.

If an engine supports hypotheses, it periodically calls the ISRGramNotifySink::PhraseHypothesis member function after the initial call to the PhraseStart member function. Many applications ignore calls to PhraseHypothesis because they have no use for the information, but some applications display the hypothesis to the user or perform preprocessing on the hypothesis.

When an engine has concluded its processing of the audio, it calls the PhraseFinish member function. The PhraseHypothesis and PhraseFinish member functions pass the following information to the application:

· The start and end time of the phrase. The application can use this information to isolate the audio or synchronize it with other events in the system.

· Two flags. One flag indicates whether the engine is confident enough about the recognition that the application should act on it, or the recognition results are unclear and the application should ask for clarification (much as a person would say "Could you repeat that, please"). The other flag indicates whether the best recognition result was obtained from this grammar or from another active grammar. An application should act upon a phrase recognition only if it was obtained from its own grammar.

· Address of a SRPHRASE structure. If the engine can recognize words from the speech, it passes the address of an SRPHRASE structure that contains a sequential list of SRWORD structures, each indicating the text and word identifier of the recognized word. An application can display these words, parse them, or do whatever it needs to do with the information. (If the grammar is context-free, the words must be in the grammar's set of words.)

· Address of a results object. Some engines support a speech-recognition results object that allows an application to get more information about what was recognized, such as accurate timing and alternative recognition possibilities. If a results object exists, the engine passes a non-NULL address of an IUnknown interface for the object. For more information about the speech-recognition results object, see "Speech-Recognition Results Objects" later in this section.

If the application needs to use the speech-recognition results object, it calls the IUnknown::QueryInterface member function to get an interface on the object (for example, ISRResBasic). In this case, it is the application's responsibility to release the results object. If the application doesn't use the results object, the engine calls the IUnknown::Release member function on the results object after the application returns from the PhraseFinish notification. The engine calls the Release member function so that an application that ignores the PhraseFinish notification does not inadvertently leave results objects in memory.

While training or double-checking its recognitions, an engine may change the result of a recognition. If so, the engine calls the ISRGramNotifySink::ReEvaluate member function to notify the application that the results object has changed and should be updated. Applications that use speech recognition for command and control typically ignore this notification, but those using it for data or text entry may use the information to improve performance.

The notification object is used by the grammar object until the latter frees itself, at which point it calls the ISRGramNotifySink::Release member function to release the notification sink. This may not occur immediately after the grammar object is released by the application because of asynchronous nature of speech recognition — that is, the engine may need to finish processing a certain amount of speech after the grammar object is released.


Lists in a Context-Free Grammar

A statically generated grammar is often not enough for complete recognition because it refers to lists of words that change dynamically, such as filenames. For example, a grammar might need a syntax such as "Open Filename"or "Send mail to Name." In these examples, Filename and Name might change continually, depending on the context in which they are used. The context-free grammar architecture supports this syntax through lists, which are sets of words that are dynamically set and that can be arbitrarily changed by the application.

To use a list in a context-free grammar, the list must be referenced by rules in the grammar, and the list's name must be specified in the SRCFGLIST section of the grammar's binary format. When the application first loads the grammar with the ISRCentral::GrammarLoad member function, all of the grammar's lists start out empty.

To specify the words for a list, the application calls the ISRGramCFG::ListSet member function and passes a list of SRWORD structures, each structure identifying a word or series of words that can be spoken. This effectively adds the words to the grammar, although it may result in slower performance because the words are not compiled and optimized as is the rest of the grammar. If the application sets the list while the engine is already processing a recognition, the changes may not take effect until processing is completed.

In addition to setting a list, engines typically support other member functions that allow an application to manipulate a list. The ListAppend member function appends new words to a list, ListRemove deletes words from a list, and ListQuery finds out whether a list has any words. An application can use the ListGet member function to retrieve the words from a list, and ListSet to fill an empty list field in a grammar or to replace an existing list.


Links in a Context-Free Grammar

Many of the rules in a grammar — such as those for recognizing numbers, dates, currencies, and times — can be used by almost every other grammar. One grammar can use another grammar's rules by linking to that grammar.

Links allow an application to create a grammar that exports common rules, serving a purpose similar to a C run-time library. Any grammar that has these rules on its imported-rules list automatically links to them when it is loaded, saving development time and system memory. Some engines may provide exported rules that are already defined and ready for the application to use. Eventually, third-party tool developers may develop and license common sets of rules.

To export a rule, the application writes an SRCFGXRULE structure in the grammar format. To link to an exported rule, the application specifies its rule identifier in the context-free grammar format in the same way as if they were internal to the grammar. However, instead of writing an SRCFGRULE structure that describes the rule, the application writes an SRCFGIMPRULE structure in the standard context-free grammar format.

When an application loads a grammar that imports rules, the engine checks all other context-free grammars in the system (including the one just loaded) to see if any of them export the requested rules. If so, the engine links the rules to the grammar. Any imported rules that are not linked at this time are automatically linked as soon as a grammar that exports them is loaded. If a link remains unconnected the grammar still works, although recognition paths that encounter the link are not possible. To determine whether or not a link is connected, the application can call the ISRGramCFG::LinkQuery member function.


Other Grammar Functions

The ISRGramCommon interface provides several other member functions that an application can use:

· Archive. Saves the compiled version of a grammar away so that it loads more quickly the next time the application uses it.

· BookMark. Sets a bookmark in the incoming audio stream, causing the application to receive a notification when it begins processing audio from a particular point in time. When the bookmark is reached, the engine calls the ISRGramNotifySink::BookMark member function and pauses so the application can change grammars, for example. An application might place bookmarks to synchronize the loading new grammars with mouse-clicks and keyboard events that change the internal state of the application.

· TrainDlg. Displays a dialog box that allows the user to train the grammar for his or her voice. The exact features of the dialog box are determined by the engine, but typically it requests that the user speak words in the grammar that are as of yet untrained, and does fine-tuning of difficult phrases.

· TrainPhrase. Queries engine for list of phrases speaker should utter to improve recognition accuracy. It is up to the application to prompt the speaker to speak the phrases, acquire recordings, and call the ISRResCorrection::Correction or ISRResCorrection::Validate member function for each phrase. A "phrase" can be a complete sentence, a partial sentence, or even an individual word.

· TrainQuery. Queries the engine for the kind of training that is needed for a particular grammar.


Grammar Deactivation

When an application no longer needs to use a particular grammar, it calls the ISRGramCommon::Deactivate member function with the name of the rule to deactivate. This deactivates the grammar as soon as possible but not immediately because recognition results may still be en route to the application.

When the application frees a grammar object, all of its rules are automatically deactivated.


Archiving and Releasing a Grammar

When an application is shutting down or is finished using a grammar, the application must release the grammar object (along with the engine object and all results objects) in order for the engine to free itself. The application releases a grammar object using the standard OLE COM mechanism of releasing all of the interfaces to the object.

Before the application releases a grammar object, however, it should call the ISRGramCommon::Archive member function. This copies the following information into memory:

· The internal representation of the grammar format. This allows the ISRCentral::GrammarLoad member function to proceed much faster the next time the grammar is used.

· Information about how the speaker uses the grammar. This might be the frequency of specific paths or words being used. Often this information is user-specific. This information improves recognition accuracy and speed the next time that the grammar is used.

· Results objects. The application can specify that data stored in all of the results objects be archived and retrieved the next time that the grammar is loaded.

The application can store the archive information anywhere, such as in a document file. The next time the application needs to use the grammar, it loads the information from the file into memory and calls the ISRCentral::GrammarLoad member function with the format SRGRMFMT_CFGNATIVE rather than SRGRMFMT_CFG (because the archived grammar is in the engine's native format). The grammar loads more quickly, and any information that the engine learned about the grammar in the last session is retained. Note that the new data format is engine-specific, so the application should maintain the grammar in standard CFG format in case the user changes speech-recognition engines in the application.

If the application archived results objects along with the grammar, the ISRGramNotifySink::UnArchive member function is called for each results object that was archived. This gives the application a chance to instantiate the archived data back into results objects and continue using them in the same state they were in before the Archive member function was called.


Dictation Grammars

Context-free grammars are sufficient for command and control or data entry, but speech recognition requires another grammar format, the dictation grammar. The dictation grammar format is much simpler than the context-free grammar format, so the application almost always generates the binary form of the dictation grammar. This format consists of the following elements:

· A broad category name such as "Medical," "Legal," or "Scientific." Some engines may have predefined language models for these and can take advantage of them.

· Several pages of sample writing from the user. This allows an engine to determine the user's writing style and vocabulary. The sample should correspond to the expected use of the grammar — for example, if the grammar is to be used for electronic mail, the sample writing should consist of e-mail messages. Or, if the session is for dictating a business letter, the samples should be business letters.

· Words to expect. If the application has information about what the user will be speaking about, it can include a set of likely words.

· Related words. An application can include sets of related words that are likely to occur in the dictation, such as city names or company names.

The application interacts with the grammar's data format and grammar objects in exactly the same way as it does with a context-free grammar, except that instead of using the ISRGramCFG interface, it uses ISRGramDictation.

The ISRGramDictation interface contains various functions that allow the application to tell the dictation engine what it expects the user to say next. These functions allow well-written applications to get an extra boost in accuracy.

The ISRGramDictation::Context member function provides the speech-recognition engine with the words that were spoken before and after the current utterance. An application calls the Context member function whenever the cursor position changes in a text document.

The Hint member function gives the speech-recognition engine information about the kind of text that the speaker is dictating. If an application has information that the user is about to speak an address, number, name, or some other specific data type, it should call the Hint member function to inform the engine.

An application can use the Words member function to give the engine a list of words that the user is likely to speak.

Archiving is more important for dictation grammars than context-free grammars because dictation grammars take longer to compile into native format and more information about the session is stored by the engine.

For more information about dictation grammars, see "Supporting Dictation in an Application" later in this section.


Speech-Recognition Results Objects

If an engine provides speech-recognition results objects, an application can use them to get more detailed information about a recognition. The following member functions of the ISRGramNotifySink notification return the address of an IUnknown interface on a results object:

· PhraseHypothesis

· PhraseFinish

· ReEvaluate

· UnArchive

After the application gets the address of IUnknown, it can call the QueryInterface member function to obtain other interfaces, such as the following (not all results objects support all of these interfaces):

· ISRResAudio. An engine may store the digital-audio information in the results object. This allows an application to access the audio so it can be played back to the user.

· ISRResBasic. This interface provides member functions that allow the application to get the same information supplied by the PhraseHypothesis and PhraseFinish member functions. It also allows an application to see the alternative recognition results.

· ISRResCorrection. Applications can and should correct the engine through this interface. If a user explicitly (or implicitly) informs the application that the engine recognized a phrase correctly or incorrectly, the application should pass this information to the engine through the ISRResCorrection interface. The engine can use the information to improve its future performance.

· ISRResEval. Allows the application to request that the engine reevaluate a recognition result to see if new training changes the result.

· ISRResGraph. An application can use these member functions to view the recognition results graph for both words and phonemes. It provides detailed information about what words or phonemes were spoken when, how loud they were, what their alternatives were, and so on.

· ISRResMemory. Results occupy a lot of memory. This interface allows an application to control how much information is stored in the results object.

· ISRResMerge. An application can use this interface to combine two temporally adjacent results objects into one.

· ISRResModifyGUI. An application can use this interface to display a small status window next to selected text for specific recognition results.

· ISRResScore. This interface provides member functions that allow the application to get scores for recognition results.

· ISRResSpeaker. Some engines support speaker identification. An application can access this to identify who spoke a particular command.

Because results objects consume so much memory, engines automatically "degrade" the results objects over time in an attempt to save memory, causing information to be lost in the process. The application can query and set the speed of this degradation by calling the ISRGramCommon::DeteriorationGet and DeteriorationSet member functions. An application can also take charge of memory management and lock a results object. Additionally, an application can take charge of memory management and lock a results object by calling the ISRResMemory::LockSet member function.


Speech-Recognition Sharing Object

When deciding upon a speech-recognition engine to use, an application should use the speech-recognition sharing object to find a suitable engine that is already in use on the desired wave-in audio device. If the sharing object cannot find such an engine, the application should use the sharing object to create a new instance of the engine that can be shared, in case another application wishes to share recognition with the application. Although an application can use a private instance of the engine without going through the sharing object, doing so prevents other applications from using the same instance and consumes more CPU time and memory. The speech-recognition sharing object is supplied by Microsoft.

An application can use a speech-recognition sharing object to accomplish the following:

· Share an engine and audio source with other applications, or with voice commands within the same application.

· Provide dictation services and command and control at the same time, if the engine supports both of these capabilities.

To create a speech-recognition sharing object, call the CoCreateInstance function with the CLSID_SRShare class identifier and an IID_IEnumSRShare interface identifier. The function returns the address of an IEnumSRShare interface on the newly created speech-recognition sharing object.

To enumerate the existing shared engine/audio-source pairs, an application uses the Clone, Next, Reset, and Skip member functions of the IEnumSRShare interface. These functions use the SRSHARE structure as the element in the enumeration sequence. If the Next member function finds a shared pair, it copies information about the pair into the SRSHARE structure. The application examines the structure to determine if the pair is suitable for its needs. If it is, the application can use the pair by calling the IEnumSRShare::Share member function, specifying the instance identifier of the desired pair.

If the enumeration process finds no suitable pairs, the application can use the New member function to create a new pair that can be shared with other applications. When calling New, the application specifies the GUID of a speech-recognition mode and the device identifier of an audio source that is not currently in use.

Both Share and New return the address of the ISRCentral interface on the engine of the pair. After obtaining this interface, the application interacts with the engine as though it had connected directly with it. The application should then hold the speech-recognition sharing object, until it does not need that engine. At that time, it needs to Detach from the sharing object. Then it can either release the sharing object or reuse it to start another engine.

When a speech-recognition sharing object is used to share an audio source among several engines, the object provides exclusive rights to the source to one engine at a time on a first-come, first-served basis. An engine requests the audio source by calling the IAudio::Claim or IAudio::Start member function. If more than one engine at a time requests the audio source, the first engine to claim the audio source is awarded exclusive use of it until that engine calls the IAudio::UnClaim or IAudio::Stop member function.

When an application is finished using a shared engine/audio-source pair, it must call the Detach member function to release the pair.

The speech-recognition sharing object has limitations that an application developer should consider:

· The object must be in a separate application, which requires marshaling and therefore is slower than a direct connection to a dynamic-link library (DLL).

· Any application can influence the recognition accuracy, confidence, and other speech-recognition attributes for a shared engine, which affects all applications sharing that engine.


Supporting Dictation in an Application

Some applications, such as word processors, should support dictation as an alternate means for the user to enter text. This section gives some suggestions on how to implement dictation in an application.


Receiving and Transcribing Text

The simplest way to perform dictation is to open a dictation grammar and activate it. Whenever a PhraseFinish notification is received, insert the text returned by PhraseFinish at the insertion point in the user's document, as if the user had typed the text.

The application should monitor the text for voice commands such as "Stop dictating" and other editing commands so it doesn't transcribe these into the user's document. Be aware that the words in "Stop dictating" may come in two different phrases, each with its own word.

The set of commands available within dictation is up to the application, but the number of commands should be kept to a minimum because any command that the application is listening for cannot be dictated. For example, if "Stop dictating" is a command, the user cannot dictate those words in a sentence such as "Please stop dictating my life."


Providing Feedback

An application can improve feedback to the user by displaying the information received with the PhraseHypothesis notification, replacing the text from the previous PhraseHypothesis each time a new one is received.

If the engine supports the ISRGramInsertionGUI member function, an application can use this interface to display a small status window next to the insertion point in the user's document. As the insertion point moves, the application calls the ISRGramGUI::Show and ISRGramGUI::Move member functions to change the window's position.


Improving Accuracy

Certain functions can be used to improve the accuracy of dictation. If an application has information about what kind of information is being dictated, or if it has information about the context, it should call the ISRGramDictation::Context member function whenever the user moves the insertion point, whether by using the mouse, the keyboard, or a voice command.

An application can also use the ISRGramDictation::Hint member function if it has information that an address or date is being entered, or the ISRGramDictation::Words member function if a likely set of words is being used.


Taking Advantage of Results Objects

To take full advantage of an engine's capabilities, an application should maintain the speech-recognition results objects returned by the ISRGramNotifySink::PhraseFinish or ISRGramNotifySink::PhraseHypothesis member functions and keep track of which results object produced which text. After this is accomplished, the application can use the results objects in a number of ways.

If the user highlights a word or set of words, an application can call the ISRResModifyGUI::Show and ISRResModifyGUI::Move member functions to display the engine's results-object modification window below the selected text. Depending on the engine, this window may allow the user to choose from different selections or take advantage of other engine features not necessarily accessible through the speech functions. If the user highlights only part of a result or highlights text from several results objects, the application can use member functions in the ISRResMerge interface to combine or divide the results objects. Some engines may not supply the ISRResModifyGUI interface.

Through a context menu on a word or phrase, an application might play an audio recording of what the user actually said to dictate the word or phrase. The applications can acquire the audio through the ISRResAudio interface.

Through a context menu on a word or phrase, an application might show the alternative recognition possibilities for the word or phrase and allow the user to choose one of the alternatives. The application can acquire a list of alternatives through either the ISRResBasic or ISRResGraph interface. For example, if the user says "pine" but the engine recognizes "line," the application would show all the alternatives, which might include "pine," "fine," and "mine."

If a user modifies or dictates the text of a results object again, the application should call the ISRResCorrection::Correction member function to inform the engine of the change. This allows a speech-recognition engine to adjust future recognitions to avoid repeating the same mistakes. If the user does not modify the text in a results object for some time, then the application should call the ISRResCorrection::Validate member function to inform the engine that the dictated text was probably correct.

An engine may reevaluate its recognition results during idle CPU time after having learned more about the document being dictated. If an engine does reevaluate a results object, it calls the ISRGramNotifySink::ReEvaluate member function to inform the application of the change.

Because storing multiple results objects consumes a lot of memory, an application may release them after the user moves on to the next sentence or paragraph. Some applications may encourage the user to do a two-pass dictation where they dictate in one pass and correct in the second. In such a case the application needs to keep all of the results objects, but it may reduce its memory requirements by calling member functions of the ISRResMemory interface.

Finally, if the user is finished working on the document for the day and saves it, the application can save the results objects along with the text by calling the ISRGramCommon::Archive member function, and it can reload the archived results objects into memory when the ISRCentral::GrammarLoad member function is called.


Sharing Recognition with Other Applications

An application using the previously described techniques is able to incorporate dictation and act as a stand-alone application. However, other applications running on the machine may also be using speech recognition and may wish to share recognition with the active application, either by using the speech-recognition sharing object or by using voice commands. To integrate well with other applications, a dictation application should be written with extra precautions.

When deciding upon a speech-recognition engine to use, the application should use the speech-recognition sharing object and find an engine that supports dictation grammars that are already in use on the desired wave-in audio device. If such an engine is not found, the application should create a new instance of a dictation engine to be shared. Applications can use private instances of the engine without going through the sharing object, but this prevents other applications from using the same instance and consumes more CPU time and memory.

The dictation grammar should not be activated unless the application's editor window has the keyboard focus and the application is in dictation mode. An application can get into dictation mode through a voice command or a menu.

When an application activates the dictation grammar it should disable voice commands through the IVCmdAttributes::EnabledSet member function and return the voice-command object to its previous state when the dictation grammar is deactivated. This allows smooth integration with voice commands.


Using Speech Recognition

This section contains examples demonstrating how to perform the following tasks:

· Create the speech-recognition objects, including the audio-source, engine, and grammar objects.

· Find an engine and audio source that meets an application's needs.

· Load a grammar and create a grammar object.

· Process notifications of recognition events.


Initializing an Application for Speech Recognition

To implement speech-recognition features, you must initialize the OLE libraries and then create instances of at least three speech-recognition objects: the audio-source object, the speech-recognition engine object, and the grammar object. The audio-source object provides speech input to the engine object that processes the speech. The grammar object provides the engine with the words and rules that determine what the engine can recognize. In addition, you can use the speech-recognition sharing object and the speech-recognition enumerator to find an engine that meets your application's requirements.

The following example demonstrates how to initialize an application for speech recognition. It shows how to create the necessary objects for implementing speech recognition. The example accomplishes the following tasks:

· Uses the CoInitialize function to initialize the OLE libraries.

· Creates an instance of the multimedia audio-source object, which allows the engine to receive input from the multimedia wave-in driver.

· Creates a speech-recognition sharing object and a speech-recognition enumerator. The application-defined FindSuitableEngine function (described in the following section) uses the sharing object and enumerator to find an engine and audio-source that meets the application's requirements.

· Creates an instance of an application-defined class that implements the ISRGramNotifySink interface (described in "Processing Recognition Notifications" later in this section). The engine calls this interface to notify the application of recognition events.

· Calls the application-defined InitializeGrammar function (described in "Loading a Grammar" later in this section) that loads a context-free grammar from a file and creates a grammar object.

· Calls the ISRGramCommon::Activate member function to activate the newly created grammar.

// BeginOLE - Initializes OLE, creates the speech-recognition objects,

// and activates a grammar.

// Returns TRUE if successful, or FALSE otherwise.

//

// Global variables:

// g_pIAMMD - address of the IAudioMultimediaDevice interface

// g_pIEnumSRShare - address of the IEnumSRShare interface

// g_pGramNotifySink - address of the ISRGramNotifySink interface

// g_szGramFile - name of the grammar file

// g_hwndMain - handle of the application's main window

BOOL BeginOLE()

{

HRESULT hRes;

PISRFIND pISRFind = NULL;

// Initialize OLE.

if (FAILED(CoInitialize(NULL)))

return FALSE;

// Create the audio-source object and retrieve the address of the

// object's IAudioMultimediaDevice interface. By default, the object

// uses the WAVE_MAPPER device.

if (CoCreateInstance(CLSID_MMAudioSource, NULL, CLSCTX_ALL,

IID_IAudioMultiMediaDevice, (LPVOID *) &g_pIAMMD) != S_OK)

return FALSE;

// Create a speech-recognition sharing object and retrieve the

// address of the object's IEnumSRShare interface.

CoCreateInstance(CLSID_SRShare, NULL, CLSCTX_ALL,IID_IEnumSRShare,

(LPVOID *) &g_pIEnumSRShare);

// Create a speech-recognition enumerator and retrieve the address

// of the enumerator's ISRFind interface.

if (CoCreateInstance(CLSID_SREnumerator, NULL, CLSCTX_ALL,

IID_ISRFind, (LPVOID *) &g_pSRFind) != S_OK)

return FALSE;

// Call an application-defined function that uses the speech-

// recognition sharing object and enumerator to find and select a

// suitable speech-recognition engine.

hRes = FindSuitableEngine(pISRFind);

pISRFind->Release();

if (hRes != NOERROR) return FALSE;

// Create the grammar notification interface based on an

// application-defined CISRGramNotifySink class.

if ((g_pGramNotifySink = new CISRGramNotifySink) == NULL)

return FALSE;

// Call an application-defined function that reads grammar data from

// a file and then loads the grammar.

if (!InitializeGrammar(g_pISRCentral, g_szGramFile))

return FALSE;

// Activate the grammar.

if ((hRes = g_pISRGramCommon->Activate(g_hwndMain, FALSE, NULL))

!= NOERROR)

return FALSE;

return TRUE;

}


Finding a Suitable Engine and Audio Source

You use the speech-recognition sharing object and the speech-recognition enumerator to find and select an engine and audio-source pair that meets the needs of your application. Because sharing an engine and audio source is more efficient in terms of system resources than using a private pair, you should try to find and use a shared pair before creating a private pair. You should follow these steps to find and select an engine and audio source:

1. Use the enumeration functions of the sharing object's IEnumSRShare interface to search for a suitable shared engine and audio-source pair. If the sharing object finds a suitable pair, use the IEnumSRShare::Share member function to use the shared pair for speech recognition.

2. If the sharing object doesn't find a suitable pair, use the speech-recognition enumerator's ISRFind or ISREnum interfaces to find a suitable engine, and then try to create a shared engine and audio-source pair by calling the IEnumSRShare::New member function.

3. If the IEnumSRShare::New member function fails, create a private engine and audio-source pair by calling the ISRFind::Select or ISREnum::Select member function.

The following example demonstrates how to find and select a suitable engine and audio-source pair. In the example, a "suitable engine" is one that supports WAVE_MAPPER as the audio-input device, recognizes English speech, and supports context-free grammars.

// FindSuitableEngine - Uses a speech-recognition enumerator to find a

// speech-recognition engine that supports context-free grammars and

// recognizes English speech.

// Returns a result code.

// pISRFind - address of the ISRFind interface

//

// Global variables:

// g_srModeInfo - SRMODEINFO structure that contains information

// about the current speech-recognition mode

// g_qwInstID - instance identifier of the shared engine/audio-

// source pair

// g_pIEnumSRShare - address of the IEnumSRShare interface

// g_pISRCentral - address of the ISRCentral interface

// g_pIAMMD - address of the IAudioMultimediaDevice interface

HRESULT FindSuitableEngine(PISRFIND pISRFind)

{

SRMODEINFO srmiFound;

SRSHARE srShare;

HRESULT hRes;

ULONG uNum;

GUID guid;

DWORD dwNumFound;

PISRENUM pISREnum;

// Erase the contents of the global SRMODEINFO structure.

ZeroMemory(&g_srModeInfo, sizeof(g_srModeInfo));

// Define the mode to look for by filling the relevant members of

// the global SRMODEINFO structure.

g_srModeInfo.language.LanguageID =

MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US);

g_srModeInfo.dwGrammars = SRGRAM_CFG;

// Use the speech-recognition sharing object to find a suitable

// shared engine/audio-source pair.

if (g_pIEnumSRShare != NULL) {

while (TRUE) {

ZeroMemory(&srShare, sizeof(srShare));

hRes = g_pIEnumSRShare->Next(1, &srShare, &uNum);

if ((hRes == S_FALSE) || (FAILED(hRes)))

break;

if ((srShare.dwDeviceID == WAVE_MAPPER) &&

(srShare.srModeInfo.language.LanguageID ==

g_srModeInfo.language.LanguageID) &&

(srShare.srModeInfo.dwGrammars ==

g_srModeInfo.dwGrammars))

break;

}

// See if a shared mode was found by using memcmp (a C run-time

// function) to check if the GUID is zero. If the GUID is valid,

// share the engine/audio-source pair.

ZeroMemory(&guid, sizeof(guid));

if (memcmp(&srShare.srModeInfo.gModeID, &guid,

sizeof(GUID)) != 0) {

hRes = g_pIEnumSRShare->Share(srShare.qwInstanceID,

(PISRCENTRAL *) &g_pISRCentral);

if (hRes == NOERROR)

return NOERROR;

}

}

// No shared mode found. Use an engine enumerator to find a suitable

// engine.

ZeroMemory(&srmiFound, sizeof(srmiFound));

if (hRes = pISRFind->QueryInterface(IID_ISREnum,

(LPVOID *) &pISREnum) != NOERROR)

return hRes;

while (hRes = pISREnum->Next(1, &srmiFound, &dwNumFound) == NOERROR)

if ((srmiFound.language.LanguageID ==

g_srModeInfo.language.LanguageID)

&& (srmiFound.dwGrammars == g_srModeInfo.dwGrammars))

break;

pISREnum->Release(); // release enumerator

// See if the enumerator found a suitable engine.

ZeroMemory(&guid, sizeof(guid));

if (memcmp(&srmiFound.gModeID, &guid, sizeof(GUID)) == 0)

return -1; // no suitable engine found

// The engine is suitable. Try to create an engine/audio-source pair

// that can be shared.

if (g_pIEnumSRShare != NULL) {

hRes = g_pIEnumSRShare->New(WAVE_MAPPER, srmiFound.gModeID,

(PISRCENTRAL *) &g_pISRCentral, &g_qwInstID);

if (hRes == NOERROR)

return NOERROR; // engine successfully shared

}

// Couldn't create a shared pair. Try to create a private

// engine/audio-source pair.

hRes = pISRFind->Select(srmiFound.gModeID, &g_pISRCentral,

(LPUNKNOWN) g_pIAMMD);

if (hRes == NOERROR)

return NOERROR; // private mode successfully created

return -1; // no suitable engine

}


Loading a Grammar

A grammar provides a speech-recognition engine with the words and rules that determine what the engine can recognize. Before an engine can use a grammar, you must create and load a grammar object by calling the ISRCentral::GrammarLoad member function. The function's arguments include an SDATA structure that specifies the address and size of the grammar's data, and the address of the ISRGramNotifySink interface that the engine uses to notify the application of recognition events. The GrammarLoad member function returns the address of the grammar object's IUnknown interface. The application can use IUnknown to retrieve the addresses of other interfaces provided by the grammar object, including ISRGramCommon and ISRGramCFG.

The following example shows how to load a grammar, create a grammar object, and obtain the addresses of the grammar object's ISRGramCommon and ISRGramCFG interfaces. The example includes a function called GrammarReadFile that creates a memory object, reads grammar data from a file into the object, and fills an SDATA structure with the address and size of the object.

// InitializeGrammar - Loads a context-free grammar and obtains the

// addresses of interfaces supported by the grammar object.

// Returns TRUE if successful, or FALSE otherwise

// pISRCentral - address of the ISRCentral interface

// pszFileName - name of the grammar file

//

// Global variables:

// g_pGramNotifySink - address of the application's

// ISRGramNotifySink interface

// g_pISRGramCommon - address of the ISRGramCommon interface

// g_pISRGramCFG - address of the ISRGramCFG interface

// g_pISRGramDictation - address of the ISRGramDictation

// interface

BOOL InitializeGrammar(PISRCENTRAL pISRCentral, LPSTR pszFileName)

{

SDATA dGramData;

HGLOBAL hgbl;

HRESULT hRes;

LPUNKNOWN lpUnk;

// Call an application-defined function that reads the contents of

// the specified grammar file into a global memory object.

if (!GrammarReadFile(pszFileName, &dGramData, &hgbl))

return FALSE;

// Load the grammar and create a grammar object.

hRes = pISRCentral->GrammarLoad(SRGRMFMT_CFG, dGramData,

(PVOID)g_pGramNotifySink, IID_ISRGramNotifySink, &lpUnk);

// Unlock and free the global memory object.

GlobalUnlock(hgbl);

GlobalFree(hgbl);

if (hRes != NOERROR) {

HandleError(hRes);

return FALSE;

}

// Get the address of the ISRGramCommon interface.

hRes = lpUnk->QueryInterface(IID_ISRGramCommon,

(void**) &g_pISRGramCommon);

if (hRes != NOERROR) {

HandleError(hRes);

lpUnk->Release();

return FALSE;

}

// Get the address of the ISRGramCFG interface.

hRes = lpUnk->QueryInterface(IID_ISRGramCFG,

(void**) &g_pISRGramCFG);

if (hRes != NOERROR) {

HandleError(hRes);

lpUnk->Release();

return FALSE;

}

// Get the address of the ISRGramDictation interface.

hRes = lpUnk->QueryInterface(IID_ISRGramDictation,

(void**) &g_pISRGramDict);

lpUnk-Release();

return TRUE;

}

// GrammarReadFile - Reads the contents of a grammar file into a global

// memory object.

// Returns TRUE if successful, or FALSE otherwise.

// pszFileName - name of the grammar file

// pData - address of a structure that receives the size and address

// of the grammar data

// phgbl - address of a variable that receives the handle of the memory

/ object

BOOL GrammarReadFile(LPSTR pszFileName, PSDATA pData, HGLOBAL *phgbl)

{

HANDLE hf;

DWORD dwBytesRead;

// Open the grammar file.

hf = CreateFile(pszFileName, GENERIC_READ, 0, NULL, OPEN_EXISTING,

FILE_ATTRIBUTE_NORMAL, NULL);

if (hf == NULL)

return FALSE;

// Retrieve the file size and allocate a global memory object

// to contain the file's contents.

pData-dwSize = GetFileSize(hf, NULL);

*phgbl = GlobalAlloc(GMEM_ZEROINIT, pData-dwSize);

pData-pData = GlobalLock(*phgbl);

if (!pData-pData) {

CloseHandle(hf);

return FALSE;

}

// Read the contents of the file into memory, filling the SDATA

// structure in the process.

ReadFile(hf, pData->pData, pData->dwSize, &dwBytesRead, NULL);

CloseHandle(hf);

return TRUE;

}


Processing Recognition Notifications

The speech-recognition engine object calls an application's ISRGramNotifySink interface to inform the application of recognition events so that the application can respond to them. To receive notifications, the application must create a COM object that supports the ISRGramNotifySink interface, and it must pass the address of the interface to the engine object when calling the ISRCentral::GrammarLoad member function.

The ISRGramNotifySink interface consists of a set of member functions that correspond to recognition events. When an event occurs, the engine object calls the member function that corresponds to the event.

The following example shows how to define an object class that implements the ISRGramNotifySink interface:

class CISRGramNotifySink : public ISRGramNotifySink {

public:

CISRGramNotifySink(void);

~CISRGramNotifySink(void);

// IUnknown members that delegate to m_punkOuter

// Non-delegating object IUnknown

STDMETHODIMP QueryInterface (REFIID, LPVOID FAR *);

STDMETHODIMP_(ULONG) AddRef();

STDMETHODIMP_(ULONG) Release();

// ISRNotifySink

STDMETHODIMP BookMark (DWORD);

STDMETHODIMP Paused ();

STDMETHODIMP PhraseFinish (DWORD, QWORD, QWORD, PSRPHRASE,

LPUNKNOWN);

STDMETHODIMP PhraseHypothesis (DWORD, QWORD, QWORD, PSRPHRASE,

LPUNKNOWN);

STDMETHODIMP PhraseStart (QWORD);

STDMETHODIMP ReEvaluate (LPUNKNOWN);

STDMETHODIMP Training (DWORD);

STDMETHODIMP UnArchive (LPUNKNOWN);

CISRGrammarObject *m_pGramObj; // address of grammar object

protected:

DWORD m_dwRefCnt;

};

The following example shows a simple implementation of the ISRGramNotifySink::PhraseFinish member function. The engine object calls PhraseFinish when the speaker has finished a phrase and the engine has recognized the words that were spoken. The member function receives the address of an SRPHRASE structure that includes one SRWORD structure for each recognized word in the phrase. The example parses the SRPHRASE structure, writes the words from the phrase into a buffer, and then causes a WM_PAINT message to be sent to a window that draws the contents of the buffer.

STDMETHODIMP CISRGramNotifySink::PhraseFinish(DWORD dwFlags,
QWORD qTimeStampBegin, QWORD qTimeStampEnd, PSRPHRASE pSRPhrase,
LPUNKNOWN lpResults )

{

DWORD dwCnt; // count of SRPHRASE bytes processed

SRWORD *psrWord;

PBYTE pb;

// Check the flags to determine whether to process the phrase.

if (!((dwFlags & ISRNOTEFIN_RECOGNIZED) &&

(dwFlags & ISRNOTEFIN_THISGRAMMAR)))

return -1;

// Initialize variables for processing the SRPHRASE structure.

dwCnt = sizeof(pSRPhrase->dwSize); // account for size of dwSize

psrWord = (SRWORD *) pSRPhrase->abWords; // point to first SRWORD

pb = (PBYTE) psrWord; // initialize index pointer

g_szPhrase[0] = '\0'; // makes lstrcat work properly

// Process the SRPHRASE structure, stopping the entire contents

// have been processed.

while (dwCnt pSRPhrase->dwSize) {

// Copy word to buffer, followed by a space to separate words.

lstrcat(g_szPhrase, (LPSTR) psrWord-szWord);

lstrcat(g_szPhrase, " ");

dwCnt += psrWord-dwSize; // add size of SRWORD to index

pb += psrWord-dwSize; // move index pointer to next SRWORD

psrWord = (PSRWORD) pb; // point to next SRWORD

}

// Cause a WM_PAINT so that the phrase is drawn in the window.

InvalidateRect(g_hwndSR, NULL, TRUE);

UpdateWindow(g_hwndSR);

return NOERROR;

}


Low-Level Speech Recognition Quick-Look


IEnumSRShare

This interface is supported by the speech-recognition sharing object.

Member function | Description
Clone | Returns another enumerator containing the same enumeration state as the current one.
New | Creates a new engine/audio-source pair that can be shared by other applications.
Next | Retrieves the specified number of items in the enumeration sequence.
Reset | Resets the enumeration sequence back to the beginning.
Share | Allows an application to share an existing engine/audio-source pair.
Skip | Skips a specified number of elements in the enumeration sequence.


ILexPronounce

This interface is supported by the speech-recognition engine.

Member function | Description
Add | Adds the pronunciation and part of speech of a word to the engine's pronunciation lexicon.
Get | Gets the pronunciation and part of speech of a word from the lexicon.
Remove | Removes the pronunciation and part of speech of a word from the lexicon.


ISRAttributes

This interface is supported by the speech-recognition engine.

Member function | Description
AutoGainEnableGet | Gets the state of the automatic gain for the incoming audio stream.
AutoGainEnableSet | Sets the state of the automatic gain for the incoming audio stream.
EchoGet | Gets the state of echo canceling on the incoming audio stream.
EchoSet | Sets the state of echo canceling on the incoming audio stream.
EnergyFloorGet | Gets the value of the current noise floor expected by a speech-recognition engine from the audio stream.
EnergyFloorSet | Sets the value of the current noise floor expected by a speech-recognition engine from the audio stream.
MicrophoneGet | Gets the name of the current microphone for this audio source.
MicrophoneSet | Sets the name of the current microphone for this audio source.
RealTimeGet | Gets the real-time setting for the engine.
RealTimeSet | Sets the real-time setting for the engine.
SpeakerGet | Gets the name of the current speaker for an audio source.
SpeakerSet | Sets the name of the current speaker for an audio source.
ThresholdGet | Gets the threshold level of the speech-recognition engine.
ThresholdSet | Sets the threshold level of the speech-recognition engine.
TimeOutGet | Gets the current time-out values for a speech-recognition engine.
TimeOutSet | Sets the time-out values for a speech-recognition engine.


ISRCentral

This interface is supported by the speech-recognition engine.

Member function | Description
GrammarLoad | Loads an archived or newly created grammar and returns a grammar object.
ModeGet | Gets information specific to the current speech-recognition mode.
Pause | Pauses the speech-recognition engine.
PosnGet | Gets the exact byte in the audio stream that is currently being received.
Register | Registers an ISRNotifySink interface to notify the application of grammar-independent recognitions.
Resume | Reduces the pause count of a paused speech-recognition engine by 1.
ToFileTime | Converts a QWORD representing the time in bytes to a Win32 FILETIME structure.
UnRegister | Releases a notification interface through which an engine sent grammar-independent notifications to an application.


ISRDialogs

This interface is supported by the speech-recognition engine.

Member function | Description
AboutDlg | Displays an About dialog box that identifies the speech-recognition engine and contains the copyright notice.
GeneralDlg | Allows general control of the engine and should invoke some of the other dialog boxes.
LexiconDlg | Allows the speaker to display and edit his or her pronunciation lexicon.
TrainMicDlg | Provides the engine with training for the microphone.
TrainGeneralDlg | Provides phonetic training, which trains the engine for a preselected set of words.


ISREnum

This interface is supported by the speech-recognition enumerator and the engine enumerator.

Member function | Description
Clone | Returns another enumerator containing the same enumeration state as the current one.
Next | Retrieves the specified number of items in the enumeration sequence.
Reset | Resets the enumeration sequence back to the beginning.
Select | Selects a speech-recognition mode and creates an engine object to represent the selected mode.
Skip | Skips over a specified number of elements in the enumeration sequence.


ISRFind

This interface is supported by the speech-recognition enumerator and the engine enumerator.

Member function | Description
Find | Finds the speech-recognition mode that most closely matches the desired characteristics.
Select | Selects a speech-recognition mode and creates an engine object to represent the selected mode.


ISRGramCFG

This interface is supported by the grammar object of a context-free grammar.

Member function | Description
LinkQuery | Indicates whether or not an imported symbol used by this grammar has been linked to another grammar.
ListAppend | Appends new words to an existing list.
ListGet | Gets the list of words that the engine is currently using.
ListQuery | Indicates whether or not a list has had words set.
ListRemove | Removes words from an existing list.
ListSet | Fills in an empty list field in a grammar or replaces an existing list.


ISRGramCommon

This interface is supported by the grammar object.

Member function | Description
Activate | Activates a grammar so that the speech-recognition engine can begin using the grammar to recognize speech.
Archive | Compiles a grammar in a standard grammar format into the native format for the engine and stores the compiled grammar in a data buffer.
Bookmark | Sets a bookmark in the incoming audio stream.
Deactivate | Deactivates an active grammar so that the speech-recognition engine no longer uses the grammar to recognize speech.
DeteriorationGet | Gets the values used by the engine to determine how long to retain data stored in speech-recognition results objects.
DeteriorationSet | Sets the values used by the engine to determine how long to retain data stored in speech-recognition results objects.
TrainDlg | Displays a dialog box so that the speaker can train the grammar.
TrainPhrase | Queries the engine for a list of phrases the speaker should utter to improve recognition accuracy.
TrainQuery | Queries the engine for the kind of training that is needed for a particular grammar.


ISRGramDictation

This interface is supported by the grammar object of a dictation grammar.

Member function | Description
Context | Provides a speech-recognition engine with context—that is, the words that were spoken before and after the current utterance.
Hint | Gives the speech-recognition engine information about the subcontext in which a document is being used.
Words | Informs the speech-recognition engine that a particular set of words is likely to be spoken (the engine can ignore these).


ISRGramInsertionGUI

This interface is supported by the speech-recognition engine.

Member function | Description
Hide | Hides a visible status window.
Move | Moves the status window away from the specified cursor position, usually just below or above it.
Show | Shows the status window (When the grammar object is first created, the status window is hidden.


ISRGramNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
Bookmark | Notifies the application that the engine has encountered a bookmark in the incoming audio stream.
Paused | Reminds the application that the engine is paused.
PhraseFinish | Informs the application that the speaker has finished a phrase and the speech-recognition engine has determined the words that were spoken.
PhraseHypothesis | Informs the application that the engine has a hypothesis about what was said over the past few seconds but does not have enough information to commit to a recognition.
PhraseStart | Notifies the application that the engine has started processing audio data for a recognition result.
ReEvaluate | Notifies the application that the engine changed a speech-recognition results object as a consequence of the engine's background processing (rather than as a result of an explicit call of the ISRResEval::ReEvaluate member function by the application).
Training | Notifies the application that the engine requires training for the current speaker, the current grammar, or the current microphone.
UnArchive | Notifies an application that a speech-recognition results object that was archived with a grammar has been loaded with the grammar.


ISRNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
AttribChanged | Notifies an application that an engine attribute has changed.
Interference | Notifies the application that the engine cannot recognize speech properly for a known reason.
Sound | Notifies the application that the engine detected a sound (or a very garbled word).
UtteranceBegin | Notifies the application that the engine has detected the beginning of an utterance or sound.
UtteranceEnd | Notifies the application that the utterance has ended.
VUMeter | Notifies the application of the correct VU (Volume Units) level, which indicates the loudness of the digital-audio stream.


ISRResAudio

This interface is supported by the speech-recognition results object.

Member function | Description
GetWAV | Gets an audio recording of an utterance in waveform-audio (.wav) file format.


ISRResBasic

This interface is supported by the speech-recognition results object.

Member function | Description
FlagsGet | Gets the recognition flags for the specified phrase.
Identify | Gets a unique identifier for a results object so the application can retrieve the object at a later time.
PhraseGet | Gets the specified phrase from the list of most likely phrases provided by the speech-recognition engine for a recognition.
TimeGet | Gets the time stamps that mark the beginning and ending of the phrase.


ISRResCorrection

This interface is supported by the speech-recognition results object.

Member function | Description
Correction | Indicates to a speech-recognition engine what the speaker actually said to cause a particular recognition result.
Validate | Notifies a speech-recognition engine that the utterance was correctly recognized.


ISRResEval

This interface is supported by the speech-recognition results object.

Member function | Description
ReEvaluate | Causes a speech-recognition engine to reevaluate the recognition result represented by this object.


ISRResGraph

This interface is supported by the speech-recognition results object.

Member function | Description
BestPathPhoneme | Gets the specified phoneme path from the list of most likely phoneme paths provided by the speech-recognition engine for a recognition.
BestPathWord | Gets the specified word path from the list of most likely phoneme paths provided by the speech-recognition engine for a recognition.
GetPhonemeNode | Gets information about the phoneme for a given node in a recognition results graph.
GetWordNode | Gets information about the word for a given node in a recognition results graph.
PathScorePhoneme | Determines the score for a particular phoneme path.
PathScoreWord | Determines the score for a particular word path.


ISRResMemory

This interface is supported by the speech-recognition results object.

Member function | Description
Free | Instructs a speech-recognition results object to discard the specified information so that it uses less memory.
LockGet | Gets the lock status for a speech-recognition results object.
LockSet | Sets the lock status for a speech-recognition results object.


ISRResMerge

This interface is supported by the speech-recognition results object.

Member function | Description
Merge | Merges another speech-recognition results object with the current object so their data can be manipulated together.
Split | Splits a copy of a speech-recognition results object into two parts so the application can examine a subset of its data.


ISRResModifyGUI

This interface is supported by the speech-recognition results object.

Member function | Description
Hide | Hides a visible status window.
Move | Moves the status window away from the specified rectangle, usually just below or above it.
Show | Shows the status window (When the results object is first created, the status window is hidden)


ISRResScore

This interface is supported by the speech-recognition results object.

Member function | Description
GetPhraseScore | Gets the score of a recognition result.
GetWordScores | Gets the score of individual words in a recognition result.


ISRResSpeaker

This interface is supported by the speech-recognition results object.

Member function | Description
Correction | Indicates to a speech-recognition engine which speaker actually spoke to cause this recognition result.
Identify | Identifies the speaker for a recognized utterance.
IdentifyForFree | Queries the speech-recognition results object to see whether or not identifying the speaker will require processing.
Validate | Notifies a speech-recognition results object that the speaker was correctly recognized.


ISRSpeaker

This interface is supported by the speech-recognition engine.

Member function | Description
Delete | Deletes all information that a speech-recognition engine has about a speaker.
Enum | Enumerates the list of speakers known to the speech-recognition engine.
Merge | Merges a speaker profile from another system into the speaker profile on the current system.
New | Creates a new speaker, using the default settings provided by the engine for training.
Query | Gets the name of the currently selected speaker.
Read | Gets speaker profile information for the specified speaker.
Revert | Restores training to its original state at the beginning of a training session.
Select | Selects the specified speaker to be the current speaker on the system.
Write | Writes speaker profile information from another system to the current system.


Speech-Recognition Structures

Structure

Description

Member functions

SDATA

Contains a pointer to data and the data's size.

IAudio::WaveFormatGet
IAudio::WaveFormatSet
IAudioMultimediaDevice:
:CustomMessage
ISRCentral::GrammarLoad
ISRGramCFG::ListAppend
ISRGramCFG::ListGet
ISRGramCFG::ListRemove
ISRGramCFG::ListSet
ISRGramCommon::TrainPhrase
ISRResAudio::GetWAV

SRMODEINFO

Provides details about a speech-recognition mode supported by a particular engine.

ISRCentral::ModeGet
ISREnum::Clone
ISREnum::Next
ISREnum::Reset
ISREnum::Skip
ISRFind::Find

SRMODEINFORANK

Ranks the members of the SRMODEINFO structure in order of importance.

ISRFind::Find

SRPHRASE

Identifies a unique phrase for a grammar's vocabulary.

ISRGramNotifySink:
:PhraseFinish
ISRGramNotifySink:
:PhraseHypothesis
ISRResBasic::PhraseGet
ISRResCorrection::Correction

SRRESPHONEMENODE

Provides information about a phoneme node in a recognition/alternative graph generated by a speech-recognition engine.

ISRResGraph::GetPhonemeNode

SRRESWORDNODE

Provides information about a word node in a recognition/alternative graph generated by a speech-recognition engine.

ISRResGraph::GetWordNode

SRSHARE

Provides information about a shared engine/audio source pair.

IEnumSRSHARE::Clone
IEnumSRSHARE::Next
IEnumSRSHARE::Reset
IEnumSRSHARE::Skip

SRWORD

Identifies a unique word for a grammar's vocabulary.

ISRResGraph::GetWordNode


Objects

Speech-recognition objects provide detailed control of speech recognition. Together these objects serve as a recognition layer for an application, allowing it to use any of the speech-recognition engines available in the system and to recognize input from multimedia wave-in devices or other audio sources. This section describes these objects and lists the interfaces they support.

In addition to the interfaces listed in this section, each object supports the IUnknown interface, which is a common interface supported by all OLE component objects. For more information about IUnknown, see the section, "Common Elements of the Speech Application Programming Interface."

If your application does not require detailed control of speech recognition, you can use voice-command objects to implement command and control. Voice commands use speech-recognition objects, so they also serve as an example of implementing speech recognition. For more information about voice commands, see the section, "Voice Commands."


Grammar Object


Purpose

Represents a grammar, which provides a set of words and phrases that can be recognized by an engine. A grammar object can represent a context-free, limited-domain, or dictation grammar. The application uses a grammar object to control how the engine uses the grammar to recognize speech. The grammar object is supplied by the engine vendor.


Created by

To create a grammar object, use the ISRCentral::GrammarLoad member function on an engine object and pass information about the grammar, such as its format and where it is located. The grammar can be new or previously archived. You can optionally pass the address of a notification interface that the grammar object can use to send notifications back to the application. To receive notifications, an application must create a COM object that supports ISRGramNotifySink as an interface. How the application responds to notifications is up to the application developer.

The GrammarLoad member function returns the address of an IUnknown interface on the newly created grammar object.


Interacts with

A grammar object provides the rules, lists of words, or context that an engine uses to parse incoming speech and recognize it. An engine object can use multiple grammar objects to recognize incoming speech.

The grammar object notifies the application of events that pertain to speech recognition with that grammar, such as a successful recognition or a hypothesis about a phrase.


Interfaces

The grammar object should support the following interfaces. The grammar object provided for a particular engine may support other interfaces with additional functionality.

Interface | Description
ISRGramCFG | (Context-free grammar only.) Controls a context-free grammar. Member functions manage the lists of words that the grammar recognizes, or determine whether the grammar is linked to another grammar.
ISRGramCommon | Controls features common to context-free, limited-domain, and dictation grammars. Member functions activate and deactivate a grammar, archive it, set a bookmark in the incoming audio stream, control storage of speech-recognition results objects, and allow the user to train the grammar to recognize his or her voice.
ISRGramDictation | (Dictation grammar only.) Controls a dictation grammar. Member functions set the context for dictation and indicate the type of text and words that are likely to be spoken.
ISRGramNotifySink | Notifies an application about events related to speech recognition with a grammar. Notifications indicate that the engine has encountered a bookmark; that the user has started a phrase or the engine has recognized a phrase; that the engine is paused; that a speech-recognition results object was reevaluated or retrieved when an archived grammar was loaded; or that training is recommended.


Purpose

Represents a speech-recognition mode of an engine, which processes and recognizes incoming speech. An engine typically provides an assortment of speech-recognition modes that can be used to recognize speech in different languages, dialects, or audio sampling rates. The engine object is supplied by the engine vendor.


Created by

To create an engine object, you first use a speech-recognition enumerator to locate the mode to use, and then call the ISREnum::Select member function with the GUID of the mode and the address of the IUnknown interface of an audio-source object.

As the engine object initializes, it calls the IAudio interface on the audio-source object to determine whether or not it can use that audio source. If so, the engine object calls the IAudio::PassNotify member function with an IAudioSourceNotifySink notification interface so the audio-source object can notify the engine of events that pertain to incoming audio. If the engine cannot use the specified audio source, the Select member function returns an error and the engine object releases itself.

If successful, the Select member function returns the address of the ISRCentral interface on the newly created engine object.


Interacts with

The engine object uses the rules, lists of words, or context provided by all active grammar objects to parse incoming speech and recognize it.

The engine object receives notifications from the audio-source object about the status of incoming audio, such as whether audio data is available.

The engine object can notify the application of events that pertain to speech recognition, such as interference in the audio signal or the beginning and end of an utterance. To receive notifications, the application calls the ISRCentral::Register member function on the engine object and passes the address of the notification interface. The application must create a COM object that supports ISRNotifySink as an interface. How the application responds to notifications is up to the application developer.


Interfaces

The engine object should support the ISRAttributes, ISRCentral, and ISRDialogs interfaces, and it may also support the ILexPronounce and ISRSpeaker interfaces. An engine object may support other interfaces defined by the engine vendor to provide additional functionality.

Interface | Description
ILexPronounce | (Optional.) Allows an application to query and control the pronunciation lexicon for a speech-recognition or text-to-speech engine.
ISRAttributes | Controls the attributes of a speech-recognition engine. Member functions allow the application to adjust the engine's automatic gain, echo canceling, energy floor, microphone information, share of the processor, speaker, recognition threshold, and time-out values.
ISRCentral | Controls an engine object. Member functions allow an application to load a grammar, get information about the speech-recognition mode, pause and resume the engine, register or release a notification interface, get the time that a byte in the audio stream was received, and convert the time to a Win32 FILETIME structure.
ISRDialogs | Displays Windows dialog boxes that allow the user to configure the speech-recognition engine, such as training the engine for the microphone or the speaker's voice.
ISRNotifySink | Notifies an application of engine-specific events related to recognizing speech, such as a change of attributes, interference or sound, the beginning and end of an utterance, and the VU meter reading for the incoming audio stream.
ISRSpeaker | (Optional.) Manages speaker profile information. Member functions allow an application to query, create, delete, and merge speaker profiles and transfer a speaker profile from one system to another.


Purpose

Enumerates the speech-recognition modes supported by a particular engine. An application does not interact with the engine enumerator directly; instead, it uses the speech-recognition enumerator, which passes function calls to the engine enumerator. The engine enumerator is supplied by the engine vendor.


Created by

The engine enumerator is created when the application calls the ISRFind::Find member function on a speech-recognition enumerator. The application passes the address of an SRMODEINFO structure that indicates the desired characteristics of the mode and the address of an SRMODEINFORANK structure that indicates the relative importance of each characteristic. As an alternative, the application can call member functions of the ISREnum interface to enumerate the modes.

The speech-recognition enumerator creates an engine enumerator for each engine registered on the system and passes the call to each engine enumerator.


Interacts with

When an engine enumerator receives a Find call from the speech-recognition enumerator, the engine enumerator searches its list of speech-recognition modes for the closest match to the characteristics specified with Find. (If the engine enumerator does not support the ISRFind interface, it uses the ISREnum interface to scan the list.)

If the engine enumerator locates an appropriate speech-recognition mode, it returns the address of an SRMODEINFO structure that contains information about the mode, including its GUID. The application then calls the ISRFind::Select member function with this GUID to select the mode and create an engine object.

Once an engine object exists, the engine enumerator is no longer needed. The engine enumerator is released automatically when the application releases the speech-recognition enumerator.


Interfaces

The engine enumerator should support the ISREnum interface and may support the ISRFind interface. The engine enumerator may support other interfaces defined by the engine vendor to provide additional functionality.

Interface | Description
ISREnum | Enumerates and selects a speech-recognition mode.
ISRFind | (Optional.) Finds and selects the speech-recognition mode that most closely matches the requested characteristics.


Speech-Recognition Enumerator


Purpose

Enumerates the speech-recognition modes provided by all of the engines available to the application. The application uses the speech-recognition enumerator to select a speech-recognition mode and create an engine object to represent that mode within the application. The speech-recognition enumerator is supplied by Microsoft.


Created by

To create a speech-recognition enumerator, call CoCreateInstance function with the CLSID_SREnumerator class identifier and the IID_ISRFind or IID_ISREnum interface identifier, depending on the interface desired.

CoCreateInstance returns the address of the requested interface on the newly created speech-recognition enumerator.


Interacts with

When a speech-recognition enumerator is created, it loads the registry entries for each engine installed on the system and creates an engine enumerator object to represent each engine.

The application calls the ISRFind::Find member function on the speech-recognition enumerator with the address of an SRMODEINFO structure that indicates the desired characteristics of the mode and the address of an SRMODEINFORANK structure that indicates the relative importance of each characteristic. As an alternative, the application can call member functions of the ISREnum interface to enumerate the modes.

The speech-recognition enumerator passes these calls to each engine enumerator until one of them returns a mode that fits the requested characteristics.

After the application has created an engine object, the speech-recognition enumerator is no longer needed and the application can release it.


Interfaces

The speech-recognition enumerator supports the following interfaces.

Interface | Description
ISREnum | Enumerates and selects a speech-recognition mode.
ISRFind | Finds and selects the speech-recognition mode that most closely matches the requested characteristics.


Speech-Recognition Results Object


Purpose

Provides detailed information about a speech-recognition event. The speech-recognition results object is optional and may or may not be provided by the engine vendor.


Created by

If an engine supports speech-recognition results objects, it creates a speech-recognition results object each time a phrase is recognized. The engine returns the address of the IUnknown interface on the results object with the corresponding ISRGramNotifySink::PhraseFinish or ISRGramNotifySink::PhraseHypothesis notification for the recognized phrase.

The engine doesn't fully create the results object until the application calls an interface on the object. If the application ignores the notification, the engine releases the results object automatically. If the application calls an interface on the results object — for example, IUnknown::QueryInterface — the engine finishes creating the results object and it is up to the application to release it.


Interacts with

An application can query a speech-recognition results object to get detailed information about a recognition. The information that is available depends on the engine, but it may include a digital-audio recording, features extracted from the audio stream, the most likely phrases recognized, and the name of the speaker.

An application can also use a results object to perform certain tasks, such as training the engine to a particular speaker's voice or reevaluating a recognition result.

The information in a results object can occupy a significant amount of memory, so an application may instruct the object to discard certain parts of its data. In addition, an engine vendor may permit the results object to "deteriorate" or discard data automatically as time passes. An application can preserve speech-recognition results objects by locking them, or it can store results objects by archiving them with the associated grammar.


Interfaces

The speech-recognition results object should support the ISRResBasic interface, and may also support any of the following interfaces. The results object for an engine may support other interfaces defined by the engine vendor to provide additional functionality.

Interface | Description
ISRResAudio | (Optional.) Gets an audio recording of an utterance from an engine.
ISRResBasic | Provides general information about a speech-recognition results object. Member functions allow an application to obtain the phrase that was recognized or one of its alternatives, the time that a phrase began and ended, and the grammar that was involved in the recognition.
ISRResCorrection | (Optional.) Confirms that an utterance was recognized correctly, or provides the correct interpretation of an incorrectly recognized utterance.
ISRResEval | (Optional.) Reevaluates the recognition.
ISRResGraph | (Optional.) Provides information from a recognition/alternative graph generated by an engine. Member functions allow an application to obtain the most likely recognition path of all those attempted by the engine, the score for a path, or the phoneme or word for a particular node in a path.
ISRResMemory | (Optional.) Gets the amount of memory used by a particular kind of information in a recognition/alternative graph, or frees memory used by the information.
ISRResMerge | (Optional.) Merges information from two temporally adjacent speech-recognition results objects into a new object.
ISRResModifyGUI | (Optional.) Allows an application to display a small status window next to selected text for specific recognition results.
ISRResScore | (Optional.) Provides scores for recognition results.
ISRResSpeaker | (Optional.) Identifies the speaker assumed by the engine and allows an application to correct or validate the speaker.


Speech-Recognition Sharing Object


Purpose

Enumerates shared engine/audio-source pairs, or creates new ones. Allows an application to share a particular engine and audio source with other applications or with other speech-recognition processes within the same application.

An application can use a speech-recognition sharing object to accomplish the following:

· Share an engine and audio source with other applications, or with voice commands within the same application.

· Provide dictation services and command and control at the same time, if the engine supports both of these capabilities.

The speech-recognition sharing object has certain limitations that an application developer should consider:

· The object must be in a separate application, which requires marshaling and therefore is slower than a direct connection to a DLL.

· Any application can influence the recognition accuracy, confidence, and other speech-recognition attributes for a shared engine, which affects all applications sharing that engine.

The speech-recognition sharing object is supplied by Microsoft. Use of this object is optional.


Created by

To create a speech-recognition sharing object, call the CoCreateInstance function with the CLSID_SRShare class identifier and an IID_IEnumSRShare interface identifier.

The CoCreateInstance function returns the address of an IEnumSRShare interface on the newly created speech-recognition sharing object.


Interacts with

The application enumerates the available engine/audio-source pairs by calling member functions of the IEnumSRShare interface on the speech-recognition sharing object. These member functions return addresses of SRSHARE structures with information about each pair, including its instance identifier.

To select an existing pair, the application calls the IEnumSRShare::Share member function with the instance identifier of the desired pair.

To create a new pair, the application calls the IEnumSRShare::New member function with the GUID of a speech-recognition mode and the device identifier of an audio source that is not currently in use.

Both Share and New return addresses of the ISRCentral interface on the engine of the pair. After obtaining this interface, the application interacts with the engine as though it had connected directly with it and can release the speech-recognition sharing object.

When a speech-recognition sharing object is used to share an audio source among several engines, the object provides exclusive rights to the source to one engine at a time on a first-come, first-served basis. An engine requests the audio source by calling the IAudio::Start member function. If more than one engine requests the audio source at a time, the first engine to call IAudio::Start is awarded exclusive use of the source until that engine calls IAudio::Stop.


Interface

The speech-recognition sharing object supports the IEnumSRShare interface, which enumerates, selects, or creates a new shared engine/audio-source pair.


Interfaces and Member Functions

Speech-recognition interfaces allow an application to recognize the user's speech using speech-recognition engines and grammars. These interfaces include the following:

Interface | Description
IEnumSRShare | Enumerates, selects, or creates a new shared engine/audio-source pair.
ISRAttributes | Controls the attributes of a speech-recognition engine.
ISRCentral | Controls recognition of speech by a speech-recognition engine, including loading a grammar, pausing and resuming the engine, getting information about the engine or the speech being received, and registering a notification sink.
ISRDialogs | Displays Windows dialog boxes that allow the user to configure the speech-recognition engine.
ISREnum | Enumerates and selects a speech-recognition mode.
ISRFind | Finds and selects the speech-recognition mode that most closely matches the requested characteristics.
ISRGramCFG | Controls a context-free grammar.
ISRGramCommon | Controls features common to both context-free and dictation grammars.
ISRGramDictation | Controls a dictation grammar.
ISRGramInsertionGUI | Displays a small status window next to the insertion point in a document or edit field.
ISRGramNotifySink | Notifies an application that the speaker has finished a phrase, that the engine has a hypothesis about a phrase, or that the engine needs training from the current speaker.
ISRNotifySink | Notifies an application of interference, noise, a possible change of speaker, the beginning or end of an utterance, or the current VU level.
ISRResAudio | Gets an audio recording of an utterance from an engine.
ISRResBasic | Provides general information about a speech-recognition results object.
ISRResCorrection | Confirms that an utterance was recognized correctly, or provides the correct interpretation of an incorrectly recognized utterance.
ISRResEval | Reevaluates the recognition.
ISRResGraph | Provides information from a recognition/alternative graph generated by an engine.
ISRResMemory | Gets the amount of memory used by a particular kind of information in a recognition/alternative graph, or frees memory used by the information.
ISRResMerge | Merges information from two temporally adjacent speech-recognition results objects into a new object.
ISRResModifyGUI | Displays a small status window next to selected text for specific recognition results.
ISRResScore | Provides scores for recognition alternatives.
ISRResSpeaker | Identifies the speaker for a recognition.
ISRSpeaker | Transfers speaker information from one system to another.

This section describes, in alphabetical order, the interfaces and member functions that support speech recognition.

In addition to the interfaces described in this section, each object supports the IUnknown interface, which is a common interface supported by all OLE component objects. A speech-recognition engine may also support ILexPronounce, an optional interface that allows a user to correct words in the engine's pronunciation lexicon. For more information about IUnknown and ILexPronounce, see the section, "Common Elements of the Speech Application Programming Interface."


IEnumSRShare Interface

The IEnumSRShare interface allows an application to enumerate the current engine/audio-source pairs and either select an existing pair or create a new pair.

This interface is supported by the speech-recognition sharing object. It has these member functions:

Clone | Reset
Detach | Share
New | Skip
Next |

All OLE enumeration interfaces contain the Clone, Next, Reset, and Skip member functions. In the IEnumSRShare interface, these member functions use SRSHARE structures as the elements in the enumeration sequence.


IEnumSRShare::Detach

HRESULT Detach(QWORD qwInstanceID);

Detaches from an engine/audio-source pair. An application must call this member function to release a speech-recognition engine; it cannot release the engine interface itself. It will decrement the ref count on that it holds on this engine in the sharing objects list of engines, and if no other application is using this engine, it will release any ISRCentral interfaces it is holding.

· Returns NOERROR if successful, or the VCMDERR_NOENGINE error value.

qwInstanceID

[in] Instance identifier of the engine/audio-source pair to be detached from.


IEnumSRShare::New

HRESULT New(DWORD dwDeviceID, GUID gModeID,

PISRCENTRAL *ppISRCentral, QWORD *pqwInstanceID);

Creates a new engine/audio-source pair that can be shared by other applications.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDMODE
SRERR_OUTOFDISK
SRERR_WAVEFORMATNOTSUPPORTED

dwDeviceID

[in] Device identifier of the wave-in audio device to use. If the device is already in use, an error is returned. An application can obtain a list of devices by calling the waveInGetNumDevs and waveInGetDevCaps Windows functions.

gModeID

[in] GUID that identifies the desired speech-recognition mode. An application can use a speech-recognition enumerator to obtain the mode.

ppISRCentral

[out] Address of a variable that receives a pointer to the ISRCentral interface for the engine object of the newly created engine/audio-source pair.

pqwInstanceID

[out] Address of a variable that receives the instance identifier of the newly created engine/audio-source pair. Use this identifier later in a call to Detach to detach from this instance of the engine/audio-source pair.


IEnumSRShare::Share

HRESULT Share(QWORD qwInstanceID, PISRCENTRAL *ppISRCentral);

Allows an application to share an existing engine/audio-source pair.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_VALUEOUTOFRANGE

qwInstanceID

[in] Instance identifier of the desired engine/audio-source pair from the SRSHARE structure. An application can obtain this from the speech-recognition sharing object.

ppISRCentral

[out] Address of a variable that receives a pointer to the ISRCentral interface for the shared engine/audio-source pair.

The instance identifier of an engine/audio-source pair is guaranteed to be unique during the run-time session of Windows. Because Win32 handles multitasking, there's a slight chance that the instance identifier will become invalid between the time the engine/audio-source pair was enumerated and the time when the engine object tries to share it. If an instance identifier is invalid, the function returns an error. Always check the result of this function and handle errors appropriately.


ISRAttributes Interface

The ISRAttributes interface allows the user of an engine to control various attributes and parameters of speech recognition. Initially, attribute values are set to an optimal level as defined by the engine. The application can change these values or maintain them.

This interface is supported by the speech-recognition engine object. It has these member functions:

AutoGainEnableGet | RealTimeGet
AutoGainEnableSet | RealTimeSet
EchoGet | SpeakerGet
EchoSet | SpeakerSet
EnergyFloorGet | ThresholdGet
EnergyFloorSet | ThresholdSet
MicrophoneGet | TimeOutGet
MicrophoneSet | TimeOutSet


ISRAttributes::AutoGainEnableGet

HRESULT AutoGainEnableGet(DWORD *pdwAutoGain);

Retrieves the state of the automatic gain for the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwAutoGain

[out] Address of a variable that receives the automatic gain.

A value of 0 indicates that automatic gain is disabled. A value of 100 indicates that the speech-recognition engine always adjusts the gain, so that if the next utterance is spoken at the same level, the gain is set perfectly. A value between 0 and 100 moderates the automatic adjustments on a linear scale. For example, a pdwAudioGain value of 50 adjusts the gain to half the extent that the engine otherwise would.

When automatic gain is enabled, the speech-recognition engine may adjust the level at the end of an utterance and increase or decrease the gain. The engine adjusts the gain by using the IAudio::LevelGet and IAudio::LevelSet member functions to communicate with the audio-source object that represents the incoming audio stream.

If the audio source does not support IAudio::LevelSet, the application cannot set the automatic gain.


ISRAttributes::AutoGainEnableSet

HRESULT AutoGainEnableSet(DWORD dwAutoGain);

Sets the state of the automatic gain for the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwAutoGain

[in] Automatic gain value. This parameter can be SRATTR_MINAUTOGAIN or SRATTR_MAXAUTOGAIN for minimum or maximum allowable values, respectively.

A value of 0 disables automatic gain. A value of 100 causes the speech-recognition engine to always adjust the gain. Values between 0 and 100 moderate the gain on a linear scale. If the value is out of range for the engine, an error is returned and the gain is not changed.


ISRAttributes::EchoGet

HRESULT EchoGet(BOOL *pfEchoOn);

Retrieves the state of echo canceling on the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pfEchoOn

[out] Address of a variable that receives the state of echo canceling. The variable receives TRUE if the engine expects echo canceling on the incoming audio stream, or FALSE if the engine expects a clean signal that hasn't undergone echo canceling. When pfEchoOn is TRUE, the engine typically ignores low-level audio signals.

The state of echo canceling indicates whether the signal should be treated as though it has residual signal on it. In general, echo canceling causes the speech-recognition engine to ignore low-level audio signals so that it doesn't attempt to recognize the residual signal of the echo cancellation.


ISRAttributes::EchoSet

HRESULT EchoSet(BOOL fEchoOn);

Sets the state of echo canceling on the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

fEchoOn

[in] State of echo canceling. TRUE turns on echo canceling, FALSE turns it off.


ISRAttributes::EnergyFloorGet

HRESULT EnergyFloorGet(WORD *pwEnergy);

Retrieves the value of the current noise floor expected by a speech-recognition engine from the audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pwEnergy

[out] Address of a variable that receives the expended energy of the noise floor, in negative decibels. For example, a value of 30 indicates an energy floor of –30 dB.

The noise floor is the noise value in the signal-to-noise ratio (SNR) for a particular environment. In general, the higher the noise value in the ratio, the more sensitive the speech-recognition engine is to background noise. For example, a quiet office would have a high SNR (low floor), whereas a noisy factory would have a low SNR (high floor).

An application might send this value to a speech-recognition engine if the application has information about the SNR to expect from the audio stream. The engine can use the value internally to adjust its calculated noise floor in expectation of the audio stream's SNR.


ISRAttributes::EnergyFloorSet

HRESULT EnergyFloorSet(WORD pwEnergy);

Sets the value of the current noise floor expected by a speech-recognition engine from the audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pwEnergy

[in] Expended energy of the noise floor, in negative decibels. For example, a value of 30 indicates an energy floor of –30 dB. An application can specify SRATTR_MINENERGYFLOOR or SRATTR_MAXENERGYFLOOR for minimum or maximum allowable values.


ISRAttributes::MicrophoneGet

HRESULT MicrophoneGet(TCHAR *pszMicrophone, DWORD dwMicrophoneSize,

DWORD *pdwNeeded);

Retrieves the name of the current microphone for this audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszMicrophone

[in/out] Address of a buffer that receives the name of the current microphone for an audio source. If the buffer receives an empty string, the current microphone is unknown.

dwMicrophoneSize

[in] Size, in bytes, of the buffer specified by pszMicrophone. If the specified size is too small, the function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the microphone string.

An application can use the microphone name to save or retrieve information about a particular microphone, such as the name of the microphone used to train the engine or the conditions in which the microphone is recording. For example, an application could use this information to preserve the original training when the user changes microphones.


ISRAttributes::MicrophoneSet

HRESULT MicrophoneSet(TCHAR *pszMicrophone);

Sets the name of the current microphone for an audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszMicrophone

[in] Address of a buffer that contains the name of the microphone to set. If the current microphone is unknown, pszMicrophone can be an empty string.


ISRAttributes::RealTimeGet

HRESULT RealTimeGet(DWORD *pdwRealTime);

Retrieves the real-time setting for the engine. The real-time setting is the percentage of processor time that the engine expects to use during constant speech.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwRealTime

[out] Address of a variable that receives the real-time setting. This value can be more than 100 for non-real-time applications (for example, transcribing prerecorded speech).

For most engines, the amount of processor time required diminishes markedly during periods of silence. For example, if pdwRealTime is 100, the engine takes one full minute of processor time to process one minute of speech. If pdwRealTime is 50, the engine takes 30 seconds of processor time to process the same minute of speech. This value is difficult to compute precisely, so it should be regarded as an estimate.


ISRAttributes::RealTimeSet

HRESULT RealTimeSet(DWORD dwRealTime);

Sets the real-time setting for the engine. The real-time setting is the percentage of processor time that the engine expects to use during constant speech.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwRealTime

[in] New real-time setting. An application can specify SRATTR_MINREALTIME or SRATTR_MAXREALTIME for minimum or maximum allowable values. If the value is out of range for the engine, the function returns an error and the real-time setting is not changed.

The engine tries to meet the real-time expectation, but the engine will not be able to consume exactly dwRealTime percentage of the CPU.


ISRAttributes::SpeakerGet

HRESULT SpeakerGet(TCHAR *pszSpeaker, DWORD dwSpeakerSize,

DWORD *pdwNeeded);

Retrieves the name of the current speaker for an audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszSpeaker

[in/out] Address of a buffer that receives the name of the current speaker. If the buffer receives an empty string, the current speaker is unknown.

dwSpeakerSize

[in] Size, in bytes, of the buffer specified by pszSpeaker. If the specified size is too small, the function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the speaker string.


ISRAttributes::SpeakerSet

HRESULT SpeakerSet(TCHAR *pszSpeaker);

Sets the name of the current speaker for an audio source.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pszSpeaker

[in] Address of a buffer that contains the speaker's name to set. If pszSpeaker is a new name, the engine creates a new speaker profile. If the speaker is unknown, pszSpeaker can be an empty string. The string is case insensitive.


ISRAttributes::ThresholdGet

HRESULT ThresholdGet(DWORD *pdwThreshold);

Retrieves the threshold level of the speech-recognition engine. The threshold level indicates how confident the speech-recognition engine must be about the validity of a given recognition in order to accept it. If the engine is not confident enough, it notifies the application that the phrase was not recognized.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwThreshold

[out] Address of a variable that receives the threshold.

The threshold level is a value from 0 to 100 that indicates the threshold below which an utterance is rejected as unrecognized. A value of 0 indicates that the engine matches any utterance to the closest phrase match. A value of 100 indicates that the engine must be absolutely certain that an utterance is the recognized phrase. The optimum threshold for the average user should be 50.


ISRAttributes::ThresholdSet

HRESULT ThresholdSet(DWORD dwThreshold);

Sets the threshold level for a speech-recognition engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwThreshold

[in] Threshold level. If the value is out of range for the engine, an error is returned and the attribute is not changed. An application can specify SRATTR_MINTHRESHOLD or SRATTR_MAXTHRESHOLD for minimum or maximum allowable values.

The threshold level is a value from 0 to 100 that indicates the threshold below which an utterance is rejected as unrecognized. A value of 0 indicates that the engine matches any utterance to the closest phrase match. A value of 100 indicates that the engine must have determined absolutely that an utterance is the recognized phrase. The optimum threshold for the average user should be 50.


ISRAttributes::TimeOutGet

HRESULT TimeOutGet(DWORD *pdwIncomplete, DWORD *pdwComplete);

Retrieves the current time-out values for a speech-recognition engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwIncomplete

[out] Address of a variable that receives the number of milliseconds that the speech-recognition engine waits before discarding an incomplete phrase because the user has stopped speaking. For example, if pdwIncomplete is 2000, a user speaking "Send mail to " could pause for 2 seconds before the engine would assume that the user has stopped speaking the phrase.

pdwComplete

[out] Address of a variable that receives the number of milliseconds that the speech-recognition engine waits before regarding a phrase as complete after the user has stopped speaking. For example, if pdwComplete is 500, a user speaking "Send mail to Fred" would see results 0.5 seconds after finishing the phrase.


ISRAttributes::TimeOutSet

HRESULT TimeOutSet(DWORD dwIncomplete, DWORD dwComplete);

Sets the time-out values for a speech-recognition engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwIncomplete

[in] Number of milliseconds that the speech-recognition engine should wait before discarding an incomplete phrase because the user has stopped speaking. An application can specify SRATTR_MINTOINCOMPLETE or SRATTR_MAXTOINCOMPLETE for minimum or maximum allowable values.

dwComplete

[in] Number of milliseconds that the speech-recognition engine should wait before regarding a phrase as complete after the user has stopped speaking. An application can specify SRATTR_MINTOCOMPLETE or SRATTR_MAXTOCOMPLETE for minimum or maximum allowable values.

If a time-out value is set to a value that is out of range for the engine, an error is returned and the attribute is not changed.


ISRCentral Interface

The ISRCentral interface provides centralized control of a speech-recognition engine, including loading a grammar, pausing and resuming the engine, getting information about the engine or the speech being received, and registering a notification sink.

This interface is supported by the speech-recognition engine object. It has these member functions:

GrammarLoad | Register
ModeGet | Resume
Pause | ToFileTime
PosnGet | UnRegister


ISRCentral::GrammarLoad

HRESULT GrammarLoad(SRGRMFMT eFormat, SDATA dData,

PVOID pNotifyInterface, IID IIDNotifyInterface,

PIUNKNOWN *ppiUnknown);

Loads a grammar that was archived in a data buffer, or that was just created, and returns a grammar object. The application uses the grammar object to activate the grammar and begin recognizing speech.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_GRAMMARERROR
SRERR_GRAMMARTOOCOMPLEX
SRERR_GRAMMARWRONGTYPE
SRERR_GRAMTOOLARGE
SRERR_INVALIDCHAR
SRERR_INVALIDINTERFACE
SRERR_NOTSUPPORTED
SRERR_TOOMANYGRAMMARS

eFormat

[in] Format of the data in dData. This parameter can be one of these values:

Value | Meaning
SRGRMFMT_CFG | The data conforms to the standard context-free grammar format.
SRGRMFMT_DICTATION | The data conforms to the standard dictation grammar format.
SRGRMFMT_LIMITEDDOMAIN | The data conforms to the standard limited-domain grammar format.
SRGRMFMT_CFGNATIVE | The data is in the engine's native format for context-free grammars.
SRGRMFMT_DICTATIONNATIVE | The data is in the engine's native format for dictation grammars.
SRGRMFMT_LIMITEDDOMAINNATIVE | The data is in the engine's native format for limited-domain grammars.

dData

[in] Address of an SDATA structure that contains the data to be passed to the speech-recognition engine. If the grammar is new and has not been previously archived, this should contain the grammar's tree. If the grammar has been previously archived, the engine may have written the grammar in its own native format to a data buffer.

As an alternative, the pData member of the SDATA structure can point to a memory-mapped file so that the application doesn't need to load all of the data into memory at once. However, it is up to the application to call the CreateFileMapping, UnmapViewOfFile, and CloseHandle functions.

pNotifyInterface

[in] Address of the notification interface through which the grammar notifies the application about recognition information. The interface identifier is specified by IIDNotifyInterface. This parameter can be NULL if the application doesn't require notification.

Because passing the address does not transfer ownership of the notification interface, the grammar must call the AddRef member function of the notification interface before returning from the call to the GrammarLoad member function. When the grammar closes, it must also call the Release member function of the notification interface. The application must release any reference counts it holds on the notification interface after calling GrammarLoad, unless it needs the notification object to be valid when the grammar releases it.

IIDNotifyInterface

[in] Identifier of the interface that receives notifications. Currently, the identifier must be IID_ISRGramNotifySinkA (for ANSI) or IID_ISRGramNotifySinkW (for Unicode). An application can use IID_ISRGramNotifySink to get ANSI or Unicode, depending on the default for the application. An engine can define additional values to support custom interfaces.

ppiUnknown

[out] Address of a variable that receives a pointer to the IUnknown interface for the newly created grammar object. If an error occurs, the function sets this parameter to NULL.

If the grammar is new and has not been archived, the application should pass a data buffer that contains the grammar's tree. If the grammar was archived and the ISRGramCommon::Archive member function returned data in native format for an engine that is still running, the application should pass that data.

If the grammar was archived with speech-recognition results objects (that is, with the fArchiveResults parameter of the Archive member function set to TRUE), the grammar object calls the ISRGramNotifySink::UnArchive member function for each results object previously archived with the grammar.

Loading a standard-format grammar (SRGRMFMT_CFG, SRGRMFMT_LIMITEDDOMAIN, or SRGRMFMT_DICTATION) may take some time — up to several seconds — because the speech-recognition engine may need to compile the grammar into its own native format. After loading such a grammar, the application should call the ISRGramCommon::Archive member function to store the grammar in native format so it can be used again later. For more information about standard-format grammars, see Appendix B, "Standard Grammar Formats."

Loading a grammar in native format (SRGRMFMT_CFGNATIVE, SRGRMFMT_DICTATIONNATIVE, or SRGRMFMT_LIMITEDDOMAINNATIVE) is faster than doing so in standard formats because the engine has already processed the information. Some speech-recognition modes may use the same grammar format so an application can use that format for multiple modes without having to recompile the grammar. If the grammar is not in the engine's native format, the engine returns an error.

If the grammar cannot be loaded because it is too complex or too large, or if it requires features not provided by the engine, the GrammarLoad member function returns an error.

When a grammar object is released, the grammar is automatically deactivated.

Some engines accept the grammar data as is and create a separate thread to process the data. If the data contains an error, calls to functions in the grammar interfaces (ISRGramCFG, ISRGramCommon, or ISRGramDictation) may fail. If a call to a function requires that the data be completely processed, the function doesn't execute until after the data is processed.


ISRCentral::ModeGet

HRESULT ModeGet(PSRMODEINFO pModeInfo);

Retrieves information about the current speech-recognition mode.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pModeInfo

[out] Address of an SRMODEINFO structure that receives information about the current speech-recognition mode.


ISRCentral::Pause

HRESULT Pause(void);

Pauses the speech-recognition engine, incrementing the pause count. The application must call the Resume member function after each call to the Pause member function.

Retrieves information specific to the current speech-recognition mode.

· Returns NOERROR if successful, or E_OUTOFMEMORY if a low-memory condition occurs.

Pausing an engine prevents it from processing future phrases but not from identifying the beginning and ending of utterances.

Calling this function during an utterance pauses the processing of digital audio. This function affects the notifications received through the ISRGramNotifySink interface, and may affect notifications received through the ISRNotifySink::VUMeter member function.

The Pause member function may be called at different times by different applications that share the same engine, or it may be called automatically when a PhraseFinish notification is sent. For this reason, the Pause function maintains a pause count that is incremented any time the function is called. The engine starts with a pause count of 0.

If an engine is paused more than a second on a real-time audio stream, it may start losing information from the audio source because internal buffers aren't large enough to accumulate that much audio.


ISRCentral::PosnGet

HRESULT PosnGet(QWORD *pqwTimeStamp);

Retrieves the exact byte in the audio stream that is currently being received (or as exact as possible given the audio device).

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_VALUEOUTOFRANGE

pqwTimeStamp

[out] Address of a variable that receives the byte in the audio stream that is currently being received.

This member function works only for real-time audio sources such as microphones; it does not work for sources such as files.

An application can use this member function to synchronize actions with events. For example, if the user clicks a mouse button, an application can call PosnGet to retrieve the byte of audio being played at that moment and later determine whether or not the mouse click occurred before or after that particular byte.


ISRCentral::Register

HRESULT Register(PVOID pNotifyInterface, IID IIDNotifyInterface,

DWORD *pdwKey);

Registers an ISRNotifySink interface for the engine to use to notify the application of grammar-independent recognitions. The application must call the UnRegister member function for each Register call.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDINTERFACE
SRERR_VALUEOUTOFRANGE

pNotifyInterface

[in] Address of the notification interface. The interface identifier is specified by IIDNotifyInterface.

Because passing the address to the engine does not transfer ownership of the notification interface, the engine must call the AddRef member function of the notification interface before returning from the call to Register. The engine must also call the Release member function of the notification interface when the ISRCentral::Unregister member function is called for the given key. The application must release any reference counts it holds on the notification interface after calling Register, unless it needs the notification object is to be valid after calling Unregister.

IIDNotifyInterface

[in] Identifier of the interface used for notification. Currently, the identifier must be IID_ISRNotifySinkA (for ANSI) or IID_ISRNotifySinkW (for Unicode. An application can use IID_ISRNotifySink to get ANSI or Unicode, depending on the default for the application. An engine can define additional values to support custom interfaces.

pdwKey

[out] Address of a variable that receives a key for the notification interface. The application must specify this key when calling the UnRegister member function to release the interface.

Grammar-dependent recognitions are notified through an ISRGramNotifySink interface, which is passed to a grammar object at the time the object is created. For more information, see the description of the ISRCentral::GrammarLoad member function.


ISRCentral::Resume

HRESULT Resume(void);

Reduces the pause count of a paused speech-recognition engine by 1. When the engine's pause count reaches 0, the engine resumes processing incoming phrases.

· Returns NOERROR if successful, or E_OUTOFMEMORY if a low-memory condition occurs.

An application must call Resume each time it pauses the engine, calls the Pause member function, or encounters a bookmark, and to resume after auto-pause activation.


ISRCentral::ToFileTime

HRESULT ToFileTime(PQWORD pqWord, FILETIME *pFT);

Converts a QWORD representing the time in bytes, to a Win32 FILETIME structure.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_VALUEOUTOFRANGE

pqWord

[in] Address of a variable that contains the time, in bytes, that was returned by the engine.

pFT

[out] Address of a FILETIME structure that receives the time.

An application can use the value returned by ToFileTime to correlate real-time events with a time value returned by the engine. The value returned by the engine is an estimate that should be accurate to within a few seconds of the present time, but becomes less accurate further away from the present due to time variations in the audio card's clock.


ISRCentral::UnRegister

HRESULT UnRegister(DWORD dwKey);

Allows an application to release a notification interface through which an engine sent grammar-independent notifications to an application. An application must call UnRegister before closing the engine object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDKEY

dwKey

[in] Key value returned by the Register member function for the notification interface when the application registered the interface.


ISRDialogs Interface

The ISRDialogs interface allows an application to display Windows dialog boxes to configure a speech-recognition engine. The controls and capabilities in these dialog boxes are determined by the particular speech-recognition engine that is being used. All dialogs supported by the engine are modal and calls to the ISRDialogs interface are synchronous. All of the ISRDialogs member functions use the same syntax.

This interface is supported by a speech-recognition engine object. It has these member functions:

AboutDlg | TrainMicDlg
GeneralDlg | TrainGeneralDlg
LexiconDlg |


ISRDialogs::AboutDlg

HRESULT AboutDlg(HWND hWndParent, PTSTR pszTitle);

Displays an About dialog box that identifies the speech-recognition engine and contains the copyright notice.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDWINDOW
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the dialog box displayed by AboutDlg, or SRERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


ISRDialogs::GeneralDlg

HRESULT GeneralDlg(HWND hWndParent, PTSTR pszTitle);

Displays a dialog box that gives the user general control of the speech-recognition engine. The dialog box may also include controls that allow the user to invoke other dialog boxes provided by the engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDWINDOW
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the dialog box displayed by GeneralDlg, or SRERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Depending on the engine, the GeneralDlg member function may allow the user to:

· Inform the engine of the speaker's gender or other speaker-specific information.

· Control how stringent the engine is about recognition.

· Control the maximum amount of disk space allotted for speaker information.

· Control optimizations that reduce processing.

· Remove speakers if there isn't enough disk space.

· Export and import speaker information to and from a disk so that this information can be transferred between systems.

· Access all of the standard dialog boxes.


ISRDialogs::LexiconDlg

HRESULT LexiconDlg(HWND hWndParent, PTSTR pszTitle);

Displays a dialog box that allows the user to view and edit his or her pronunciation lexicon. For example, the user can edit the phonetics of mispronounced words so that the engine pronounces them properly.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDWINDOW
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the dialog box displayed by LexiconDlg, or SRERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.


ISRDialogs::TrainGeneralDlg

HRESULT TrainGeneralDlg(HWND hWndParent, PTSTR pszTitle);

Displays a dialog box that allows the user to provide phonetic training, which trains the engine for a preselected set of words

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDWINDOW
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the dialog box displayed by TrainGeneralDlg, or SRERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Ideally, this dialog box trains the engine for the speaker's voice only, but it may also train for the microphone and any surrounding sounds.

If the engine does not supply this dialog box, the speaker cannot perform phonetic training; however, the application will still work.


ISRDialogs::TrainMicDlg

HRESULT TrainMicDlg(HWND hWndParent, PTSTR pszTitle);

Displays a dialog box that allows the user to train the engine for a particular microphone. This dialog box should be displayed at the request of the engine or whenever the speaker wants to display it.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDWINDOW
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window. If this parameter is NULL, the engine returns NOERROR if it supports the dialog box displayed by TrainMicDlg, or SRERR_NOTSUPPORTED if it does not.

pszTitle

[in] Address of the string to use as the title of the dialog box. This parameter can be NULL to direct the engine to provide a default title.

Training information is associated with both the speaker and the microphone. Ideally, the TrainMicDlg member function trains the engine only for the microphone and any surrounding sounds, but it may also train for the speaker's voice.

If the engine does not supply this dialog box, the speaker cannot train it for the microphone; however, the application will still work.


ISREnum Interface

The ISREnum interface allows an application to enumerate the different speech-recognition modes and to select a mode for which to create an engine object.

Calling ISREnum on an engine enumerator allows the application to iterate through the engine's list of known speech-recognition modes. Calling ISREnum on a speech-recognition enumerator calls engine enumerators for all of the speech-recognition engines on the system.

The ISREnum interface is provided by the speech-recognition enumerator and engine-enumerator objects. It has these member functions:

Clone | Select
Next | Skip
Reset |

All OLE enumeration interfaces contain the member functions Clone, Next, Reset, and Skip. In the ISREnum interface, these member functions use SRMODEINFO structures as the elements in the enumeration sequence. For more information about Clone, Next, Reset, and Skip, see the section, "Common Elements of the Speech Application Programming Interface."


ISREnum::Select

HRESULT Select(GUID gModeID, PISRCENTRAL *ppiSRCentral,

PIUNKNOWN pIUnknownForAudio);

Selects a speech-recognition mode and creates an engine object to represent the selected mode.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDMODE
SRERR_OUTOFDISK
SRERR_WAVEFORMATNOTSUPPORTED

gModeID

[in] GUID from the SRMODEINFO structure for the speech-recognition mode that is selected. The function creates an engine object to represent this mode.

ppiSRCentral

[out] Address of a variable that receives a pointer to the ISRCentral interface for the newly created engine object.

pIUnknownForAudio

[in] Address of the IUnknown interface for the audio-source object that provides digital-audio data for the engine object. The engine will AddRef the interface, and release the interface when the engine is destroyed. Because of this the application will need to call Release on the audio object after the select call or audio objects will be leaked.

After the application enumerates the speech-recognition modes and determines which one to use, it calls the Select member function to create the engine object.

As the engine object initializes, it calls the IAudio interface on the audio-source object specified by pIUnknownForAudio to determine whether or not it can use that audio source. If so, the member function returns an ISRCentral interface for the newly created engine object. If not, the engine object releases itself and the Select member function returns an error.

After the engine object is created, it calls the IAudio::PassNotify member function to pass its own notification sink to the audio-source object so that object notifies the engine when audio data is available. When all references to the engine object are released, the object calls the IAudio::Release member function.

The multimedia audio-source object can be used to obtain audio from a wave-in device. The Select member function call passes the IUnknown interface rather than the IAudioSource interface because, although all engines should support IAudioSource, some engines may talk to custom interfaces for dealing with special hardware or drivers and IUnknown can be used to obtain such interfaces. For more information about the multimedia audio-source object, see the section, "Multimedia Audio Objects ."

The pIUnknownForAudio parameter is obtained by using the CoCreateInstance function. As such, the reference count for the IUnknown interface begins at 1, and the interface should be released after the Select member function has returned.

The negotiation with the audio-source object occurs within the Select call, so if Select returns without an error, then the negotiation succeeded.


ISRFind Interface

The ISRFind interface allows an application to search through speech-recognition modes, to select a mode that most closely matches the desired characteristics, and to create an engine object that represents the selected mode.

Because ISRFind selects a speech-recognition mode based on the relative importance of various characteristics, it may not always select the optimal mode (just as the Windows font-matching function does not always select the optimal font). ISRFind is best used to select a default speech-recognition mode. An application should also allow the user to select a mode, and should store its GUID so that the mode can be retrieved later.

The ISRFind interface for the speech-recognition enumerator calls ISRFind for all of the registered speech-recognition engines and then selects the closest result. If an engine does not supply an ISRFind interface, the speech-recognition enumerator uses the engine's ISREnum interface to simulate this functionality.

The ISRFind interface is provided by the speech-recognition enumerator and engine-enumerator objects. It has these member functions:

Find

Select


ISRFind::Find

HRESULT Find(PSRMODEINFO psrInfo, PSRMODEINFORANK psrInfoRank,

PSRMODEINFO psrInfoFound);

Finds the speech-recognition mode that most closely matches the desired characteristics.

· Returns NOERROR if successful, or one of these error values:

E_FAIL
E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDMODE

psrInfo

[in] Address of an SRMODEINFO structure that contains the desired characteristics.

psrInfoRank

[in] Address of an SRMODEINFORANK structure that contains the relative importance of each characteristic. If this parameter is NULL, the object uses the default ranking provided by the engine.

psrInfoFound

[out] Address of an SRMODEINFO structure that receives information about the speech-recognition mode that was found. It is valid only if the Find member function does not return an error value.

If no engines can be found or the system runs out of memory, Find returns an error value.


ISRFind::Select

HRESULT Select(GUID gModeID, PISRCENTRAL *ppiSRCentral,

PIUNKNOWN pIUnknownForAudio);

Selects a speech-recognition mode and creates an engine object to represent the selected mode.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDMODE
SRERR_OUTOFDISK
SRERR_WAVEFORMATNOTSUPPORTED

gModeID

[in] GUID from the SRMODEINFO structure for the speech-recognition mode that is selected. The member function creates an engine object to represent this mode.

ppiSRCentral

[out] Address of a variable that receives a pointer to the ISRCentral interface for the newly created engine object.

pIUnknownForAudio

[in] Address of the IUnknown interface for the audio-source object that provides digital-audio data for the engine object. The engine will AddRef the interface, and release the interface when the engine is destroyed. Because of this the application will need to call Release on the audio object after the select call or audio objects will be leaked.

The pIUnknownForAudio parameter is obtained by using the CoCreateInstance function. As such, the reference count for the IUnknown interface begins at 1, and the interface should be released after the Select member function has returned.


ISRGramCFG Interface

The ISRGramCFG interface provides control of certain features that are specific to context-free grammars. This interface is not used by dictation grammars. For more general control of context-free and dictation grammars, use the ISRGramCommon interface.

The ISRGramCFG interface is supported by the grammar object. It has these member functions:

ListAppend | ListRemove
ListGet | ListSet
ListQuery |


ISRGramCFG::LinkQuery

HRESULT LinkQuery(TCHAR *pszLinkName, BOOL *pfExist);

Indicates whether an imported symbol used by this grammar has been linked to another grammar.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDRULE
SRERR_NOTSUPPORTED

pszLinkName

[in] Address of a buffer that contains the name of the link.

pfExist

[out] Address of a variable that receives TRUE if the link exists, or FALSE if it does not.

If the engine does not support links, it should return SRERR_NOTSUPPORTED.


ISRGramCFG::ListAppend

HRESULT ListAppend(TCHAR *pszListName, SDATA dWord);

Appends new words to an existing list.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDCHAR
SRERR_INVALIDLIST
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

pszListName

[in] Address of a buffer that contains the name of the list to which to append the words. If the list is not valid for the grammar, the member function returns an error value.

dWord

[in] Sequential list of SRWORD structures. (This is not an array, because the size of each structure varies according to its contents.)

If a word already exists in the list, the word's existing identifier is replaced with the new identifier. If a word does not already exist in the list, it is appended to the list.

If the engine does not support lists, it returns SRERR_NOTSUPPORTED.


ISRGramCFG::ListGet

HRESULT ListGet(TCHAR *pszListName, PSDATA pdWord);

Retrieves the list of words that the engine is currently using.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDLIST
SRERR_NOTSUPPORTED

pszListName

[in] Address of a buffer that contains the name of the list.

pdWord

[out] Address of an SDATA structure that receives information about the list. The dwSize member of the structure receives the number of bytes needed for the list information and the pData member receives the address of a memory object that contains a sequential list of SRWORD structures, one for each word in the list. (This is not an array, because the size of each structure varies with its contents.) It is not necessary to explicitly terminate the list; it is terminated implicitly because no more SRWORD structures fit in the specified amount of memory.

The application allocates the SDATA structure and passes its address to the ListGet member function. ListGet allocates memory (by using the OLE task allocator) for the returned data and sets the pData member of the SDATA structure to point to the memory. If the allocation fails, the pData member is set to NULL and the dwSize member is set to zero. The application must free the memory pointed to by pData, as well as the SDATA structure itself.

This member function allocates memory and fills in a list of words. If a list has not been set or is empty, the memory will be empty and the member function returns SRERR_INVALIDLIST.

If the engine does not support lists, it should return SRERR_NOTSUPPORTED.

The application must free the memory allocated by the member function by using the CoTaskMemAlloc and CoTaskMemFree functions.


ISRGramCFG::ListQuery

HRESULT ListQuery(TCHAR *pszListName, BOOL *fExist);

Indicates whether or not a list has had words set.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDLIST
SRERR_NOTSUPPORTED

pszListName

[in] Address of a buffer that contains the name of the list.

fExist

[out] Address of a variable that receives TRUE if the list exists, or FALSE if it doesn't.


ISRGramCFG::ListRemove

HRESULT ListRemove(TCHAR *pszListName, SDATA dWord);

Removes words from an existing list.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDCHAR
SRERR_INVALIDLIST
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

pszListName

[in] Address of a buffer that contains the name of the list.

dWord

[in] An SDATA structure that contains a sequential list of SRWORD structures. (This is not an array because the size of each structure varies according to its contents.)

If a specified word is not in the list, it is skipped. If the engine does not support lists, it returns SRERR_NOTSUPPORTED.


ISRGramCFG::ListSet

HRESULT ListSet(TCHAR *pszListName, SDATA dWord);

Fills in an empty list field in a grammar, or replaces an existing list.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDCHAR
SRERR_INVALIDLIST
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

pszListName

[in] Address of a buffer that contains the name of the list. If the list is already set for the grammar, the old list is discarded and the new list is loaded. If the list name is not valid for the grammar, an error is returned.

dWord

[in] An SDATA structure that contains a sequential list of SRWORD structures. (This is not an array because the size of each structure varies according to its contents.) The number of SRWORD structures is determined by the dwSize member of the SDATA structure.

The list provides the identified words. If a word from the list is spoken, the engine returns its word identifier just as if the word had been precompiled into the grammar.

A list can be used to "fill in the blanks" in a grammar with information that changes frequently, such as filenames or other entries. For example, a grammar might provide the sentence "Send mail to Names-List." For this sentence to be recognized correctly, the application should call the ListSet member function using Names-List and specify a list of e-mail names.

ListSet can be called at any time, and the newly set list takes effect immediately. Because of this flexibility, recognizing speech based on a list may be slower than a precompiled grammar and should be used only when precompiling is impossible.

If the engine does not support lists, it returns SRERR_NOTSUPPORTED.


ISRGramCommon Interface

The ISRGramCommon interface provides control of features that are common to both context-free and dictation grammars. For control of more specific features, see the ISRGramCFG and ISRGramDictation interfaces.

The ISRGramCommon interface is supported by the grammar object. It has these member functions:

Activate | DeteriorationSet
Archive | TrainDlg
BookMark | TrainPhrase
Deactivate | TrainQuery
DeteriorationGet |


ISRGramCommon::Activate

HRESULT Activate(HWND hWndListening, BOOL fAutoPause, PTCHAR pszInfo);

Activates a grammar so that the speech-recognition engine can begin using the grammar to recognize speech.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_GRAMMARTOOCOMPLEX
SRERR_INVALIDRULE
SRERR_INVALIDWINDOW
SRERR_RULEALREADYACTIVE

hWndListening

[in] Handle of the window for which this grammar should be used for recognition. This window is compared against the window returned by the GetForegroundWindow function at the beginning of the speech. If the windows match, the grammar is used; if they don't match, the grammar is not used. If hWndListening is NULL, the grammar is used continuously, which allows the application to recognize a global vocabulary. However, in case of a recognition conflict, a grammar whose hWndListening parameter is NULL has a lower priority than a grammar whose hWndListening parameter matches the foreground window. If the window is invalid, the member function returns E_INVALIDARG.

fAutoPause

[in] TRUE if the engine should automatically pause when this grammar is involved in a recognition (whether successful or unsuccessful) and send a PhraseFinish notification to the application. FALSE if the engine should not pause for this grammar. (The engine pauses once for any grammar for which this parameter is TRUE.) Even if more than one rule of a context-free grammar is active, this still increases the pause count by only 1.

pszInfo

[in] For a dictation or limited-domain grammar, this parameter is not used and should be NULL. For a context-free grammar, this parameter points to the name of the rule to activate. If the rule is already activated, this parameter is ignored. More than one rule can be active at a time; however, too many active rules can diminish recognition accuracy. If pszInfo is NULL for a context-free grammar, it activates the rule that has the rule name of "Start." If the rule is already active, the Activate member function returns an error.

Many engines restrict the number and type of grammars that can be active at the same time. Typically, an engine allows many context-free grammars but only one dictation or limited-domain grammar to be active at a time.

Because speaking is asynchronous, changes to the active grammar must be made before the speaker begins to speak. Changing the active grammar during speech may not have any effect until after the utterance ends. In addition, the engine may respond as if it has activated a grammar and postpone its real activation until an opportune time, such as when an utterance stops.

If any rule in a grammar has the auto-pause flag set, and the grammar has at least one rule active, the pause count is increased by 1 after a PhraseFinish notification. If multiple rules have the auto-pause flag set, the pause count is still increased only by 1. Likewise, only one ISRGramNotifySink::Paused notification is sent, even if more than one rule has the auto-pause flag set.


ISRGramCommon::Archive

HRESULT Archive(BOOL fArchiveResults, PVOID pBuf, DWORD dwBufSize,

DWORD *pdwBufNeeded);

Serializes a grammar object into memory, using a memory format that is native to the engine. The native format for the engine is stored in a data buffer.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDFLAG
SRERR_NOTSUPPORTED

fArchiveResults

[in] TRUE if the engine should archive all information in existing speech-recognition results objects, or FALSE if no results objects should be archived. Archiving the information in results objects may use a lot of memory, so an application should do this only if it needs the results objects for the next use of the application or document.

pBuf

[out] Address of a buffer that receives the archived data — the engine's native format and results objects, if any. As an alternative to a buffer, pBuf can point to memory in a memory-mapped file.

dwBufSize

[in] Number of bytes available in the buffer specified by pBuf. If the number is too small, Archive returns SRERR_NOTENOUGHDATA and fills pdwBufNeeded with the number of bytes needed.

pdwBufNeeded

[out] Number of bytes needed for the data to be archived.

If the engine does not support the grammar's standard format, Archive returns an error.

If fArchiveResults is TRUE, existing speech-recognition results objects are archived with the grammar. Information about lists is not archived. The application can save the archive data to disk or leave it in the buffer.

To reuse the grammar, the application calls the GrammarLoad member function and passes in the native format, which loads more quickly than standard format. If the archived grammar includes speech-recognition results objects, the engine calls the ISRGramNotifySink::UnArchive member function for each results object present when archive was called so the application can retrieve them.

While a grammar is activated, the speech-recognition engine may modify its engine-specific grammar structure as it acquires more information about the patterns of the speakers' speech. For this reason, the Archive member function should be called each time a grammar is deactivated and just before it is released, to ensure that the engine's modifications are preserved for the next use of the grammar.


ISRGramCommon::BookMark

HRESULT BookMark(QWORD qwTime, DWORD dwBookMarkID);

Sets a bookmark in the incoming audio stream.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

qwTime

[in] Time the bookmark is to take effect.

dwBookMarkID

[in] Bookmark identifier. The grammar passes this identifier back to the application with the ISRGramNotifySink::BookMark member function.

Setting a bookmark pauses the engine when it starts processing the audio at or after the time specified by qwTime. The time is in bytes and can be retrieved by calling the ISRCentral::PosnGet member function. If the bookmark takes effect before an utterance, it prevents the engine from recognizing the utterance. When a bookmark is encountered, the grammar notifies the application by calling the ISRGramNotifySink::BookMark member function followed by the ISRGramNotifySink::Paused member function.


ISRGramCommon::Deactivate

HRESULT Deactivate(PTCHAR pszInfo);

Deactivates an active grammar so that the speech-recognition engine no longer uses the grammar to recognize speech.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDRULE
SRERR_RULENOTACTIVE

pszInfo

[in] Address of a buffer that contains the name of the rule to deactivate for a context-free grammar. For a dictation or limited-domain grammar, pszInfo is not used and should be NULL. If the rule is already deactivated, pszInfo is ignored. If pszInfo is NULL, the context-free grammar deactivates the rule that has the rule name of "Start."

If the grammar is deactivated during an utterance, recognition might not stop until the utterance is finished. If this is the last active grammar to be deactivated, the engine should notify the audio-source object to stop providing digital-audio data. This saves resources and minimizes processor consumption.


ISRGramCommon::DeteriorationGet

HRESULT DeteriorationGet(DWORD *pdwMemory, DWORD *pdwTime,

DWORD *pdwObjects);

Retrieves the values used by the engine to determine how long to retain data stored in speech-recognition results objects.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwMemory

[out] Address of a variable that receives the amount of memory allowed for storage of speech-recognition results objects in this grammar object. If the memory required to store the existing results objects is larger than pdwMemory, older results objects begin to deteriorate.

pdwTime

[out] Address of a variable that receives the number of seconds a results object is allowed to exist in this grammar object. If a results object is older than the number of seconds it is allowed to exist, it begins to deteriorate.

pdwObjects

[out] Address of a variable that receives the maximum number of results objects allowed at one time in this grammar object. If more than maximum number of results objects exist, older results objects start to deteriorate.

As a results object ages, an engine discards certain amounts of its data to conserve memory (exactly which data is discarded depends on the engine). An application can prevent a results object from deteriorating by calling the ISRResMemory::LockSet member function to lock the object.

If an object deteriorates, the application's requests for data will fail. An application can identify what data is available by calling the ISRResMemory interface.


ISRGramCommon::DeteriorationSet

HRESULT DeteriorationSet(DWORD dwMemory, DWORD dwTime, DWORD dwObjects);

Sets the values used by the engine to determine how long to retain data stored in speech-recognition results objects.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwMemory

[in] Amount of memory to allow for storing speech-recognition results objects.

dwTime

[in] Number of seconds to allow a results object to exist.

dwObjects

[in] Maximum number of results objects to allow at one time.

To save memory, an engine allows data in results objects to deteriorate, unless a results object is locked by using the ISRResMemory::LockSet member function.

If an object deteriorates, the application's requests for data will fail. An application can identify what data is available by calling the ISRResMemory interface.


ISRGramCommon::TrainDlg

HRESULT TrainDlg(HWND hWndParent, PTSTR pszTitle);

Displays a dialog box so the speaker can train the grammar.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_INVALIDWINDOW
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

hWndParent

[in] Handle of the parent window for the modal dialog box. If this parameter is NULL, the engine returns NOERROR if it supports that dialog box, or SRERR_NOTSUPPORTED if it doesn't.

pszTitle

[in] Address of the string to use as the title of the dialog box. Can be NULL to direct the engine to provide a default title.

The dialog box is provided by a speech-recognition engine. The dialog box should give the speaker a choice of training only untrained entries and indicate the entries that are not yet trained. This allows the speaker to train the grammar for individual words when necessary. Training information is "attached" to both the speaker and the microphone.

If the engine does not supply this dialog box, the speaker cannot train the grammar; however, the application will still work.


ISRGramCommon::TrainPhrase

HRESULT TrainPhrase(DWORD dwExtent, PSDATA pData);

Queries the engine for a list of phrases the speaker should utter to improve recognition accuracy.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwExtent

[in] Flag that describes the extent of the training that the application needs to accomplish. This parameter can be one of these values:

Value | Meaning
SRTQEX_RECOMMENDED | The engine returns a list of recommended phrases that, when the engine is trained for them, improve recognition accuracy to an acceptable level.
SRTQEX_REQUIRED | The engine returns a list of only those phrases that are required for recognition to occur using the grammar. This list probably includes words for which there are no acoustic models or phonemic representations.

pData

[out] Address of an SDATA structure that receives a pointer to a list of SRPHRASE structures. The phrases are provided in order of importance, with the most important phrases first.

The application allocates the SDATA structure and passes its address to the TrainPhrase member function. TrainPhrase allocates memory (by using the OLE task allocator) for the returned data and sets the pData member of SDATA to point to the memory. If the allocation fails, pData is set to NULL and the dwSize member is set to zero. The application must free the memory pointed to by pData, as well as the SDATA structure itself.

The engine returns a buffer filled with a list of SRPHRASE structures. It is up to the application to prompt the speaker to speak the phrases, acquire recordings, and call the ISRResCorrection::Correction or ISRResCorrection::Validate member function for each phrase. A "phrase" can be a complete sentence, a partial sentence, or even an individual word.

The application must free the memory allocated by the function by using the CoTaskMemAlloc and CoTaskMemFree functions.

The engine returns SRERR_NOTSUPPORTED if it does not support TrainPhrase.


ISRGramCommon::TrainQuery

HRESULT TrainQuery(DWORD *pdwFlags);

Queries the engine for the kind of training needed for a particular grammar.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pdwFlags

[out] Address of a variable that receives a value indicating the kind of training to be done. This parameter can be one of these values:

Value | Meaning
SRGRAMQ_DIALOG | The application should call the ISRGramCommon::TrainDlg member function.
SRGRAMQ_GENERALTRAIN | General training should occur for the speaker because he or she hasn't trained before.
SRGRAMQ_PHRASE | Training should occur for certain phrases so that they can be recognized correctly. The application should call the ISRGramCommon::TrainPhrase member function.


ISRGramDictation Interface

The ISRGramDictation interface provides control of certain features that are specific to dictation grammars. Context-free grammars do not use this interface.

For more general control of context-free and dictation grammars, see the description of the ISRGramCommon interface.

The ISRGramDictation interface is supported by the grammar object. It has these member functions:

Context

Hint

Words


ISRGramDictation::Context

HRESULT Context(PTSTR pszPrior, PTSTR pszAfter);

Provides a speech-recognition engine with context — that is, the words spoken before and after the current utterance.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

pszPrior

[in] Address of a null-terminated string that contains the words spoken before the current utterance. For example, the words "This is" might precede the utterance "a test." The string can include punctuation. This parameter can be NULL.

pszAfter

[in] Address of a null-terminated string that contains the words spoken after the current utterance. For example, "of the emergency broadcasting system." The string can include punctuation. This parameter can be NULL.

For best performance, the application should pass only a sentence or two for each parameter.

An application calls Context whenever the cursor position changes in the document being dictated. Because the application may call Context several times in quick succession while the speaker scrolls through a document, the engine should not do a lot of processing during the call.

If an engine does not support contexts, it ignores this function.


ISRGramDictation::Hint

HRESULT Hint(PTSTR pszHint);

Gives the speech-recognition engine information about the subcontext in which a document is being used.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

pszHint

[in] Address of a null-terminated string that indicates the kind of text that is being dictated. This parameter can be one of these strings:

String | Description
"Address" | An address
"Date" | A date
"Text" | Free-form text


ISRGramDictation::Words

HRESULT Words(PTSTR pszWords);

Informs the speech-recognition engine that a particular set of words is likely to be spoken. The engine can ignore the set of words.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

pszWords

[in] Address of a list of words separated by spaces (even in Japenese and other languages that don't use space). The list is terminated by a null character.


ISRGramInsertionGUI Interface

The ISRGramInsertionGUI interface is used by an application to display a small status window next to the insertion point in a document or edit field. The status window gives the user information about the progress of recognition. The window is hidden until the application specifies otherwise. An engine may or may not support this interface.

The information provided in the status window is defined by the engine developer, but it may include the text of recognitions and hypotheses, a VU meter, indication of the beginning and end of an utterance, and whether the engine is processing. Some engines allow the user to drag the status window away from the insertion point and drop it at another location.

The ISRGramInsertionGUI interface is supported by the speech-recognition engine object. It has these member functions:

Hide

Move

Show


ISRGramInsertionGUI::Hide

HRESULT Hide{void);

Hides a visible status window.

· Returns NOERROR if successful, or the E_OUTOFMEMORY error value otherwise.


ISRGramInsertionGUI::Move

HRESULT Move(RECT rCursor);

Moves the status window away from the specified cursor position, usually just below or above it.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

rCursor

[in] Position or size of the cursor, in screen units.


ISRGramInsertionGUI::Show

HRESULT Show(HWND hWnd);

Shows the status window. When the grammar object is first created, the status window is hidden.

· Returns NOERROR if successful, or one of these error values:

E_OUTOFMEMORY
SRERR_INVALIDWINDOW

hWnd

[in] Handle of the application window for which to display the status window. When hWnd is the foreground window, the status window appears; when hWnd loses the focus, the status window disappears. An application can use this as a convenient alternative to calling the ISRGramInsertionGUI::Hide and ISRGramInsertionGUI::Show member functions when the focus changes. If hWnd is NULL, the status window is displayed continuously, regardless of which application window has the focus.


ISRGramNotifySink Interface

The ISRGramNotifySink interface is used by a speech-recognition engine to notify an application of recognition-related events. An application specifies the address of this interface when it creates the grammar object. A grammar may or may not call the PhraseHypothesis and Training member functions.

An engine processes one "phrase" at a time and follows every PhraseStart member function with a PhraseFinish member function before going on to process the next phrase. This means that the engine may fall behind in recognitions if several phrases or commands are spoken in quick succession. However, the application need not be concerned about this. If the engine is paused, it does not go on to the next phrase.

The ISRGramNotifySink interface must match the ISRCentral interface from which the GrammarLoad member function is being called — ISRGramNotifySinkA for ISRCentralA (ANSI) or ISRGramNotifySinkW for ISRCentralW (Unicode).

The ISRGramNotifySink interface is supported by an application. It has these member functions:

BookMark | PhraseStart
Paused | ReEvaluate
PhraseFinish | Training
PhraseHypothesis | UnArchive


ISRGramNotifySink::BookMark

HRESULT BookMark(DWORD dwBookMarkID);

Notifies the application that the engine has encountered a bookmark in the incoming audio stream.

· Return value is ignored.

dwBookMarkID

[in] Unique bookmark identifier.


ISRGramNotifySink::Paused

HRESULT Paused(void);

Reminds the application that the engine is paused.

· Return value is ignored.

If the grammar was activated with the fAutoPause parameter set to TRUE, it sends this notification after every call to the PhraseFinish member function. After receiving the Paused notification, the application must eventually call the ISRCentral::Resume member function or recognition ceases.


ISRGramNotifySink::PhraseFinish

HRESULT PhraseFinish(DWORD dwFlags, QWORD qTimeStampBegin,

QWORD qTimeStampEnd, PSRPHRASE pSRPhrase,
PIUNKNOWN pIUnknownResult);

Informs the application that the speaker has finished a phrase and the speech-recognition engine is certain about the words that were spoken.

· Return value is ignored.

dwFlags

[in] This parameter can be one or both of the following values:

Value | Meaning
ISRNOTEFIN_RECOGNIZED | The recognition was above the threshold set with the ISRAttributes::ThresholdSet member function. If this flag is not set, the engine did not have enough confidence in the recognition, and it should be declared as an "unknown utterance."
ISRNOTEFIN_THISGRAMMAR | This grammar had the highest score of all active grammars and claimed the recognition. If this flag is not set, another grammar had a higher score and claimed the recognition.

For an application to accept a recognition as valid, both ISRNOTEFIN_RECOGNIZED and ISRNOTEFIN_THISGRAMMAR should be set.

qTimeStampBegin

[in] Time, in bytes, when the phrase began. Most applications ignore this parameter.

qTimeStampEnd

[in] Time, in bytes, when the phrase ended. Most applications ignore this parameter.

pSRPhrase

[in] Address of an SRPHRASE structure that contains the engine's best recognition choice for the phrase. If dwFlags does not contain either ISRNOTEFIN_RECOGNIZED or ISRNOTEFIN_THISGRAMMAR, pSRPhrase may be NULL.

pIUnknownResult

[in] Address of an IUnknown interface of the speech-recognition results object created for this recognition. If this parameter is NULL, this engine has no object to return. The IUnknown interface and the object are released when the application returns from the PhraseFinish member function. Unless the application calls the IUnknown::QueryInterface member function on one of the object's interfaces, the results object is freed. This allows the application to ignore the pIUnknownResult parameter without causing results objects to accumulate and consume memory needlessly.

Depending on the grammar, a phrase can be the following:

· Context-free grammar. An ordered list of spoken words that traverse and complete the grammar tree from start to finish. A phrase should be sent even if it is grammatically incorrect.

· Limited-domain or dictation grammar. An ordered set of words. Typically, the PhraseFinish member function is sent once per utterance, but this is not guaranteed. In some cases, PhraseFinish may be sent several times per utterance; in other cases one PhraseFinish may be sent for several utterances.

The information provided in pSRPhrase should satisfy most applications. Those that need more information can use the speech-recognition results object, if the engine provides it. This object probably supports one or more of the speech-recognition results interfaces (ISRResAudio, ISRResBasic, and so on), but the application must query for each interface desired.

The application can keep a speech-recognition results object for later reference, although this object can consume large amounts of memory. However, from the engine's point of view, it doesn't really have to create the whole object until the QueryInterface member function is called.

If two or more grammars contain the same phrase, and this phrase is recognized, it is indeterminant which of the grammars will cause the engine to set the ISRNOTEFIN_THISGRAMMAR flag in the dwFlags parameter of the ISRGramNotifySink::PhraseFinish member function.


ISRGramNotifySink::PhraseHypothesis

HRESULT PhraseHypothesis(DWORD dwFlags, QWORD qTimeStampBegin,

QWORD qTimeStampEnd, PSRPHRASE pSRPhrase,
PIUNKNOWN pIUnknownResult);

Informs the application that the engine has a hypothesis about what was said over the past few seconds but will not commit to a recognition.

· Return value is ignored.

dwFlags

[in] This parameter can be one or both of these values:

Value | Meaning
ISRNOTEFIN_RECOGNIZED | The recognition was above the threshold set with the ISRAttributes::ThresholdSet member function. If this flag is not set, the engine did not have enough confidence in the recognition, and it should be declared as an "unknown utterance."
ISRNOTEFIN_THISGRAMMAR | This grammar had the highest confidence of all active grammars and claimed the recognition. If this flag is not set, another grammar had higher confidence and claimed the recognition.

For an application to accept a recognition as valid, both ISRNOTEFIN_RECOGNIZED and ISRNOTEFIN_THISGRAMMAR should be set.

qTimeStampBegin

[in] Time, in bytes, when the phrase began.

qTimeStampEnd

[in] Time, in bytes, when the phrase ended.

pSRPhrase

[in] Address of an SRPHRASE structure that contains the engine's best choice for the phrase.

pIUnknownResult

[in] Address of an IUnknown interface of the speech-recognition results object created for this recognition. If this parameter is NULL, this engine has no object to return. An engine is unlikely to provide a speech-recognition results object for a hypothesis.

A phrase is uniquely identified by the time stamp when the recognition began (qTimeStampBegin). If an engine calls PhraseHypothesis, it must eventually call the PhraseFinish member function with the same beginning time stamp.

The PhraseHypothesis member function is more useful for dictation than for command and control because preliminary results can be displayed to the speaker, providing an opportunity for the speaker to make corrections if needed.


ISRGramNotifySink::PhraseStart

HRESULT PhraseStart(QWORD qTimeStampBegin);

Notifies the application that the engine has started processing audio data for a recognition result.

· Return value is ignored.

qTimeStampBegin

[in] Time, in bytes, when the phrase began.

After sending PhraseStart, the engine may send one or more PhraseHypothesis notifications, but it must always conclude with a PhraseFinish.


ISRGramNotifySink::ReEvaluate

HRESULT ReEvaluate(PIUNKNOWN pIUnknown);

Notifies an application that the engine changed a speech-recognition results object as a consequence of the engine's background processing (rather than an explicit ReEvaluate call by the application).

· Return value is ignored.

pIUnknown

[in] Address of the IUnknown interface on the speech-recognition results object that was reevaluated.

The application can compare the IUnknown interface of the reevaluated results object against the list of existing results objects or use the information returned by the object to determine the text to which it applies. The application does not have to call the ISRResEval::ReEvaluate member function to obtain the new information.

The application does not have to (and should not) release pIUnknown because the ISRGramNotifySink::ReEvaluate member function does not call the AddRef member function. This allows the application to ignore ReEvaluate notifications.


ISRGramNotifySink::Training

HRESULT Interference(DWORD dwTrain);

Indicates to the application that the engine requires training for the current speaker, the current grammar, or the current microphone.

· Return value is ignored.

dwTrain

[in] Flag that indicates the recommended type of training. This parameter can be one of these values:

Value | Meaning
SRGNSTRAIN_MICROPHONE | Training for the current microphone.
SRGNSTRAIN_GRAMMAR | Training for the current grammar.
SRGNSTRAIN_GENERAL | General training.

The application must call the appropriate training dialog box for the engine.


ISRGramNotifySink::UnArchive

HRESULT UnArchive(PIUNKNOWN pIUnknownResult);

Notifies an application that a speech-recognition results object that was archived with a grammar has been loaded along with the grammar.

· Return value is ignored.

pIUnknownResult

[in] Address of the IUnknown interface on the speech-recognition results object that was loaded.

The application receives an UnArchive notification for each speech-recognition results object in existence when the grammar was archived. The unique identifier of the results object is maintained, so the application can tell which results object belongs to which text.

Once the application returns from the UnArchive member function, both the IUnknown interface and the speech-recognition results object are be released unless the application calls the IUnknown::QueryInterface member function on one of the object's interfaces. This allows an application to ignore the parameter without causing results objects to accumulate and consume memory needlessly. The results object is likely to support the speech-recognition results interfaces (ISRResAudio, ISRResBasic, and so on), but the application must query for each interface desired.

An application can keep a results object for later reference although the object can consume large amounts of memory. However, from the engine's point of view, it doesn't really need to re-create the entire results object until the QueryInterface member function is called.


ISRNotifySink Interface

The ISRNotifySink interface is used by an engine to return status information to the application. This interface is passed to the engine by the speech-recognition enumerator when it registers the engine. All functions are optional and may or may not be supported by an engine.

All notifications are sent as soon as possible, even though the engine has not begun processing the audio data. These notifications are generally for the application's user interface and exist to provide immediate feedback.

The ISRNotifySink interface has these member functions:

AttribChanged | UtteranceBegin
Interference | UtteranceEnd
Sound | VUMeter


ISRNotifySink::AttribChanged

HRESULT AttribChanged(DWORD dwAttribute);

Notifies an application that an engine attribute has changed. This notification is necessary because multiple processes or applications may be using the same speech-recognition engine through the speech-recognition sharing object, so another application may have changed the attribute.

· Return value is ignored.

dwAttribute

[in] Engine attribute that was changed. This parameter can be one of these values:

Value | Meaning
ISRNSAC_AUTOGAINENABLE | Automatic gain
ISRNSAC_ECHO | Echo cancellation status
ISRNSAC_ENERGYFLOOR | Energy noise floor
ISRNSAC_MICROPHONE | Current microphone
ISRNSAC_REALTIME | Percentage of processor time the engine expects to use during constant speech
ISRNSAC_SPEAKER | Name of the current speaker
ISRNSAC_THRESHOLD | Confidence threshold
ISRNSAC_TIMEOUT | Time-out duration for completing phrases


ISRNotifySink::Interference

HRESULT Interference(QWORD qTimeStampBegin, QWORD qTimeStampEnd,

DWORD dwType);

Notifies the application that the engine cannot recognize speech properly for a known reason.

· Return value is ignored.

qTimeStampBegin

[in] Time, in bytes, when the interference began.

qTimeStampEnd

[in] Time, in bytes, when the engine notified the application of the interference.

dwType

[in] Flag that indicates the type of interference. This parameter can be one of these values:

Value | Meaning
SRMSGINT_AUDIODATA_STARTED | The engine has resumed receiving audio data from the audio source.
SRMSGINT_AUDIODATA_STOPPED | The engine has stopped receiving audio data from the audio source.
SRMSGINT_IAUDIO_STARTED | The engine has started (opened) the audio input device
SRMSGINT_IAUDIO_STOPPED | The engine has stopped (closed) the audio input device.
SRMSGINT_NOISE | The background noise is too high.
SRMSGINT_NOSIGNAL | The engine cannot detect a signal. This may mean that the microphone has been turned off or unplugged.
SRMSGINT_TOOLOUD | The user is speaking too loudly and recognition results may be degraded.
SRMSGINT_TOOQUIET | The user is speaking too quietly and recognition results may be degraded.


ISRNotifySink::Sound

HRESULT Sound(QWORD qTimeStampBegin, QWORD qTimeStampEnd);

Notifies the application that the engine detected a sound (or a very garbled word).

· Return value is ignored.

qTimeStampBegin

[in] Time, in bytes, when the sound began.

qTimeStampEnd

[in] Time, in bytes, when the engine notified the application of the sound.

The engine determines how long to wait before sending this notification.


ISRNotifySink::UtteranceBegin

HRESULT UtteranceBegin(QWORD qTimeStampBegin);

Notifies the application that the engine has detected the beginning of an utterance or sound.

· Return value is ignored.

qTimeStampBegin

[in] Time, in bytes, when the sound began.

Detecting the beginning of an utterance doesn't necessarily have any bearing on the recognition of words or the completion of a phrase. Because a call to UtteranceBegin implicitly signals the end of the silence, an application can use this function to detect long periods of silence and react accordingly.


ISRNotifySink::UtteranceEnd

HRESULT UtteranceEnd(QWORD qTimeStampBegin, QWORD qTimeStampEnd);

Notifies the application that an utterance has ended.

· Return value is ignored.

qTimeStampBegin

[in] Time, in bytes, when the utterance began.

qTimeStampEnd

[in] Time, in bytes, when the utterance ended.

The engine sends this notification 0.25 to 0.5 seconds after speech stops. This doesn't necessarily have any bearing on the recognition of words or completion of a phrase. Because a call to UtteranceEnd implicitly signals the beginning of silence, an application can use this function to detect long periods of silence and react accordingly.


ISRNotifySink::VUMeter

HRESULT VUMeter(QWORD qTimeStampBegin, WORD wLevel);

Notifies the application of the current VU level, which indicates the loudness of the digital-audio stream.

· Return value is ignored.

qTimeStampBegin

[in] Time, in bytes, when the VU meter reading was taken.

wLevel

[in] Value from 0 to 0xFFFF that indicates a linear level. 0xFFFF is the loudest volume that the VU meter can register.

The VU level returned by this function is only a rough estimate used to display a visual indicator on the screen. The engine sends this notification about eight times per second.


ISRResAudio Interface

The ISRResAudio interface allows an application to acquire the digital-audio data for an utterance and either save it to a waveform-audio (.wav) file or play it back to the speaker. This interface may or may not be supported by an engine.

The ISRResAudio interface is supported by the speech-recognition results object. It has a single member function: GetWAV.


ISRResAudio::GetWAV

HRESULT GetWAV(PSDATA pWav);

Retrieves an audio recording of an utterance in waveform-audio (.wav) file format.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA

pWav

[out] Address of an SDATA structure that receives audio data for the recognition. The dwSize member receives the amount of memory and file size needed to store the audio data. The pData member receives the address of a memory object that contains the file format.

The application allocates the SDATA structure and passes its address to the GetWAV member function. GetWAV allocates memory (using the OLE task allocator) for the returned data and sets the pData member of SDATA to point to the memory. If the allocation fails, the pData member is set to NULL and the dwSize member is set to zero. The application must free the memory pointed to by pData, as well as the SDATA structure itself.

If the engine has maintained an audio recording of the utterance, it packages the audio into waveform-audio file format. The application can then save the audio to a file or use the SndPlaySound function to play it back. If no digital-audio information is stored, this returns an error.

The application must free the memory allocated by the member function, by using the CoTaskMemAlloc and CoTaskMemFree functions.

For more information about the WAV file format, see the Microsoft Win32 Software Development Kit (SDK).


ISRResBasic Interface

The ISRResBasic interface provides an application with general information about a speech-recognition results object.

This interface is supported by the speech-recognition results object. It has these member functions:

FlagsGet | PhraseGet
Identify | TimeGet


ISRResBasic::FlagsGet

HRESULT FlagsGet(DWORD dwRank, DWORD *pdwFlags);

Retrieves the recognition flags for the specified phrase.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_VALUEOUTOFRANGE

dwRank

[in] Value that specifies the alternative phrase to get from the alternatives provided by the engine, starting with dwRank = 0 for the strongest alternative. If dwRank is greater than the number of alternatives, FlagsGet returns an error.

pdwFlags

[out] Address of a variable that receives one or more of these values:

Value | Meaning
ISRNOTEFIN_RECOGNIZED | The recognition was above the threshold set with the ISRAttributes::ThresholdSet member function. If this flag is not set, the engine did not have enough confidence in the recognition, and it should be declared as an "unknown utterance."
ISRNOTEFIN_THISGRAMMAR | This grammar had the highest score of all active grammars and claimed the recognition. If this flag is not set, another grammar had a higher score and claimed the recognition.
ISRNOTEFIN_FROMTHISGRAMMAR | The phrase was generated from this grammar (as opposed to another grammar). (New for 3.0)

For an application to accept a recognition as valid, both ISRNOTEFIN_RECOGNIZED and ISRNOTEFIN_THISGRAMMAR should be set.


ISRResBasic::Identify

HRESULT Identify(GUID *pgIdentity);

Retrieves a unique identifier for the results object so that an application can identify the results object at a later point.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pgIdentity

[out] Address of a variable that receives the GUID of the results object, which uniquely identifies that object.

Each engine provides its own mechanism for generating a results GUID, probably incorporating the timestamps returned by the ISRResBasic::TimeGet member function (in the qwTimeStampBegin and qwTimeStampEnd parameters) and some other differentiator, such as the time as specified in a Win32 FILETIME structure when the results object was generated.


ISRResBasic::PhraseGet

HRESULT PhraseGet(DWORD dwRank, PSRPHRASE pSRPhrase,

DWORD dwPhraseSize, DWORD *pdwPhraseNeeded);

Retrieves the specified phrase from the list of most likely phrases provided by the speech-recognition engine for a recognition. The phrases are specific to the grammar and do not show alternatives for other grammars.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_VALUEOUTOFRANGE

dwRank

[in] Value that specifies the alternative phrase to get from the alternatives provided by the engine, starting with dwRank = 0 for the strongest alternative. If dwRank is greater than the number of alternatives, PhraseGet returns an error.

pSRPhrase

[out] Address of an SRPHRASE structure to receive the phrase.

dwPhraseSize

[in] Number of bytes available in the structure specified by pSRPhrase. If the number is too small, PhraseGet returns SRERR_NOTENOUGHDATA and fills in pdwPhraseNeeded with the number of bytes needed.

pdwPhraseNeeded

[out] Address of a variable that receives the number of bytes needed for the phrase data.


ISRResBasic::TimeGet

HRESULT TimeGet(PQWORD pqTimeStampBegin, PQWORD pqTimeStampEnd);

Retrieves the timestamps that mark the beginning and ending of the phrase.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA

pqTimeStampBegin

[out] Address of a QWORD that receives the time, in bytes, when the phrase began. This parameter can be NULL if the application doesn't need this value.

pqTimeStampEnd

[out] Address of a QWORD that receives the time, in bytes, when the phrase ended. This parameter can be NULL if the application doesn't need this value.


ISRResCorrection Interface

The ISRResCorrection interface allows an application to inform the speech-recognition engine what the user actually said to cause a recognition result. This interface may or may not be supported by an engine.

This interface is supported by the speech-recognition object. It has these member functions:

Correction

Validate


ISRResCorrection::Correction

HRESULT Correction(PSRPHRASE pSRPhrase, WORD wConfidence);

Indicates to a speech-recognition engine what the speaker actually said to cause a particular recognition result.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

pSRPhrase

[in] Address of an SRPHRASE structure that contains the phrase that the speaker actually said.

wConfidence

[in] Application's level of confidence that it knows what the speaker actually said. This parameter can be one of these values:

Value | Meaning
SRCORCONFIDENCE_SOME | The application is somewhat confident because the speaker somehow corrected the application without explicitly telling it what the correct result was.
SRCORCONFIDENCE_VERY | The application is very confident because the speaker explicitly informed it of what he or she said.


ISRResCorrection::Validate

HRESULT Validate(WORD wConfidence);

Notifies a speech-recognition engine that the utterance was correctly recognized.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

wConfidence

[in] Application's level of confidence that it knows what the speaker actually said. This parameter can be one of these values.

Value | Meaning
SRCORCONFIDENCE_SOME | The application is somewhat confident because the speaker has not indicated that the utterance was recognized incorrectly.
SRCORCONFIDENCE_VERY | The application is very confident because the speaker explicitly verified that the utterance was recognized correctly.


ISRResEval Interface

The ISRResEval interface allows an application to request reevaluation of a recognition result, if the engine is capable of maintaining the necessary information. This interface may or may not be supported by an engine.

The ISRResEval interface is supported by the speech-recognition object. It has this member function: ReEvaluate.


ISRResEval::ReEvaluate

HRESULT ReEvaluate(BOOL *pfChanged);

Causes a speech-recognition engine to reevaluate the recognition result represented by this object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

pfChanged

[out] Address of a variable that receives TRUE if reevaluating the recognition changed the result. An engine may not be able to reevaluate a result because it doesn't have enough information.

Reevaluating a recognition allows the engine to take advantage of new knowledge that it has recently acquired.

Typically, ReEvaluate is used for dictation in which the application has the speaker do a speaking pass followed by a correction pass. As the speaker corrects the document, the application may call the ISRResCorrection::Correction and ISRResCorrection::Validate member functions, which ultimately provide the engine with the following information:

· New proper names or words are identified, and their text is mapped to phonetic representations. Subsequent occurrences of these phonemes should be correctly recognized.

· Misrecognitions are corrected. This effect sometimes ripples through a document because a misrecognition can produce non sequiturs that confuse the engine's grammar rules, and corrections can affect subsequent recognitions.

· The engine has managed to adapt to the user's speech and grammar patterns, which corrects some misrecognitions.

· By the end of a document the engine may be able to better identify the context of the dictation and the vocabulary used within the document.

An advanced engine can take advantage of all of these clues.


ISRResGraph Interface

The ISRResGraph interface allows an application to examine the recognition/alternative graph generated by the engine. An application that needs detailed results uses this interface. This interface may or may not be supported by an engine. The GetPhonemeNode member function is optional.

The interface assumes that the engine has created a word graph while recognizing an utterance. The graph has both a beginning and alternate recognition paths that were explored by the engine in the process of recognizing the utterance. The interface also supports the same kind of graph for phonemes. Visually, such a graph might look like this:

The ISRResGraph interface is supported by the speech-recognition results object. It has these member functions:

BestPathPhoneme | GetWordNode
BestPathWord | PathScorePhoneme
GetPhonemeNode | PathScoreWord


ISRResGraph::BestPathPhoneme

HRESULT BestPathPhoneme(DWORD dwRank, DWORD *padwPath, DWORD dwPathSize,

DWORD *pdwPathNeeded);

Retrieves the specified phoneme path from the list of most likely phoneme paths provided by the speech-recognition engine for a recognition.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwRank

[in] Specifies the alternative path to get from the alternatives provided by the engine, starting with dwRank==0 for the strongest alternative. If dwRank is greater than the number of alternatives, BestPathPhoneme returns an error.

padwPath

[out] Address of an array that receives the node numbers that constitute the path.

dwPathSize

[in] Number of bytes available in padwPath. If the number is too small, BestPathPhoneme returns an error and fills in pdwPathNeeded with the number of bytes needed.

pdwPathNeeded

[out] Number of bytes needed to store the path data.

You can calculate the number of node numbers returned in padwPath by dividing the value of pdwPathNeeded by the size of a double word (sizeof(DWORD)).


ISRResGraph::BestPathWord

HRESULT BestPathWord(DWORD dwRank, DWORD *padwPath,

DWORD dwPathSize, DWORD *pdwPathNeeded);

Retrieves the specified word path from the list of most likely phoneme paths provided by the speech-recognition engine for a recognition.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwRank

[in] Value that specifies the alternative path to get from the alternatives provided by the engine, starting with dwRank = 0 for the strongest alternative. If dwRank is greater than the number of alternatives, BestPathWord returns an error.

padwPath

[out] Address of an array that receives the node numbers that constitute the path.

dwPathSize

[in] Number of bytes available in padwPath. If the number is too small, BestPathWord returns an error and fills in pdwPathNeeded with the number of bytes needed.

pdwPathNeeded

[out] Number of bytes needed to store the path data.

You can calculate the number of node numbers returned in padwPath by dividing the value of pdwPathNeeded by the size of a double word (sizeof(DWORD)).


ISRResGraph::GetPhonemeNode

HRESULT GetPhonemeNode(DWORD dwPhonemeNode, PSRRESPHONEMENODE pNode,

PWCHAR pcIPA, PTCHAR pcEngine);

Retrieves information about the phoneme for a given node in a recognition results graph.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwPhonemeNode

[in] Number of the desired node. The beginning node in a graph is always 1. An application can obtain the number of a particular node by checking the SRRESPHONEMENODE or SRRESWORDNODE structures of one node for next, previous, and alternative nodes.

pNode

[out] Address of an SRRESPHONEMENODE structure that receives information about the node.

pcIPA

[out] Address of a wide character that receives the International Phonetic Alphabet (IPA) phoneme for the node. This parameter can be NULL if an IPA phoneme must be represented by two or more engine phonemes or if the engine does not support the IPA.

pcEngine

[out] Address of the character that receives the engine-specific phoneme for the node. This parameter can be NULL if an engine phoneme must be represented by two or more IPA phonemes.


ISRResGraph::GetWordNode

HRESULT GetWordNode(DWORD dwWordNode, PSRRESWORDNODE pNode,

PSRWORD pSRWord, DWORD dwMemSize, DWORD *pdwMemNeeded);

Retrieves information about the word for a given node in a recognition results graph.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwWordNode

[in] Number of the desired node. The beginning node in a graph is always 1. An application can obtain the number of a particular node by checking the SRRESPHONEMENODE or SRRESWORDNODE structures of one node for next, previous, and alternative nodes.

pNode

[out] Address of an SRRESWORDNODE structure that receives information about the node.

pSRWord

[out] Address of an SRWORD structure that receives the word (if the allocated memory is large enough). If this parameter is NULL, no information is copied.

dwMemSize

[in] Number of bytes available in the structure specified by pSRWord. If the number is too small, the member function returns an error and fills pdwMemNeeded with the number of bytes needed to store the word.

pdwMemNeeded

[out] Number of bytes needed to store the entire word structure.


ISRResGraph::PathScorePhoneme

HRESULT PathScorePhoneme(DWORD *paNodes, DWORD dwNumNodes,

LONG *plScore);

Determines the score for a particular phoneme path.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

paNodes

[in] Address of an ordered array of nodes that defines the path. The path must start at node 1 or one of its alternate nodes, end at a terminal node, and be contiguous. If these conditions are not met, the engine cannot calculate the score and it returns an error.

dwNumNodes

[in] Number of nodes in the path and, therefore, also the number of elements in the array.

plScore

[out] Address of a variable that receives the score for the path. The score is normalized, so the engine considers any score less than 0 to be unrecognizable (that is, the utterance is not in the grammar). Phoneme score values from the same speech-recognition results object can be compared against each another.

An application can use the score to determine whether the engine determined the recognition was valid or whether one path is better than another.


ISRResGraph::PathScoreWord

HRESULT PathScoreWord(DWORD *paNodes, DWORD dwNumNodes, LONG *plScore);

Determines the score for a particular word path.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

paNodes

[in] Address of an ordered array of nodes that defines the path. The path must start at node 1 or one of its alternate nodes, end at a terminal node, and be contiguous. If these conditions are not met, the engine cannot calculate the score, and it returns an error.

dwNumNodes

[in] Number of nodes in the path and, therefore, also the number of elements in the array.

plScore

[out] Address of a variable that receives the score for the path. The score is normalized, so the engine considers any score less than 0 to be a misrecognition. Word score values from the same speech-recognition results object can be compared against each another.

An application can use the score to determine whether the engine determined the recognition was valid or whether one path is better than another.


ISRResMemory Interface

The ISRResMemory interface allows an application to determine, and possibly reduce, the amount of memory used by a speech-recognition results object. Support this interface only if applications are likely to keep results objects around. This interface may or may not be supported by an engine.

The ISRResMemory interface is supported by the speech-recognition object. It has these member functions:

Free | LockGet
Get | LockSet


ISRResMemory::Free

HRESULT Free(DWORD dwKind);

Instructs a speech-recognition results object to discard the specified information so that it uses less memory.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

dwKind

[in] Kind of information to free. This parameter can be a combination of these values:

Value | Meaning
SRRESMEMKIND_AUDIO | A digital-audio recording used by the ISRResAudio interface to get an utterance in waveform-audio file format.
SRRESMEMKIND_CORRECTION | Information used by the ISRResCorrection interface to correct or validate recognitions.
SRRESMEMKIND_EVAL | Information used by the ISRResEval interface to reevaluate a recognition.
SRRESMEMKIND_PHONEMEGRAPH | Phoneme graph information used by the ISRResGraph interface.
SRRESMEMKIND_WORDGRAPH | Word graph information used by the ISRResGraph interface.

Freeing a particular kind of information prevents certain functions from subsequently using that information.


ISRResMemory::Get

HRESULT Get(DWORD *pdwKind, DWORD *pdwMemory);

Retrieves the kind of information stored in a speech-recognition results object and the amount of memory used by that object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pdwKind

[out] Address of a variable that receives an array of flags that describe the kind of information stored. This parameter can be a combination of these values:

Value | Meaning
SRRESMEMKIND_AUDIO | A digital-audio recording used by the ISRResAudio interface to get an utterance in waveform-audio file format.
SRRESMEMKIND_CORRECTION | Information used by the ISRResCorrection interface to correct or validate recognitions.
SRRESMEMKIND_EVAL | Information used by the ISRResEval interface to reevaluate a recognition.
SRRESMEMKIND_PHONEMEGRAPH | Phoneme graph information used by the ISRResGraph interface.
SRRESMEMKIND_WORDGRAPH | Word graph information used by the ISRResGraph interface.

pdwMemory

[out] Address of a variable that receives an estimate of the number of bytes used by the entire object.


ISRResMemory::LockGet

HRESULT LockGet(BOOL *pfLock);

Retrieves the lock status for a speech-recognition results object. A results object that is locked does not deteriorate and lose information.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

pfLock

[out] TRUE if the speech-recognition results object is locked and its data does not deteriorate, or FALSE if the object is allowed to deteriorate. The default is FALSE.

Because results objects occupy so much memory, an engine may discard some of a results object's data over time so that memory isn't consumed by results objects. An application can prevent deterioration of a results object by calling the LockSet member function.

The effects of the deterioration are engine-dependent, but they probably involve a loss of data, starting with the largest data first. This means that digital-audio data is lost first, followed by the information necessary for training. If the information has deteriorated, calling any function that uses the missing information returns an error.


ISRResMemory::LockSet

HRESULT LockSet(BOOL fLock);

Sets the lock status for a speech-recognition results object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

fLock

[in] TRUE if the results object should be locked and if its data should be prevented from deteriorating, or FALSE if the results object should be allowed to deteriorate.

An application can preserve a results object indefinitely by calling LockSet with fLock set to TRUE.


ISRResMerge Interface

The ISRResMerge interface allows an application to merge data from two speech-recognition results objects so that the training or other information from those objects can be retrieved as a unit. This interface may or may not be supported by an engine.

The ISRResMerge interface is supported by the speech-recognition results object. It has these member functions:

Merge

Split


ISRResMerge::Merge

HRESULT Merge(LPUNKNOWN pIUnkAdjacent, PIUNKNOWN ppIUnkMerge);

Merges another speech-recognition results object with the current object so their data can be manipulated together.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

pIUnkAdjacent

[in] Address of the IUnknown interface of the results object to merge with the current one.

ppIUnkMerge

[out] Address of a variable that receives a pointer to the IUnknown interface for the new results object created by the merge. The original results objects are not affected.

The two results objects must be temporally adjacent, or Merge returns an error.

Merging results objects is especially useful for obtaining more detailed information in a results graph.


ISRResMerge::Split

HRESULT Split(QWORD qwSplitTime, PIUNKNOWN ppIUnkLeft,

PIUNKNOWN ppIUnkRight);

Splits a speech-recognition results object into two parts, keeping the original object intact. An application can use this function to examine smaller parts of a recognition result and to perform training or correction on the parts.

If splitting the results object at qwSplitTime would result in only one object, then the function returns SRERR_VALUEOUTOFRANGE.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

qwSplitTime

[in] Time stamp at which the result should be split. Because in many cases a word cannot be split in the middle, the engine maps this to the closest time that the result can be split.

ppIUnkLeft

[out] Address of a variable that receives a pointer to the IUnknown interface for the new results object created by the split. This is the portion of the original results object that occurred before the time specified in qwSplitTime. If no portion of the results object occurred before this time, ppIUnkLeft receives NULL. The original results object is not affected.

ppIUnkRight

[out] Address of a variable that receives a pointer to the IUnknown interface for the new results object created by the split. This is the portion of the original results object that occurred after the time specified in qwSplitTime. If no portion of the results object occurred after this time, ppIUnkRight receives NULL. The original results object is not affected.


ISRResModifyGUI Interface

The ISRResModifyGUI interface is used by an application to display a small status window next to selected text for specific recognition results. If the text selection includes portions of more than one results object, the application should either change the selection to cover only the called results object or generate a new results object from the selections. Initially, the window is hidden until the application calls the Show member function.

The information provided in the status window is defined by the engine developer. Typically, the window displays alternatives for a recognition result and allows the user to select one, although the engine may provide a more powerful user interface. This interface may or may not be supported by a results object.

The ISRResModifyGUI interface is supported by the speech-recognition results object. It has these member functions:

Hide

Move

Show


ISRResModifyGUI::Hide

HRESULT Hide(void);

Hides a visible status window.

· Returns NOERROR if successful, or the E_OUTOFMEMORY error value otherwise.


ISRResModifyGUI::Move

HRESULT Move(RECT * rSelection);

Moves the status window away from the specified rectangle, usually just below or above it.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

rSelection

[out] Address of a RECT structure that contains the position or size of the selection, in screen units.


ISRResModifyGUI::Show

HRESULT Show(HWND hWnd);

Shows the status window. When the results object is first created, the status window is hidden.

· Returns NOERROR if successful, or one of these error values:

E_OUTOFMEMORY
SRERR_INVALIDWINDOW

hWnd

[in] Handle of the application window for which to display the status window. When hWnd is the foreground window, the status window appears; when hWnd loses the focus, the status window disappears. An application can use this as a convenient alternative to calling the ISRResModifyGUI::Hide and ISRResModifyGUI::Show member functions when the focus changes. If hWnd is NULL, the status window is displayed continuously, regardless of which application window has the focus.


ISRResScore Interface

(New for 3.0)

The ISRResScore interface provides an application with scores for alternative recognitions and their individual words.

This interface is supported by the speech-recognition results object. It has these member functions:

GetPhraseScore

GetWordScores


ISRResScore::GetPhraseScore

HRESULT GetPhraseScore(DWORD dwRank, long *plScore);

Returns the score of the recognized phrase or alternative phrases. An application can get the text string for the alternatives by using ISRResBasic.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

dwRank

[in] Rank of the score. 0 is the final recognition result, 1 is the second best, and so on. If this exceeds the total number of alternatives in the results object then an error is returned.

plScore

[out] Filled in with a score from -100 to 100. Because of the inaccuracy of speech technology exact meanings for the scores are impossible. -100 is definitely a rejection. 0 is on the border between recognized and unrecognized. 100 is a very positive and strong recognition.


ISRResScore::GetWordScores

HRESULT GetWordScores(DWORD dwRank, long *palScore,

DWORD dwWordScoreSize, DWORD *pdwWordScoreRequired);

Returns the score of the recognized phrase or alternative phrases. An application can get the text string for the alternatives by using ISRResBasic.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTSUPPORTED

dwRank

[in] Rank of the score. 0 is the final recognition result, 1 is the second best, and so on. If this exceeds the total number of alternatives in the results object then an error is returned.

palScore

[out] This points to an array of longs. The array is filled in with the scores for individual words in the recognition results, starting at the beginning. Each score is filled in with a score from -100 to 100. Because of the inaccuracy of speech technology exact meanings for the scores are impossible. -100 is definitely a rejection. 0 is on the border between recognized and unrecognized. 100 is a very positive and strong recognition.

dwWordScoreSize

[in] Number of bytes in the memory pointed to by palScore. If the buffer is not large enough then GetWordScores returns S_FALSE and returns the required amount of space in the location pointed to by pdwWordScoreRequired.

pdwWordScoreRequired

[out] Filled in either with the number of bytes copied to palScore, or with the number of bytes required to copy the data if S_FALSE is returned.


ISRResSpeaker Interface

The ISRResSpeaker interface allows an application to identify the speaker for a recognition. An engine supports this interface for speech-recognition results objects if the engine can identify speakers. This interface may or may not be supported by an engine.

The ISRResSpeaker interface is supported by the speech-recognition results object. It has these member functions:

Correction | IdentifyForFree
Identify | Validate


ISRResSpeaker::Correction

HRESULT Correction(PTCHAR pszName, WORD wConfidence);

Indicates to a speech-recognition engine which speaker actually spoke to cause this recognition result.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

pszName

[in] Address of a string that contains the name of the person who actually spoke (rather than the name reported by the engine).

wConfidence

[in] Application's level of confidence that it knows the name of the speaker. This parameter can be one of these values:

Value | Meaning
SRCORCONFIDENCE_SOME | The application is somewhat confident because the user somehow corrected the computer without explicitly telling the computer who was speaking.
SRCORCONFIDENCE_VERY | The application is very confident because the user explicitly informed it who was speaking.


ISRResSpeaker::Identify

HRESULT Identify(DWORD dwRank, TCHAR *pszName, DWORD dwNameSize,

DWORD *pdwNeeded, LONG *plScore);

Identifies the speaker for a recognized utterance.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED
SRERR_VALUEOUTOFRANGE

dwRank

[in] Rank of the speaker. An engine ranks the candidates with integers, starting with 0 for the most likely speaker. If dwRank is greater than the number of speakers ranked by the engine, the function returns an error.

pszName

[out] Address of a buffer that receives the name of the person identified by dwRank.

dwNameSize

[in] Number of bytes allocated in pszName. If the number is too small, the function copies only part of the name string and fills pdwNeeded with the buffer size needed for the whole string.

pdwNeeded

[out] Address of a location in memory that receives the number of bytes needed to copy the entire name string into pszName.

plScore

[out] Address of a variable that receives the engine's level of confidence that the speaker in pszName actually spoke the utterance. The score is normalized, so that values less than 0 are not considered accurate enough for the engine to commit that the speaker in pszName is speaking.

An application can call Identify to learn the identity of the speaker, querying for the most to least likely choice and learning the confidence of the engine in that choice. The engine gets information about speakers through training and correction done while the ISRAttributes::SpeakerSet member function has set the specific speaker's name string.

Calling Identify can take additional processor time beyond that needed for speech recognition.


ISRResSpeaker::IdentifyForFree

HRESULT IdentifyForFree(BOOL *pfFree);

Queries the speech-recognition results object to determine whether identifying the speaker requires processing.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA
SRERR_NOTSUPPORTED

pfFree

[out] TRUE if the identification information does not require additional processing, or FALSE if it does. The processing won't be done until other member functions that require the information are called.

Some engines may take time (more than a few milliseconds) to identify a speaker. An application can query the object to see whether identifying the speaker takes less than a few milliseconds or extra calculations need to be done.


ISRResSpeaker::Validate

HRESULT Validate(WORDwConfidence);

Notifies a speech-recognition engine that the speaker was correctly recognized.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOTENOUGHDATA

wConfidence

[in] Application's level of confidence that it determined the name of the person who actually spoke. This parameter can be one of these values:

Value | Meaning
SRCORCONFIDENCE_SOME | The application is somewhat confident because the user has not indicated that the speaker was recognized incorrectly.
SRCORCONFIDENCE_VERY | The application is very confident because the user explicitly verified that the speaker was recognized correctly.


ISRSpeaker Interface

The ISRSpeaker interface allows the transfer of speaker information, or the speaker profile, from one system to another, so that a speaker doesn't have to perform training again when he or she switches to another system. For command and control applications, the effort of transferring a speaker profile may outweigh the benefits; however, it is extremely useful for dictation systems (which often require a lot of training) and speech servers (which need to share training). An engine may or may not support this interface.

The speaker profile should include all of the information that the engine has about the speaker. For example, a speaker profile can include:

· A data header to make sure that the application isn't passing information to the wrong engine

· The languages for which training has been done

· Known patterns of speech and the language model

· How specific words are pronounced

· Phonetic training

· Speaker identity

· Speaker preferences

· The number of hours of training done to create the profile (useful for merging speakers)

The ISRSpeaker interface is supported by the speech-recognition engine object. It has these member functions:

Delete | Read
Enum | Revert
Merge | Select
New | Write
Query |


ISRSpeaker::Delete

HRESULT Delete(TCHAR *pszSpeakerName);

Deletes all information that a speech-recognition engine has about a speaker.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSpeakerName

[in] Address of a string that contains the name of the speaker.

If the speaker is currently being used, the Delete member function returns an error.


ISRSpeaker::Enum

HRESULT Enum(PTSTR *ppszBuffer, DWORD *pdwBufSize);

Enumerates the list of speakers known to the speech-recognition engine.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

ppszBuffer

[out] Address of a variable that receives a pointer to a list of speaker names. Each speaker name is a null-terminated string. The list is also terminated with a NULL.

pdwBufSize

[out] Address of a variable that receives the number of bytes of data copied into pszBuffer.

The application must free the memory allocated by the member function, using the CoTaskMemAlloc and CoTaskMemFree functions.


ISRSpeaker::Merge

HRESULT Merge(TCHAR *pszSpeakerName, PVOID pSpeakerData,

DWORD dwSpeakerData);

Merges a speaker profile from another system into the speaker profile on the current system.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSpeakerName

[in] Address of a string that contains the name of the speaker.

pSpeakerData

[in] Address of the speaker profile data to merge with the profile of the speaker on the current system.

dwSpeakerData

[in] Size, in bytes, of the data pointed to by pSpeakerData.


ISRSpeaker::New

HRESULT New(TCHAR *pszSpeakerName);

Creates a new speaker, using the default settings provided by the engine for training.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSpeakerName

[in] Address of a string that contains the name of the speaker.

If the speaker already exists, the New function returns an error.


ISRSpeaker::Query

HRESULT Query(TCHAR *pszSpeakerName, DWORD dwSize, DWORD *pdwNeeded);

Retrieves the name of the currently selected speaker.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_NOUSERSELECTED

pszSpeakerName

[out] Address of a buffer that receives the name of the current speaker.

dwSize

[in] Number of bytes available in pszSpeakerName. If the number is too small, the Query member function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the speaker name.


ISRSpeaker::Read

HRESULT Read(TCHAR *pszSpeakerName, PVOID *ppBuffer, DWORD *pdwBufSize);

Retrieves speaker profile information for the specified speaker.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSpeakerName

[in] Address of a string that contains the name of the speaker.

ppBuffer

[out] Address of a variable that receives a pointer to the memory allocated for the speaker profile information.

pdwBufSize

[out] Address of a variable that receives the number of bytes copied into ppBuffer.

The application must free the memory allocated by the function, using the CoTaskMemAlloc and CoTaskMemFree functions.


ISRSpeaker::Revert

HRESULT Revert(TCHAR *pszSpeakerName);

Restores training to its original state at the beginning of the session, which is the last time the speaker information was loaded (either by calling the ISRAttributes::SpeakerSet or ISRSpeaker::Select member function).

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSpeakerName

[in] Address of a string that contains the name of the speaker.

A session begins when a speaker is selected with a call to the Select member function (unless that speaker was already selected.) An application can use Revert to cancel undesirable changes, such as those caused by extraneous noise or unrelated conversations during a training session.


ISRSpeaker::Select

HRESULT Select(TCHAR *pszSpeakerName, BOOL fLock);

Selects the specified speaker to be the current speaker on the system.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszSpeakerName

[in] Address of a string that contains the name of the speaker. If pszSpeakerName is NULL, the speaker is set to "Unknown".

fLock

[in] TRUE if the speaker name is to be locked and can be changed only by calls from the application, or FALSE if the engine can regard pszSpeakerName as a recommendation and change the current speaker if it believes that another person is speaking.

If the speaker doesn't exist, the engine creates a new speaker. The ISRSpeaker::Select member function is identical to the ISRAttributes::SpeakerSet member function except that Select also allows the application to lock the speaker name.


ISRSpeaker::Write

HRESULT Write(TCHAR *pszSpeakerName, PVOID pSpeakerData,

DWORD dwSpeakerData);

Writes speaker profile information from another system to the current system.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
SRERR_SPEAKEREXISTS

pszSpeakerName

[in] Address of a string that contains the name of the speaker.

pSpeakerData

[in] Address of the speaker profile data to write.

dwSpeakerData

[in] Size, in bytes, of the data pointed to by pSpeakerData.

If the specified speaker already exists on the current system, the function returns SRERR_SPEAKEREXISTS. If this happens, the application can either call the ISRSpeaker::Merge member function to combine the speaker profiles or delete the current profile and call Write again.


Structures

Certain speech-recognition functions use structures at run time to store and retrieve information about a particular grammar or speech-recognition mode. This section describes these structures.


SRMODEINFO

typedef struct { // srmi

GUID gEngineID;

TCHAR szMfgName[SRMI_NAMELEN];

TCHAR szProductName[SRMI_NAMELEN];

GUID gModeID;

TCHAR szModeName[SRMI_NAMELEN];

LANGUAGE language;

DWORD dwSequencing;

DWORD dwMaxWordsVocab;

DWORD dwMaxWordsState;

DWORD dwGrammars;

DWORD dwFeatures;

DWORD dwInterfaces;

DWORD dwEngineFeatures;

} SRMODEINFO, *PSRMODEINFO;

Provides details about a speech-recognition mode supported by a particular engine.

gEngineID

GUID that identifies the engine. It is possible for more than one mode to have the same engine identifier. In this case, the engine is capable of recognizing the different languages or voices of the modes.

szMfgName

Name of the manufacturer — for example, "Acme Corporation."

szProductName

Name of the product — for example, "Acme's Ultra-Listener."

gModeID

GUID that identifies the mode. These numbers are guaranteed to be unique, so it is not necessary to monitor and record them. In addition, because the GUID is not stored or used directly by OLE, it is not necessary to load it into the registry.

szModeName

Name of the recognition mode — for example, "English Ultra-Listener."

language

LANGUAGE structure that contains the name of the language and dialect.

dwSequencing

Recognition scheme. This member can be one of these values:

Value | Meaning
SRSEQUENCE_CONTINUOUS | Performs continuous recognition. Given a continuous utterance with no pauses between words, the engine can recognize the words that were spoken.
SRSEQUENCE_DISCRETE | Recognizes discrete utterances. Every word must be isolated by a pause before and after the word for the engine to recognize it.
SRSEQUENCE_WORDSPOT | Performs word spotting. A series of words may be spoken in a continuous utterance, but the engine recognizes only selected words or phrases.
SRSEQUENCE_CONTCFGDISCDICT | Performs continuous recognition for context-free grammars. It also supports discrete dictation. (New for 3.0)

dwMaxWordsVocab

Maximum number of words allowed in an active vocabulary at any time.

dwMaxWordsState

Maximum number of words in any state of a grammar.

dwGrammars

Types of grammar supported by the engine. This member can be a combination of these values:

Value | Meaning
SRGRAM_CFG | Context-free grammar
SRGRAM_DICTATION | Dictation grammar
SRGRAM_LIMITEDDOMAIN | Limited-domain grammar

dwFeatures

Features that are available. This member can be a combination of these values:

Value | Meaning
SRFEATURE_ANYWORD | The engine can attempt to recognize any word.
SRFEATURE_GRAMLINK | The engine supports automatic links between grammars.
SRFEATURE_GRAMLIST | The grammars support recognition lists.
SRFEATURE_GRAMRECURSIVE | The engine accepts recursive, context-free grammars; otherwise, it cannot have recursive grammars.
SRFEATURE_INDEPMICROPHONE | The engine is microphone-independent (indifferent to background noise and microphone) and does not require retraining if the microphone changes.
SRFEATURE_INDEPSPEAKER | The engine is speaker-independent and works well without training.
SRFEATURE_IPAUNICODE | The engine supports the IPA in Unicode.
SRFEATURE_MULTILINGUAL | The engine can recognize speech in several languages at a time. This is necessary for systems that run two or more applications written for different languages.
SRFEATURE_PCOPTIMIZED | The voice is optimized to work over the PC, which provides better sound quality.
SRFEATURE_PHONEOPTIMIZED | The voice is designed to work over the telephone and uses an 8-kHz sampling rate.
SRFEATURE_TRAINPHONETIC | The speaker can train the engine on a preselected set of words and thereby train all of the phonemes used for recognition. This is done in general training.
SRFEATURE_TRAINWORD | The speaker can train the engine on individual words.
SRFEATURE_WILDCARD | The engine supports wildcards in context-free grammars.

dwInterfaces

Interfaces that the engine supports. This member can be a combination of these values:

Value | Meaning
SRI_ILEXPRONOUNCE | The ILexPronounce interface
SRI_ISRATTRIBUTES | The ISRAttributes interface
SRI_ISRCENTRAL | The ISRCentral interface
SRI_ISRDIALOGS | The ISRDialogs interface
SRI_ISRGRAMCOMMON | The ISRGramCommon interface
SRI_ISRGRAMCFG | The ISRGramCFG interface
SRI_ISRGRAMDICTATION | The ISRGramDictation interface
SRI_ISRGRAMINSERTIONGUI | The ISRGramInsertionGUI interface
SRI_ISRESBASIC | The ISRResBasic interface
SRI_ISRESMERGE | The ISRResMerge interface
SRI_ISRESAUDIO | The ISRResAudio interface
SRI_ISRESCORRECTION | The ISRResCorrection interface
SRI_ISRESEVAL | The ISRResEval interface
SRI_ISRESGRAPH | The ISRResGraph interface
SRI_ISRESMEMORY | The ISRResMemory interface
SRI_ISRESMODIFYGUI | The ISRResModifyGUI interface
SRI_ISRESSPEAKER | The ISRResSpeaker interface
SRI_ISRSPEAKER | The ISRSpeaker interface
SRI_ISRESSCORES | The ISRResScore interface

dwEngineFeatures

Features that are specific to the engine. These values are defined by the engine author. For more information, see the documentation for the engine.

The features provided by a given engine are defined by the engine author. An application can enumerate the engines to identify which engine is being used in order to determine the features it provides. For more information about enumerating engines, see the description of the ISREnum interface.

Although an engine can support both PC optimized and phone optimized voice qualities, most support either one or the other.

Some engines support recognition from several languages at a time. For more information about recognizing multiple languages, see the description of the ISRCentral::GrammarLoad member function.


SRMODEINFORANK

typedef struct { // srmir
DWORD dwEngineID;
DWORD dwMfgName;
DWORD dwProductName;
DWORD dwModeID;
DWORD dwModeName;
DWORD dwLanguage;
DWORD dwDialect;
DWORD dwSequencing;
DWORD dwMaxWordsVocab;
DWORD dwMaxWordsState;
DWORD dwGrammars;
DWORD dwFeatures;
DWORD dwInterfaces;
DWORD dwEngineFeatures;

} SRMODEINFORANK, *PSRMODEINFORANK;

Ranks the members of the SRMODEINFO structure in the order of importance.

dwEngineID

Relative importance of a matching engine identifier.

dwMfgName

Relative importance of the specified manufacturer name.

dwProductName

Relative importance of the specified product name.

dwModeID

Relative importance of a matching mode identifier.

dwModeName

Relative importance of the specified mode name.

dwLanguage

Relative importance of the specified language.

dwDialect

Relative importance of the specified dialect. This value is stored in the LANGUAGE structure of the SRMODEINFO structure.

dwSequencing

Relative importance of the specified sequencing type.

dwMaxWordsVocab

Relative importance of the maximum vocabulary size. Modes that do not meet or exceed the specified value are eliminated from consideration.

dwMaxWordsState

Relative importance of the maximum number of words in any state of a grammar. Modes that do not meet or exceed the specified value are eliminated from consideration.

dwGrammars

Relative importance of the requested grammars. Modes that do not support the requested grammars are eliminated from consideration.

dwFeatures

Relative importance of the requested features. Modes that lack the features requested are eliminated from consideration.

dwInterfaces

Relative importance of the requested interfaces. Modes that lack the interfaces requested are eliminated from consideration.

dwEngineFeatures

Relative importance of the requested engine features. Modes that lack the features requested are eliminated from consideration.

Each member of the SRMODEINFORANK structure contains a priority value for the corresponding feature. Values are relative to each other — that is, a higher value indicates a higher priority, but the value itself is not meaningful.

The ISRFind::Find member function searches for the speech-recognition mode that most closely matches the attributes specified in the SRMODEINFO structure, using the rankings of SRMODEINFORANK to select the mode with the most desirable features.


SRPHRASE

typedef struct { // srp
DWORD dwSize;
BYTE abWords[0];
} SRPHRASE, *PSRPHRASE;

Identifies a unique phrase for a grammar's vocabulary. A phrase is an ordered collection of words.

dwSize

Size of SRPHRASE plus the size of data for abWords.

abWords

Memory containing a list of SRWORD structures, in temporal order.


SRRESPHONEMENODE

typedef struct {
DWORD dwNextPhonemeNode;
DWORD dwUpAlternatePhonemeNode;
DWORD dwDownAlternatePhonemeNode;
DWORD dwPreviousPhonemeNode;
DWORD dwWordNode;
QWORD qwStartTime;
QWORD qwEndTime;
DWORD dwPhonemeScore;
WORD wVolume;
WORD wPitch;
} SRRESPHONEMENODE, *PSRRESPHONEMENODE;

Provides information about a phoneme node in a recognition/alternative graph generated by a speech-recognition engine.

dwNextPhonemeNode

Number of the node that follows next in time. If the next node has several alternatives, this pointer should continue along the path that produces the highest path score at the final node. If there is nothing left in the path, this can be NULL.

dwUpAlternatePhonemeNode

Number of an alternative node with a higher score than that of the phoneme in this node. The alternative nodes should be ordered according to the highest score for the phoneme. If there are no more alternatives, this can be NULL.

dwDownAlternatePhonemeNode

Number of an alternative node with a lower score than that of the phoneme in this node. The alternative nodes should be ordered according to the highest score for the phoneme. If there are no more alternatives, this can be NULL.

dwPreviousPhonemeNode

Number of the previous node. If there are several previous nodes, this maps back along the highest path score at the final node. If there is nothing before this in the path, this can be NULL.

dwWordNode

Node number of the word node to which this phoneme most likely belongs. If this phoneme is spurious and doesn't belong to a word node, this is NULL.

qwStartTime

Time stamp, in bytes, when the audio for this node started. If the value is indeterminate, this is zero.

qwEndTime

Time stamp, in bytes, when the audio for this node ended. If the value is indeterminate, this is zero.

dwPhonemeScore

Score for this individual phoneme. It is valid to compare this score only against the scores of this phoneme's alternatives. Engines should make this value a linear probability if possible. If an engine has no information about the value, this is zero.

wVolume

Linear volume level of the phoneme, from 1 to 0xFFFF. If the engine has no information about the volume, this should be zero. An application can use this information to determine emphasis or sentence type (for example, question or command) or to do transplanted prosody.

wPitch

Average pitch of the phoneme, in hertz. Engines that do not know this value set it to zero. Obviously, unvoiced sounds have no pitch. Applications can use this information to determine emphasis or sentence type (for example, question or command) or to do transplanted prosody.


SRRESWORDNODE

typedef struct { // srrwn
DWORD dwNextWordNode;
DWORD dwUpAlternateWordNode;
DWORD dwDownAlternateWordNode;
DWORD dwPreviousWordNode;
DWORD dwPhonemeNode;
QWORD qwStartTime;
QWORD qwEndTime;
DWORD dwWordScore;
WORD wVolume;
WORD wPitch;
VOICEPARTOFSPEECH pos;
DWORD dwCFGParse;
DWORD dwCue;
} SRRESWORDNODE, *PSRRESWORDNODE;

Provides information about a word node in a recognition/alternative graph generated by a speech-recognition engine.

dwNextWordNode

Node number of the node that follows next in time. For example, if the current node contains "Mail," the next node would contain "To." If the next node has several alternatives, this pointer should continue along the path that produces the highest path score at the final node. If there is nothing left in the path, this can be NULL.

dwUpAlternateWordNode

Number of an alternative node with a higher score than that of the word in this node. For example, if this node contains "Nail," the alternative node might contain "Mail." The alternative nodes should be ordered according to the highest word score. If there are no more alternatives, this can be NULL.

dwDownAlternateWordNode

Number of an alternative node with a lower score than that of the word in this node. For example, if this node contains "Mail," the alternative node might contain "Nail." The alternative nodes should be ordered according to the highest word score. If there are no more alternatives, this can be NULL.

dwPreviousWordNode

Node number of the previous node. If there are several previous nodes, this maps back along the highest path score at the final node. If there is nothing before this in the path, this can be NULL.

dwPhonemeNode

Node number of the first phoneme in the word. If an engine doesn't support phonemes or if such a mapping is inappropriate, this should be NULL.

qwStartTime

Time stamp, in bytes, when the audio for this node started. If the value is indeterminate, this is zero.

qwEndTime

Time stamp, in bytes, when the audio for this node ended. If the value is indeterminate, this is zero.

dwWordScore

Score for this individual word. It is valid to compare this score only against the scores of this word's alternatives. Engines should make this value a linear probability if possible. If an engine doesn't know the value, this is zero.

wVolume

Linear volume level of the word, from 1 to 0xFFFF. If the engine doesn't know the volume, this should be zero. Applications can use this information to determine emphasis or sentence type (for example, question or command) or to do transplanted prosody.

wPitch

Average pitch of the word in hertz. Engines that do not have information about this value set it to zero. Applications can use this information to determine emphasis or sentence type (for example, question or command) or to do transplanted prosody.

pos

Part of speech of the word, if known.

dwCFGParse

Unique identifier of the rule used to parse the word. This is the rule identifier from the SRCFGRULE structure that defines the rule in the grammar. (This member is valid only for context-free grammars.)

If an imported rule was used to parse the word, dwCFGParse contains the unique rule identifier from the SRCFGIMPRULE structure that defines the imported rule in the grammar. An application can use this information to determine the semantics of a word and save itself the work of parsing the tree. For example, suppose the context-free grammar contains the follow rules:

1 = SEQ(Send mail to 2)

2 = ALT(Mike, Bob, Fred)

If the engine recognizes "Send mail to Bob," the value of dwCFGParse would be 1 (for "Send mail to") and 2 (for "Bob").

dwCue

Cue that the engine has recognized but cannot translate into a word string. A node with a cue may not have a word associated with it. An engine is not required to return cues so an application should not expect them, although it should take advantage of cues that are returned. This member can be one of these values.

Value | Meaning
SRRESCUE_COMMA | A comma should go here.
SRRESCUE_DECLARATIVEBEGIN | A declarative sentence has begun. This cue must be followed by a closing SRRESCUE_DECLARATIVEEND.
SRRESCUE_DECLARATIVEEND | A declarative sentence has ended.
SRRESCUE_IMPERATIVEBEGIN | An imperative sentence has begun. This cue must be followed by a closing SRRESCUE_IMPERATIVEEND.
SRRESCUE_IMPERATIVEEND | An imperative sentence has ended.
SRRESCUE_INTERROGATIVEBEGIN | An interrogative sentence has begun. This cue must be followed by a closing SRRESCUE_INTERROGATIVEEND.
SRRESCUE_INTERROGATIVEEND | An interrogative sentence has ended.
SRRESCUE_NOISE | A noise has occurred.
SRRESCUE_PAUSE | The speaker has paused.
SRRESCUE_SENTENCEBEGIN | A sentence has begun. This cue must be followed by a closing SRRESCUE_SENTENCEEND. If the engine has identified the sentence as interrogative, declarative, or imperative, it should use the more specific SRRESCUE_INTERROGATIVEBEGIN, SRRESCUE_DECLARATIVEBEGIN, or SRRESCUE_IMPERATIVEBEGIN cues.
SRRESCUE_SENTENCEEND | A sentence has ended. If the engine has identified the sentence as interrogative, declarative, or imperative, it should use the more specific SRRESCUE_INTERROGATIVEEND, SRRESCUE_DECLARATIVEEND, or SRRESCUE_IMPERATIVEEND cues.
SRRESCUE_UM | The speaker has said a nonsense sound, such as "Um."
SRRESCUE_WILDCARD | This is a placeholder for a wildcard word in a context-free grammar.
SRRESCUE_WORD | The node contains a word.


SRSHARE

typedef struct { // srs
QWORD qwInstanceID;
DWORD dwDeviceID;
SRMODEINFO srModeInfo;
} SRSHARE, *PSRSHARE;

Provides information about a shared engine/audio-source pair.

qwInstanceID

Instance identifier used to identify an instance of sharing an engine/audio-source pair.

dwDeviceID

Device identifier of the audio source.

srModeInfo

SRMODEINFO structure that provides details about the speech-recognition mode.


SRWORD

typedef struct { // srw
DWORD dwSize;
DWORD dwWordNum;
TCHAR szWord[ ];
} SRWORD, *PSRWORD;

Identifies a unique word for a grammar's vocabulary.

dwSize

Size of SRWORD plus the size of szWord (string) rounded up to the nearest multiple of four. The dwSize member must be DWORD-aligned to work on processors that require alignment, such as MIPS® and Alpha APX™.

dwWordNum

Unique word identifier specific to this grammar. The same string can have a different word identifier in different grammars or in the same grammar if the word participates in the grammar in different ways. For example, the noun "project" and the verb "project" each would have a unique word identifier within the same grammar.

The application always specifies the word identifier when it loads a grammar. The application cannot specify a word identifier of zero because the grammar also uses a word identifier to notify the application when the corresponding word is spoken, and zero is reserved for situations when the word identifier is unknown.

szWord

Null-terminated string of variable size containing alphabetic characters and intra-word punctuation. The null at the end of the string is sufficient to indicate the end of the string.

In most European languages, szWord is a string such as "hello" or "car." The text should not contain pronounced symbols. For example "345" is not a valid word string. The text also should not have ambiguous pronunciation. For example, instead of the string "IEEE," use three words: "I" "triple" "E".

Certain languages such as Japanese use two character: a phonemic set and a symbolic set. The phonemic character set should be passed in szWord. If an application is trying to associate the symbolic representation of the word, it should use dwWordNum to point to the symbolic word.

In some languages the pronunciation cannot be deduced easily from the symbols that represent the word. In these languages, the phonetic character set appears first, followed by a backslash, followed by the commonly displayed character set. The engine should use the text before the backslash to determine the pronunciation of the word. For example, the Japanese names Honcho and Motomachi are pronounced differently but "spelled" with the same kanji characters.


Speech-Recognition Error Values

The following errors may be returned by speech-recognition objects. If no error occurs, the member functions return NOERROR.

Error value | Description
SRERR_GRAMMARERROR | Error in the grammar.
SRERR_GRAMMARTOOCOMPLEX
 | Grammar is too complex to load or to be activated.
SRERR_GRAMMARWRONGTYPE
 | This grammar type is not supported.
SRERR_GRAMTOOLARGE | Grammar is too large to load.
SRERR_INVALIDCHAR | Invalid character in a grammar or in a list.
SRERR_INVALIDFLAG | One of the flags is invalid.
SRERR_INVALIDINTERFACE | Invalid interface for notification.
SRERR_INVALIDKEY | Invalid key value.
SRERR_INVALIDLIST | Invalid list.
SRERR_INVALIDMODE | Invalid speech mode.
SRERR_INVALIDRULE | Invalid rule is activated.
SRERR_INVALIDWINDOW | Window is invalid.
SRERR_NOUSERSELECTED | No speaker was specified.
SRERR_NOTENOUGHDATA | Not as much data in SDATA as anticipated; or, no longer has any digital-audio data because the object has deteriorated or the memory has been freed.
SRERR_NOTSUPPORTED | A function, link, archive, or something in a grammar is not supported by this engine.
SRERR_OUTOFDISK | Out of disk space.
SRERR_RULEALREADYACTIVE | The rule is already active.
SRERR_RULENOTACTIVE | The rule is not active.
SRERR_SPEAKEREXISTS | Speaker already exists on the current system.
SRERR_TOOMANYGRAMMARS | Cannot load the grammar because too many are already loaded.
SRERR_VALUEOUTOFRANGE | A value is out of range.
SRERR_WAVEFORMATNOTSUPPORTED
 | No wave formats supported.

In addition to the error values described in the preceding list, member functions can return error values generated by lower-level component objects or calls to the Win32 API. For example, a member function of the ISREnum interface might return an AUDERR_ value generated by the audio-destination object. For more information, see "Object Levels and Error Values."


Standard Grammar Formats

A grammar defines the words or phrases that an application can recognize. Speech recognition is based on grammars. This appendix describes the standard grammar formats used by context-free and dictation grammars.


Types of Grammars

An application can perform speech-recognition by using three different kinds of grammars: context-free, dictation, and limited domain. Each grammar uses a different strategy for narrowing the set of sentences it will recognize:

· Context-free grammar. Uses rules that predict the next words that might possibly follow the word just spoken, reducing the number of candidates to evaluate in order to recognize the next word.

· Dictation grammar. Defines a context for the speaker by identifying the subject of the dictation, the expected style of language, and what dictation has already been done.

· Limited-domain grammar. Does not provide strict syntax structures, but does provide a set of words to recognize. Limited-domain grammar is a hybrid between a context-free grammar and a full dictation grammar.

The approach used by a speech-recognition engine to accept verbal input influences the approach used by the grammar to recognize a phrase. A speech-recognition engine accepts verbal input using one of the following technologies:

· Discrete speech. Each word must be isolated by pauses before and after the word.

· Word spotting. Only certain words are recognized in an utterance.

· Continuous speech. Given a continuous utterance with no pauses between words, the engine can recognize the words that were spoken.

Each of these technologies can influence how the grammar recognizes a phrase in the following ways:

Technology | Context-free grammar | Dictation or limited-domain grammar
Discrete speech | A phrase is completed when one path of the grammar is traversed by the isolated words. A path is abandoned when an incorrect word is spoken or the delay between words is too long. | A phrase is sent to the application when the engine isolates the word.
Word spotting | The grammar paths specify what words are to be spotted in what sequence. For example, if one of the grammar paths is "mail" followed by "Fred," then "Send all of my MAIL to FRED Smith" would complete the path. | Keywords are spotted in the utterance and a phrase is sent when the engine determines that the keyword was spoken.
Continuous speech | Words can be spoken in a fluid manner and are allowed to blend into one another, although slight pauses are allowed. A path is abandoned when an incorrect word is spoken or the delay between words is too long. | The engine parses the continuous stream, and when it determines a sequence of words has been spoken, it collects them into a phrase and sends them to the application.


General Format of a Grammar

Context-free, dictation, and limited-domain grammars use the same general format, which consists of a header, followed by a list of information chunks. Each grammar can contain one or more information chunks that follow each other sequentially. Several chunks of the same type—for example, Sample Text—can be included in the same grammar. The chunk structure allows the grammar to be extensible and allows a speech-recognition engine to skip information that is not relevant or not known to the engine.

A grammar can have only one SRCK_LANGUAGE chunk, which identifies the language of the grammar. If the grammar does not have a SRCK_LANGUAGE chunk, the grammar will use the current speech-recognition mode's default language. Some engines may not be able to load a grammar if it is for the wrong language.

The rest of this appendix describes significant features of context-free, dictation, and limited-domain grammars and provides reference information for each structure.


Context-Free Grammars

To reduce failed recognitions and computation, current speech-recognition technology requires a list of valid sentences that can be spoken. However, such a list could include thousands or even a million entries. A context-free grammar scheme allows an engine to reduce the possible entries to a list of several hundred rules.


Examples of Typical Rules

The following examples show typical rules that might be defined for a context-free grammar.

The rule syntax shown here is for purposes of illustration only, and is not actually understood by the engine. The syntax for rules is up to the grammar designer.

Rules That Specify Lists

A rule might specify a list of names such as "Bob," "Fred," "George," and "Bill," allowing the speaker to say only one of those names. Such a rule might appear as follows:

<Names>=ALT("Bob", "Fred", "George")

In this example, ALT indicates that only one of the words can be spoken.

If the names are longer than one word, the rule might take either of the following forms:

<Names>=ALT("Bob Marks", "Fred Smith", "George Hill")

or

<Names>=ALT(SEQ("Bob", "Marks"), SEQ("Fred", "Smith"), ...)

In this example, SEQ indicates a sequence of words to be spoken.

As an alternative, the last name might be optional, in which case the rule might take the following form:

<Names>=ALT(SEQ("Bob",OPT("Marks")), ...

In this example, OPT indicates that the word can be spoken but is not required. This is equivalent to:

<Names>=ALT("Bob", "Bob Marks", "Fred", "Fred Smith", ...)

Rules That Refer to Lists

A list of names is insufficient by itself for most recognition, but the list can be used as a component of another rule. For example, if several different actions — such as "Send mail to <Name>" or "Who is <Name>" — could be performed on a specific name, a rule might include alternatives and sequences that refer to the <Names> rule. For example:

<Actions>=ALT(SEQ("Send mail to",<Name>), SEQ("Who is", <Name>)), ...)

This example is equivalent to the following:

<Actions>=ALT("Send mail to Bob", "Send mail to Bob Marks",

"Send mail to Fred", "Send mail to Fred Smith", ...)

A repeat operator, which allows its contents to be repeated indefinitely, is also useful. Rules are recursive, so the restriction of a single operand for repeat is not significant. For example:

<MoreActions>=SEQ(<Actions>, OPT(REP(SEQ("and", <Names>))))

This kind of rule allows the speaker to say such things as "Send mail to Fred" or "Send mail to Fred and Bob and Bill."

Lists That Are Filled at Run Time

A rule can also refer to a list that is filled at run time after the grammar has been loaded. Rather than send text strings to the engine, which is too slow, the grammar sends a list of structures filled with the equivalent operations and symbols. Although lists may require more processing to recognize, they allow grammars to be modified dynamically for such things as filenames that change frequently and often depend upon context. For example:

<Actions> = ALT(LIST(<Names>))

For example, if the <Names> list contains "Bob Marks" and "Fred Smith" at run time, the speaker can say "Send mail to Bob Marks" and "Send mail to Fred Smith."


Rule Activation

To begin using a context-free grammar to recognize speech, the application must first activate a specific rule in the grammar. This puts the rule at the starting node of the recognition.

Continuing the examples described at the beginning of this section, if an application activates the <Actions> rule, the speaker can say "Send mail to Bob Marks." However, if the application activates the <Names> rule instead, the user can say "Bob Marks" but cannot say "Send mail to Bob Marks."

Note that a grammar can have several rules activated at one time. For example, if both <Actions> and <Names> are activated, the speaker can say "Send mail to Bob Marks" or "Bob Marks."

The context-free grammar format requires a list of all of the rules that can be activated (called exported rules). Grammar designers should keep this list to a minimum because the efficiency of the grammar — both the processing required and the accuracy of recognition — can be affected by the number of exported rules.


Automatic Linking

Some engines support automatic linking between grammars. When grammars are linked, one grammar indicates that it imports a rule from another grammar and identifies the rule name by a string. If an application loads another grammar that exports a rule of that name, the first grammar borrows that rule and all of the vocabulary referenced by the rule from the second grammar, even if the second grammar is not activated. In the case of a conflict, the first grammar links to the grammar that has been loaded the longest.

For example, a grammar that allows the speaker to say "Set the time to <Time>" might list <Time> as an imported rule. A second grammar might have a <Time> rule that provides the vocabulary and syntax for recognizing the time. As soon as the second grammar is loaded, the first grammar would automatically link its <Time> rule to the second grammar's <Time> rule.

Automatic linking allows third parties to develop improved or shared grammars for commonly used syntaxes, such as numbers, dates, and times. If a third party wishes to keep the grammar format secret, the grammar can be compiled into the custom formats of each engine that is supported, rather than shipping the grammar in a standard format.

Automatic linking should be used with care because name conflicts can occur or grammars can change invisibly to the application. If an imported rule cannot be linked, the original grammar should still work even though portions of it may be inaccessible. In addition, IDs of words that originate from different grammars will be invalid because the application won't be able to identify which grammar provided the word.


Run-Time Lists

Some engines support run-time lists, which allow a grammar to indicate a list that will be supplied at run time. This provides fill-in-the-blank functionality for a grammar for information such as e-mail names, filenames, fonts, and other values that cannot be predicted when the grammar is compiled.

The syntax for using a run-time list is the same as for words as rules. If a list is not filled in at run time, the grammar should still work even though portions of it may be inaccessible.


Recursion and Perplexity

Context-free grammars are very powerful tools because of their recursion ability, but they can also be made overly complex. The efficiency of a speech-recognition engine deteriorates when using grammar with too much recursion (more than several levels) or too much perplexity, which is essentially the number of possible choices at a given node in a recognition path.


Information Chunks in Context-Free Grammars

The context-free grammar follows the general grammar format described in the section "General Format of a Grammar," earlier in this section. A context-free grammar contains one (and only one) of each of the following information chunks:

· SRCKCFG_WORDS (optional)

· SRCKCFG_RULES (required)

· SRCKCFG_EXPORTRULES (required)

· SRCKCFG_IMPORTRULES (optional)

· SRCKCFG_LISTS (optional)

Words (SRCKCFG_WORDS)

This chunk contains a list of SRWORD structures that specify the words in the grammar.

Rules (SRCKCFG_RULES)

This chunk contains a list of SRCFGRULE structures that specify the rules for the grammar.

Exported Rules (SRCKCFG_EXPORTRULES)

This chunk contains a list of SRCFGXRULE structures that specify the exported rules for the grammar. When an application activates a context-free grammar, it must specify which exported rule to activate. If the engine supports automatic linking, this is a list of rules to link. A grammar must have at least one exported rule. Many engines lose efficiency when using too many exported rules, so keep exported rules to a minimum. Rule names are case insensitive.

Imported Rules (SRCKCFG_IMPORTRULES)

This chunk contains a list of SRCFGIMPRULE structures that specify the imported rules for the grammar. Imported rules are automatically linked to exported rules as soon as the application loads another grammar that has the same rule name as an imported rule. Some engines do not support automatic linking and cannot load grammars with imported rules. Rule names are case insensitive.

Run-Time Lists (SRCKCFG_LISTS)

This chunk contains a list of SRCFGLIST structures that specify the run-time lists that the grammar requires. List names are case insensitive.


Text Representation of Context-Free Grammars

Speech-recognition tools and engine vendors will supply tools for compiling context-free grammars into the standard context-free grammar binary format. These compilers will support both engine-specific formats and the following engine-independent text format.

The file containing a grammar in text format is an ANSI text file that consists of a series of comments, word definitions, and rule definitions.

Comments

A comment takes one of two forms, which follow the same rules as standard comments in C:

// Comment on a single line

/* Comment that can span multiple lines */

Word Definitions

A word definition uses the following syntax:

WordName = Number;

The number specifies a numeric ID that identifies the word in an SRWORD structure. If a word's number is not defined in the file before it is referenced by a rule, a unique DWORD ID above 0x8000000 will be chosen automatically.

The DWORD is unique within the context of the grammar and for a specific object type. If the words loaded in a list use the same ID, the application will not have sufficient information to identify which word was spoken, unless it processes the text being returned with the word. Of course, an application should do this if, for example, "zero" and "oh" both mapped to the same semantic information.

When a rule is imported by a grammar, the word IDs referenced by the imported rules are not necessarily unique compared to the grammar's word IDs.

Rule Definitions

A rule definition uses the following syntax:

<RuleName> = Expression;

This specifies a rule to be put in the grammar's list of SRCFGRULE structures.

The rule name must be enclosed in angled brackets (for example, <Start> or <_Number> ). Unless the rule name has an underscore as its second character, it will be exported and put in the grammar's list of SRCFGXRULE structures.

Expressions

An expression can be one of the following:

Expression | Result
{RuleName} | The expression refers to another rule. If the rule is not defined in the text file, then it will be used as an imported rule in the SRCFGIMPRULE structure.
<ListName> | The expression refers to a list. If the list has not already been used, then a unique identifier for it will be generated by the compiler and stored in a SRCFGLIST structure.
Word | The expression consists of a word. A word can start with any alphabetic character, underscore (_), or hyphen (-). The rest of the word can consist of one or more alphanumeric characters. A word cannot contain white space. The word identifier is obtained from a previous word definition. If the word has not been already defined, then a definition will be implicitly generated.
(Expression) | The parentheses control the order of operation.
[Expression] | The expression is optional, using the SRCFGO_OPTIONAL value.
Expression+ | (Repeat0) The expression can be repeated one or more times. It compiles to a SRCFGO_REPEAT value.
Expression* | (Repeat1) The expression can be repeated zero or more times. This is equivalent to [(Expression)+].
Expression Expression | (Sequence) Two or more expressions separated by a space compile into a set of words that are spoken sequentially, using the SRCFGO_SEQUENCE value.
Expression | Expression | (Alternative) Two or more expressions separated by a vertical bar compile into an alternate set of words, using the SRCFGO_ALTERNATIVE value.

The order of operations is:

1. Expressions in parentheses, optional expressions

2. Repeat0, Repeat1

3. Sequence

4. Alternative

Example of a Text Representation of a Grammar

For example, suppose a grammar is supposed to handle the following commands:

· What time is it?

· What day is it?

· Send mail to {E-mail}

The text representation of the grammar would look like the following:

time = 103;

day = 104;

send = 105;

{Start} = What {_TimeDay} is it | Send mail to <Email>;

{_TimeDay} = time | day;


Dictation Grammars

A dictation grammar defines a context in which to speak, which loosely identifies the subject of the dictation, the kind of language style that is expected, and what dictation has been done in the past.

A dictation grammar does not contain any information about the language model (the engine has that information) nor does it specify all of the words that can be spoken. It specifies only unusual words or expected groups of words.


Information Chunks in Dictation Grammars

The dictation grammar supports the following kinds of information chunks.

· SRCKD_COMMON

· SRCKD_GROUP

· SRCKD_SAMPLE

· SRCKD_TOPIC

· SRCKD_NGRAM

All chunks are optional, and the grammar can contain more than one of each.

Common Words (SRCKD_COMMON)

This chunk contains a list of common words, stored as a sequential list of null-terminated word strings. Common words will probably be used in the dictation, but they may not be expected by the language model. For example:

"Automobile\0Franchise\0Repair\0Muffler\0Drive Shaft"

Word Groups (SRCKD_GROUP)

This chunk contains the same information as SRCKD_COMMON. Word groups are related in some way but are not automatically associated by the language model. For example, a dictation system for the air travel industry would supply a list of cities with major airports, such as:

"New York City\0Boston\0Los Angeles\0Chicago\0Seattle"

Sample Text (SRCKD_SAMPLE)

This chunk contains sample text, stored as a single null-terminated string. The sample text might be from a document written by the speaker or a document similar to the one that the speaker will dictate. It allows the engine to identify a topic, common words, and authoring style. The text can be any length, although most sample text will be only a few pages long (not enough to form the basis for a language model).

Topic Keyword (SRCKD_TOPIC)

This chunk contains a null-terminated string of a topic keyword. Topic keywords categorize the dictation into topics, such as "legal" and medical."

A grammar might have several topics ranging from very general to very specific, so that if the engine doesn't recognize a specific topic it may recognize a more general one.

N-Gram Probability File (SRCKD_NGRAM)

(New for 3.0)

This chunk contains N-gram information so an engine will know the probabilities of words given a context. An application might wish to scan documents similar to what the user will be dictating and produce an N-gram chunk. The N-gram chunk has the following format:

typedef struct {

QWORD dwTotalCounts

DWORD dwNumWordsClasses;

DWORD dwWordClassNameOffset;

DWORD dwNumContextGroups;

DWORD dwContextGroupOffset;

DWORD dwNumClasses;

DWORD dwClassOffset;

BYTE bBitsPerWord

BYTE abFiller[3];

DWORD adwProbability[256];

} NGRAMHDR, * PNGRAMHDR;

dwTotalCounts - Total number of words "scanned" to produce the N-gram data chunk. This affects how much the N-gram overrides the existing language model. At -1, the default language model provided by the dictation engine is completely overridden. For lower values, the engine will do some interpolation between its built in language model and the application model. The amount of interpolation is left up to the engine.

dwNumWordsClasses - Number of words/classes used by the N-gram.

dwWordClassNameOffset - Offset (in bytes) from the beginning of the chunk where the word/class name data begins. The words/class name data is stored as list of NULL-terminated strings. Words/classes are automatically assigned word ID, sequentially starting at 1. It's possible to have an empty string for a class. If a word/class ID does not appear in the class definitions then it is a word ID. To minimize memory, applications may want to sort classes and words so the most frequent classes and words appear at the top of the list.

dwNumContextGroups - Number of context groups stored in the N-gram chunk.

dwContextGroupOffset - Offset (in bytes) from the beginning of the chunk where the context groups begin. Context groups will be stored sequentially, one after the other. An application can jump to the next context group by skipping forward NGRAMCGHDR->dwSize from the beginning of the current context group.

dwNumClasses - Number of classes stored in the N-gram chunk.

dwClassOffset - Offset (in bytes) from the beginning of then chunk where the class definitions begin. . Class definitions will be stored sequentially, one after the other. An application can jump to the next class definition by skipping forward NGRAMCGHDR->dwSize from the beginning of the current class definition.

bBitsPerWord - Number of bits to represent a word ID. This can be 16 or 32.

adwProbability - This is an array of 256 DWORDs, which map a probability token (stored in a byte) into a 4-byte linear probability value. 0xfffffff = 1.0, 0x80000000 = 0.5, etc. Thus, if a probability for a word in a context is token 52, then the real probability is adwProbability[52].

An N-gram chunk will have thousands (perhaps millions) or context groups. Context groups must be written in sorted order by adwWordClassID[0..3]. (Example: The context (25,54,0,0) comes before (25, 55, 0, 0), and also comes before (25, 54, 10, 0).) Each context group has a header:

typedef struct {

DWORD dwSize;

QWORD dwTotalCounts

DWORD adwWordClassID[4];

DWORD dwNumSequential;

DWORD dwNumRandomAccess;

BYTE bBackOffProbToken;

BYTE abFiller[3];

// WORD or DWORD awRandomAccessWordsClasses[dwNumRandomAccess];

// BYTE abProbToken[dwNumSequential+dwNumRandomAccess];

} NGRAMCGHDR, *PNGRAMCGHDR;

dwSize - Number of bytes in this context group, including the header, rounded up to the nearest DWORD (4 bytes). The dwSize field can be used to skip to the next context group.

dwTotalCounts - Total number of words "scanned" to produce context. This affects how much the N-gram overrides the existing language model. At -1, the default language model (for this context) provided by the dictation engine is completely overridden. For lower values, the engine will do some interpolation between its built in language model and the application model. The amount of interpolation is left up to the engine.

adwWordClassID - Array of word ID's that define the context. Unused context information is set to 0. Thus, to represent a context of "new York" for a tri-gram table, adwWordClassID[0] would be the ID of "new", and adwWordClassID[1] would be the ID of "York", while all other values would be 0. This array must be unique within the context groups in the grammar; There cannot be two "new York" contexts. Applications should avoid using a class ID in the context since its effect is undetermined.

dwNumSequential - Number of sequential word/class probability tokens. (See below for details.)

dwNumRandomAccess - Number of random access word/class probability tokens. (See below for details.)

bBackOffProbToken - Probability token that indicates the probability that the context does not contain the "correct" word sequence and that the (N-1)-gram model should be used. Thus, with the context "new York", this is the probability that the "new" context should be used instead. The probability token can be converted to a real probability by indexing it into adwProbability[].

Immediately after each NGRAMCGHDR comes the following data:

awRandomAccessWordsClasses[dwNumRandomAccess] - Array of WORD or DWORD structures (depending upon bBitsPerWord) with dwNumRandomAccess elements. These are the word IDs used to reference the probability tokens. Word/class IDs must be in ascending order so that an engine can access the data quickly.

abProbToken[dwNumSequential+dwNumRandomAccess] - Array of (dwNumSequential + dwNumRandomAccess) bytes representing probability tokens. The first dwNumSequential bytes correspond to the probabilities for word ID's 1 through dwNumSequential. The remaining dwNumRandomAccess bytes correspond to the word ID defined by awRandomAccessWords. (The probabilities, including the back-off, must sum to 1.0) The probability (prob) of word/class N is:

if (N <= dwNumSequential)

prob = adwProbability[abProbToken[N-1]];

else {

for (I = 0; I < dwNumRandomAccess; I++)

if (awRandomAccessWordsClasses[I] == N)

break;

if (I < dwNumRandomAccess)

prob = adwProbability[abProbToken[I+dwNumSequential]];

else

prob = abProbToken[bBackOffProbToken] *

(N-1)-gram context probability

}

An N-gram chunk might have class definitions that group like words and serve to compress the N-gram's size. Class definitions must be written in sorted order by dwClassID. (Example: The class definition for class ID 10 comes before the definition for class ID 28.) Each class definition has a header:

typedef struct {

DWORD dwSize;

QWORD qwTotalCounts;

DWORD dwClassID;

DWORD dwNumSequential;

DWORD dwNumRandomAccess;

// WORD or DWORD awRandomAccessWordsClasses[dwNumRandomAccess];

// BYTE abProbToken[dwNumSequential+dwNumRandomAccess];

} NGRAMCLASSHDR;

dwSize - Number of bytes in this class definition, including the header, rounded up to the nearest DWORD (4 bytes). The dwSize field can be used to skip to the next class definition.

dwTotalCounts - Total number of words "scanned" to produce class definition. This affects how much the N-gram overrides the existing language model.

dwClassID - ID of the class being defined. A class can only be defined once in an N-gram.

dwNumSequential - Number of sequential word probability tokens. (See below for details.)

dwNumRandomAccess - Number of random access word probability tokens. (See below for details.)

Immediately after each NGRAMCLASSHDR comes the following data:

awRandomAccessWordsClasses[dwNumRandomAccess] - Array of WORD or DWORD structures (depending upon bBitsPerWord) with dwNumRandomAccess elements. These are the word IDs used to reference the probability tokens. Word/class IDs must be in ascending order so that an engine can access the data quickly.

abProbToken[dwNumSequential+dwNumRandomAccess] - Array of (dwNumSequential + dwNumRandomAccess) bytes representing probability tokens. The first dwNumSequential bytes correspond to the probabilities for word ID's 1 through dwNumSequential. The remaining dwNumRandomAccess bytes correspond to the word ID defined by awRandomAccessWords. (The probabilities must sum to 1.0) The probability (prob) of word/class N is:

if (N <= dwNumSequential)

prob = adwProbability[abProbToken[N-1]];

else {

for (I = 0; I < dwNumRandomAccess; I++)

if (awRandomAccessWordsClasses[I] == N)

break;

if (I < dwNumRandomAccess)

prob = adwProbability[abProbToken[I+dwNumSequential]];

else

prob = 0;

}


Limited-Domain Grammars

A limited-domain grammar is a hybrid between a context-free grammar and a full dictation grammar. As when using a dictation grammar, the application does not provide strict syntax structures, but it does provide a set of words to recognize. An application might use a limited-domain grammar for the following:

· Command and control that uses natural-language processing to interpret the meaning of the commands

· Forms data entry in which the scope of the vocabulary is known ahead of time

· Text entry in which the scope of the vocabulary is known ahead of time


Information Chunks in Limited-Domain Grammars

The limited-domain grammar uses the general grammar data-format described earlier in this appendix. It supports the following information chunks:

· SRCKLD_WORDS

· SRCKLD_GROUP

· SRCKLD_SAMPLE

The grammar must contain at least one SRCKLD_WORDS chunk; the SRCKLD_GROUP and SRCKLD_SAMPLE chunks are optional, and the grammar can contain more than one of each.

Words (SRCKLD_WORDS)

This chunk contains a list of words that the engine will recognize, stored as a sequential list of null-terminated word strings. For example:

"Automobile\0Franchise\0Repair\0Muffler\0Drive Shaft"

(The last '\0' is implicit in C notation.)

This list does not have to include words from the closed set of the language, which contains the most common words that are not nouns, verb, or adjectives, because these words are supplied by the engine. The closed set of a language is usually fewer than two thousand words. The closed set of the English language includes words such as "the," "and," "to," and "me."

Word Groups (SRCKLD_GROUP)

This chunk contains the same information as the SRCKLD_WORDS chunk, except the words are also related in some way. For example:

"New York City\0Boston\0Los Angeles\0Chicago\0Seattle"

(The last '\0' is implicit in C notation.)

Sample Text (SRCKLD_SAMPLE)

This chunk contains sample text, stored as a single null-terminated string. The sample text might be from a document written by the speaker or a document similar to the one that the speaker will dictate. It allows the engine to identify a topic, common words, and authoring style. The text can be any length, although most sample text will only be a few pages long (not enough to form the basis for a language model).


Reference

This section describes the structures used by context-free, dictation, and limited-domain grammars.


SRCFGIMPRULE

typedef struct {
DWORD dwSize;
DWORD dwRuleNum;
TCHAR szString[];
} SRCFGIMPRULE, *PSRCFGIMPRULE;

Identifies a rule to be imported by a context-free grammar.

dwSize

Size, in bytes, of SRCFGIMPRULE plus the size of the szString member, rounded up to the nearest multiple of four. The dwSize member must be DWORD-aligned to work on processors that require alignment, such as MIPS and Alpha.

dwRuleNum

Unique rule number specific to this grammar. The syntax portion of the grammar identifies the rule by using this number.

szString

Null-terminated string of variable size that an application uses to identify this rule. The null at the end of this member is sufficient to indicate the end of the string. This is case insensitive.


SRCFGLIST

typedef struct {
DWORD dwSize;
DWORD dwListNum;
TCHAR szString[];
} SRCFGLIST, *PSRCFGLIST;

Identifies a list of words and phrases to be used by a context-free grammar at run time.

dwSize

Size, in bytes, of SRCFGLIST plus the size of the szString member, rounded up to the nearest multiple of four. The dwSize member must be DWORD-aligned to work on processors that require alignment, such as MIPS and Alpha.

dwListNum

A unique list number specific to this grammar. The syntax portion of the grammar identifies the list by using this number.

szString

Null-terminated string of variable size that applications use to identify this list. The null at the end of this member is sufficient to indicate the end of the string. This is case insensitive.


SRCFGRULE

typedef struct {
DWORD dwSize;
DWORD dwUniqueID;
BYTE abData[];
} SRCFGRULE;

Defines a rule in a context-free grammar.

dwSize

Size, in bytes, of SRCFGRULE including the abData member.

dwUniqueID

Rule identifier that is unique to the grammar. A rule identifier can be defined only once in a grammar, and it cannot be 0. Rule identifiers do not have to be ordered or contiguous.

abData

Array of SRCFGSYMBOL structures that define the symbols used in the rule. The first symbol must be an operation.


SRCFGSYMBOL

typedef struct {
WORD wType;
WORD wProbability;
DWORD dwValue;
} SRCFGSYMBOL;

Defines a symbol in a context-free grammar. Each symbol corresponds to an element of a rule.

wType

Symbol type. This member can be one of these values:

Value | Meaning
SRCFG_ENDOPERATION | The end of an operation — for example, ")". The ending of the type of operation specified by the dwValue member.
SRCFG_LIST | A list identifier. Includes a list of words created at run time, identified by the unique list identifier in the dwValue member. The list identifier is specific to the current grammar. The run-time list name can be found in the SRCFGLIST structures stored in the context-free grammar. (May not be supported by all engines.)
SRCFG_RULE | Another rule — for example, "<Names>" A rule is identified by the unique rule identifier in the dwValue member. The identifier is unique for the current grammar.
SRCFG_STARTOPERATION | The beginning of an operation — for example, "ALT(". Everything from here to the ENDOPERATION type is included in the operation specified by the dwValue member. Several occurrences of the same operation type might be contained within this operation.
SRCFG_WILDCARD | Indicates that the engine should accept any amount of verbal input until the next word (specified by SRCFG_WORD) is heard. (May not be supported by all engines.)
SRCFG_WORD | A word — for example, "Bill." A word is identified by the unique word identifier in the dwValue member. The identifier is unique for the current grammar.

The SRCFG_WILDCARD value allows an engine to do a hybrid of word spotting and continuous recognition. For example, a grammar could have "{wild-card} mail {wild-card} <name>" which would allow the user to say "Send mail to Fred," "I want to mail Fred," or even "Mail, yes, I want it to go to Fred." An engine indicates that it supports wildcards by setting the SRFEATURE_WILDCARD flag.

wProbability

Value that indicates a probability that the specific path will be taken, given past experiences. This parameter is valid only for words and rules in a branching operation. This is not on a fixed scale, but the meaning of this member depends upon the other wProbability values within the operation.

The probability of the speaker taking the action can be computed by adding up all of the wProbability values for the operation and dividing that into the wProbability value of an individual word or rule. If the sum of all wProbability values is 0, then all branches are equally probable.

dwValue

Value of the wType member. If wType is SRCFG_STARTOPERATION or SRCFG_ENDOPERATION, dwValue can be one of these values:

Value | Meaning
SRCFGO_ALTERNATIVE | One of the operands must be spoken.
SRCFGO_OPTIONAL | The operand is optional and might not be spoken by the speaker. This can contain only one operand.
SRCFGO_REPEAT | The operand can be spoken by the speaker one or more times. This can contain only one operand.
SRCFGO_SEQUENCE | The operands must be spoken in the given order for the rule to be traversed.

For other wType values, dwValue can be one of these values:

Value | Meaning
SRCFG_LIST | The grammar's unique identifier for the list.
SRCFG_RULE | The grammar's unique identifier for the rule.
SRCFG_WORD | The grammar's unique identifier for the word.


SRCFGXRULE

typedef struct {
DWORD dwSize;
DWORD dwRuleNum;
TCHAR szString[];
} SRCFGXRULE, *PSRCFGXRULE;

Identifies a rule to be exported by a context-free grammar.

dwSize

Size, in bytes, of SRCFGXRULE plus the size of the szString member rounded up to the nearest multiple of four. The dwSize member must be DWORD-aligned to work on processors that require alignment, such as MIPS and Alpha.

dwRuleNum

Unique rule number specific to this grammar. The syntax portion of the grammar identifies the rule by using this number.

szString

Null-terminated string of variable size that applications use to identify this rule. The null at the end of this member is sufficient to indicate the end of the string. This is case insensitive.


SRCHUNK

typedef struct {
DWORD dwChunkID;
DWORD dwChunkSize;
BYTE avInfo[dwChunkSize];
} SRCHUNK, * PSRCHUNK;

Defines an information chunk in a grammar.

dwChunkID

Type of information in the chunk. If the engine does not understand the information type, it skips the chunk. This member can be one of these values:

Value | Meaning
SRCK_LANGUAGE | A LANGUAGE structure. A grammar can have only one of these chunks. If the grammar does not have an SRCK_LANGUAGE chunk, it uses the current speech-recognition mode's default language.
SRCKCFG_EXPORTRULES | A list of SRCFGXRULE structures that specify the exported rules for a context-free grammar.
SRCKCFG_IMPORTRULES | A list of SRCFGIMPRULE structures that specify the imported rules for a context-free grammar.
SRCKCFG_LISTS | A list of SRCFGLIST structures that specify the run-time lists required by a context-free grammar.
SRCKCFG_RULES | A list of SRCFGRULE structures that specify the rules for a context-free grammar.
SRCKCFG_WORDS | A list of SRWORD structures that specify the words in a context-free grammar.
SRCKD_COMMON | A list of common words for a dictation grammar, stored as a sequential list of null-terminated word strings.
SRCKD_GROUP | A list of related words for a dictation grammar, stored as a sequential list of null-terminated word strings.
SRCKD_TOPIC | A topic keyword for a dictation grammar, stored as a single null-terminated string.
SRCKLD_GROUP | A list of related words that an engine will recognize with a limited-domain grammar.
SRCKLD_SAMPLE | Sample text for a limited-domain grammar, stored as a single null-terminated string.
SRCKLD_WORDS | A list of words that an engine will recognize with a limited-domain grammar.

dwChunkSize

Number of bytes used to store the information. This does not include the size of the dwChunkID and dwChunkSize members.

avInfo

Information buffer defined by the dwChunkID member. This must be DWORD-aligned. If the data does not fall on a DWORD boundary, add the number of bytes needed to end the buffer on a DWORD boundary without changing the dwChunkSize member.


SRHEADER

typedef struct {
DWORD dwType,
DWORD dwFlags
} SRHEADER, * PSRHEADER;

Defines the header of a grammar.

dwType

Type of grammar. This member can be one of these values:

Value | Meaning
SRHDRTYPE_CFG | Context-free grammar
SRHDRTYPE_DICTATION | Dictation grammar
SRHDRTYPE_LIMITEDDOMAIN | Limited-domain grammar

dwFlags

If the data stored in the grammar is in Unicode, this member is set to SRHDRFLAG_UNICODE. If the flag is not set, the data is stored in ANSI.


Tips and Tricks


Using Extra Threads

Sometimes calls into the speech recognition engine might take a few hundred to a few thousand milliseconds to complete, especially loading a grammar. If this is too slow for your application, you can move the speech recognition engine into a different thread. Just create a new thread with it's own message loop, initialize OLE COM so it is using apartment model threading, and create the engine.


About Multimedia Audio Objects

Most application developers who incorporate speech recognition or text-to-speech features in their applications want the speech engine to obtain speech input from, or send speech output to, a multimedia wave device. Microsoft provides a pair of component objects, called the multimedia audio-source object and the multimedia audio-destination object, that an application can use to create audio-source and audio-destination objects that give the engine access to the multimedia wave-in and wave-out devices.

To use a multimedia audio object, an application must create an instance of the object and then pass the address of the object's IUnknown interface to the engine. For information about how to create a multimedia audio-source object for use in a speech-recognition application, see the section, "Low-Level Speech Recognition API." For information about how to create a multimedia audio-destination object for use in a text-to-speech application, see the section, "Low-Level Text-to-Speech API."

If your application needs to use an audio source or destination other than a multimedia wave device, you must either write a custom audio object that gives the engine access to that source or destination or use a custom audio object provided by a third-party vendor. A custom audio object must support the IAudio and IAudioSource interfaces.

This section contains reference information about the interfaces and member functions that the multimedia audio objects support. Because the interfaces are intended to be called by an engine rather than an application, the information in this section is intended primarily for engine developers rather than application developers. This information is also intended for developers who need to write a custom audio object. For more information about developing speech engines, see Appendix G, "Developing a Speech Engine."


Objects

Multimedia audio objects allow an application to create audio objects that give a speech engine access to multimedia wave devices. This section describes these objects and the interfaces that they support.


Multimedia Audio-Destination Object


Purpose

The multimedia audio-destination object allows an application to create an audio-destination object based on a multimedia-device driver. The object can be used for text-to-speech.


Created by

The application calls the CoCreateInstance function with the CLSID_MMAudioDest class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-destination object.


Interacts with

When an audio-destination object is first created, it uses the multimedia wave mapper as the destination device, but a valid wave format has not been specified. After an application creates the audio-destination object, it should call the IAudioMultiMediaDevice::DeviceNumSet member function with the device identifier of the audio device to use for output. The engine calls the IAudio::WaveFormatSet member function to set the wave format for the device. The engine should also call the IAudio::PassNotify member function with the address of a notification interface — that is, IAudioDestNotifySink — so that the audio-destination object can send status information to the engine.


Interfaces

The multimedia audio-destination object supports the following interfaces.

Interface | Description
IAudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioDest | Sends information and data to an audio-destination object.
IAudioDestNotifySink | Notifies a text-to-speech engine of changes to the internal buffer of an audio-destination object.
IAudioMultiMediaDevice | Allows an audio-destination or audio-source object to access features specific to multimedia devices.


Multimedia Audio-Source Object


Purpose

The multimedia audio-source object allows an application to create an audio-source object based on a multimedia-device driver. The object can be used for speech recognition.


Created by

The application calls the CoCreateInstance function with the CLSID_MMAudioSource class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-source object.


Interacts with

When an audio-source object is first created, it uses the multimedia wave mapper as the source device, but a valid wave format has not been specified. After an application creates the audio-source object, it should call the IAudioMultiMediaDevice::DeviceNumSet member function with the device identifier of the audio device to use for input. The engine calls the IAudio::WaveFormatSet member function to set the wave format for the device. The engine should also call the IAudio::PassNotify member function with the address of a notification interface — that is, IAudioSourceNotifySink — so that the audio-source object can send status information to the engine.


Interfaces

The multimedia audio-source object supports the following interfaces.

Interface | Description
IAudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioMultiMediaDevice | Allows an audio-destination or audio-source object to access features specific to multimedia devices.
IAudioSource | Gets information and data from an audio-source object.
IAudioSourceNotifySink | Notifies a speech-recognition engine of changes to the internal buffer of an audio-source object.


Instrumented Audio-Source Object

(New for 3.0)


Purpose

The instrumented audio-source object allows an application to record the audio passing through to the recognizer, or replace the audio going to the recognizer with audio from a wave file. The object can be used for speech recognition.


Created by

The application calls the CoCreateInstance function with the CLSID_InstAudioSource class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-source object.


Interacts with

When an audio-source object is first created, it must be passed an IUnknown to a "real" audio source. By default, the instrumented audio source passes through the audio and notifications from the "real" audio source through to the object that's using the instrumented audio source.

An application can make calls to IAudioSourceInstrumented to have the audio coming from the audio source written to disk, or to have the audio from the audio source replaced by data from disk.


Interfaces

The multimedia audio-source object supports the following interfaces.

Interface | Description
IAudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioSource | Gets information and data from an audio-source object.
IAudioSourceInstrumented | Controls the audio data instrumentation.
IAudioSourceNotifySink | Notifies a speech-recognition engine of changes to the internal buffer of an audio-source object.


File Audio-Source Object

(New for 3.0)


Purpose

The file audio-source object allows an application to send the audio from one or more wave files to a speech recognizer.


Created by

The application calls the CoCreateInstance function with the CLSID_AudioSourceFile class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-source object.


Interacts with

When an file audio-source object is first created, it has an empty file queue. After an application creates the audio-source object, it should call the IAudioFile::Add member function for each of the files to be recognized. The engine calls the IAudio::WaveFormatSet member function to set the wave format for the object. If the wave format does not match the engine's requested format then an error will be returned.


Interfaces

The file audio-source object supports the following interfaces.

Interface | Description
Iaudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioSource | Gets information and data from an audio-source object.
IAudioFile | Controls which files are to be read from or written to.
IAudioFileNotifySink | Notifies the application when a file has been completed, and other status information.
IAudioSourceNotifySink | Notifies a speech-recognition engine of changes to the internal buffer of an audio-source object.


File Audio-Destination Object

(New for 3.0)


Purpose

The file audio-destination object allows an application to write the audio from a text-to-speech engine to a file.


Created by

The application calls the CoCreateInstance function with the CLSID_AudioDestFile class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-destination object.


Interacts with

When an file audio-destination object is first created an application should call the IAudioFile::Set member function to specify the file output.


Interfaces

The file audio-source object supports the following interfaces.

Interface | Description
IAudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioDest | Sends information and data to an audio-destination object.
IAudioFile | Controls which files are to be read from or written to.
IAudioFileNotifySink | Notifies the application when a file has been completed, and other status information.
IAudioDestNotifySink | Notifies a text-to-speech engine of changes to the internal buffer of an audio-destination object.


DirectSound Audio-Destination Object

(New for 3.0)


Purpose

The DirectSound audio destination object sends the audio through a DirectSound object, rather than a multimedia wave-device or wave file.


Created by

The application calls the CoCreateInstance function with the CLSID_AudioDestDirect class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-destination object.


Interacts with

When an DirectSound audio-destination object is first created an application should call the IAudioDirect::Init member function to pass in the DirectSound object. The audio object will create a DirectSound buffer to use for the text-to-speech voice.


Interfaces

The file audio-source object supports the following interfaces.

Interface | Description
IAudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioDest | Sends information and data to an audio-destination object.
IAudioDirect | Used by the application to specify the DirectSound object.


DirectSound Audio-Source Object

(New for 3.0)


Purpose

The DirectSound audio source object records the audio through a DirectSound object, rather than a multimedia wave-device or wave file.


Created by

The application calls the CoCreateInstance function with the CLSID_AudioSourceDirect class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-source object.


Interacts with

When an DirectSound audio-source object is first created an application should call the IAudioDirect::Init member function to pass in the DirectSound Capture object. The audio object will create a DirectSound Capture buffer to use for the text-to-speech voice.


Interfaces

The file audio-source object supports the following interfaces.

Interface | Description
IAudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioSource | Gets information and data to an audio-source object.
IAudioDirect | Used by the application to specify the DirectSound object.


Suspending Audio-Source Object

(New for 3.0)


Purpose

The suspending audio-source object allows an application to temporarily suspect the audio going into speech recognition. While suspended, the wave-in/out device will be freed, so an application can play a wave file or do other recording.


Created by

The application calls the CoCreateInstance function with the CLSID_SuspAudioSource class identifier. CoCreateInstance returns the address of the IUnknown interface on the newly created audio-source object.


Interacts with

When a suspended audio-source object is first created, it must be passed an IUnknown to a "real" audio source. The suspended audio source should then be passed to the speech recognition engine.

An application can make calls to IAudioSourceSusp to pause or resume the audio.


Interfaces

The multimedia audio-source object supports the following interfaces.

Interface | Description
IAudio | Allows an audio-destination or audio-source object to manage its internal buffer and the control attributes of the audio device it represents.
IAudioSource | Gets information and data from an audio-source object.
IAudioSourceSusp | Pause/resume the audio.
IAudioSourceNotifySink | Notifies a speech-recognition engine of changes to the internal buffer of an audio-source object.


Audio Source/Destination Interfaces and Member Functions


IAudio

This interface is supported by the audio-source object.

Member function | Description
Claim | Acquires the multimedia wave device for use by the audio object.
Flush | Clears the object's internal buffer and resets the audio device but does not stop the playing or recording of audio data.
LevelGet | Gets the playback or recording level of the audio device.
LevelSet | Sets the playback or recording level of the audio device.
PassNotify | Passes a pointer to a notification interface to the audio object so the object can notify the application when the amount of available data changes or other events occur.
PosnGet | Gets the exact byte in the audio stream that is currently being played or recorded.
Start | Informs an audio object to start playing or recording data.
Stop | Informs an audio object to stop playing or recording data.
ToFileTime | Converts a QWORD representing the time in bytes to a Win32 FILETIME structure.
TotalGet | Gets the total number of bytes played or recorded by an audio device.
UnClaim | Releases the multimedia wave device.
WaveFormatGet | Gets the wave format that is expected and used by the audio device.
WaveFormatSet | Sets the wave format for the object for subsequent audio, if this is allowed by the audio device.


IAudioDest

This interface is supported by the audio-destination object.

Member function | Description
BookMark | Attaches a bookmark to the most recent data in the audio-destination object's internal buffer.
DataSet | Writes digital-audio data to the end of the object's internal buffer.
FreeSpace | Returns the number of bytes that are free in the object's internal buffer.


IAudioDirect

This interface is supported by the DirectSound audio-destination object. Applications use it to pass in a pointer to the DirectSound object.

Member function | Description
Init | The application passes in the DirectSound objects with this.


IAudioDestNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
AudioStart | Notifies the application that data has begun flowing into the audio stream.
AudioStop | Notifies the application that no more data will be played on the audio-destination device for an indefinite period of time.
BookMark | Notifies the application that the audio-destination device has encountered a bookmark in the audio stream.
FreeSpace | Notifies the application that the amount of data in the audio-destination object's internal buffer has changed.


IAudioFile

(New for 3.0)

This interface is supported by the file audio-source and file audio-destination object.

Member function | Description
Add | Adds a file to the queue of files.
Flush | Clears the queue of files.
RealTimeGet | Returns information about how quickly audio is allowed to pass through the object.
RealTimeSet | Specified how quickly audio is allowed to pass through the object.
Register | Passes a pointer to a notification interface to the audio object so the object can notify the application when files are finished, or the current position has changed.
Set | Clears the file queue and sets a file.


IAudioFileNotifySink

(New for 3.0)

This interface is supported by the file audio-source and file audio-destination object. Applications provide the notification sink so they are alerted when the current file being listened to has changed or of the current position within the file.

Member function | Description
FileBegin | Sent when a file is opened and processing begins on it.
FileEnd | Sent when a file is closed.
Posn | Sent so the application knows how much data has been read/written.
QueueEmpty | Sent when the file queue is empty.


IAudioInstrumented

(New for 3.0)

This interface is supported by the instrumented audio-source. It allows the application to record audio going into the speech recognition engine, or replace the audio with that from a wave file.

Member function | Description
AudioSource | Specify the audio source object to use.
RegistryGet | Returns the registry key being monitored.
RegistrySet | Sets the registry key to be monitored.
StateGet | Returns the current state of the instrumented audio object, whether it's paused, acting as a pass through, recording, or playing audio.
SateSet | Sets the current state of the instrumented audio object, whether it's paused, acting as a pass through, recording, or playing audio.
WaveFileReadGet | Returns the wave file that is being read from.
WaveFileReadSet | Sets the file to be read from.
WaveFileWriteGet | Returns the wave file to be written to.
WaveFileWriteSet | Sets the wave to write to.


IAudioMultiMediaDevice

This interface is supported by the audio-source object.

Member function | Description
CustomMessage | Sends a custom message to a multimedia driver.
DeviceNumGet | Returns the device ID of the multimedia device that is represented by the audio-source or audio-destination object.
DeviceNumSet | Sets a new device ID for the multimedia device that is the source of the audio stream.


IAudioSource

This interface is supported by the audio-source object.

Member function | Description
DataAvailable | Returns the number of bytes of data in the internal buffer of an audio-source object.
DataGet | Reads digital-audio data from the internal buffer of an audio-source object, removing data from the buffer as it is read.


IAudioSourceInstrumented

This interface is supported by the instrumented audio-source object.

Member function | Description
AudioSource | Pass the IUnknown for the audio source that inputs to the instrumented version.
RegistryGet | Returns the registry key that can be used to externally control the instrumentation.
RegistrySet | Changes the registry key that can be used to externally control the instrumentation.
StateGet | Returns the current that the instrumented version is in.
StateSet | Changes the instrumentation mode.
WaveFileReadGet | Gets the wave file that is to be read from.
WaveFileReadSet | Sets the wave file that is to be read from.
WaveFileWriteGet | Gets the wave file that is to be written to.
WaveFileWriteSet | Sets the wave file that is to be written to.


IAudioSourceNotifySink

This interface is a notification sink that is supported by an application.

Member function | Description
AudioStart | Notifies the application that data has begun flowing into the audio stream.
AudioStop | Notifies the application that no more data will be coming into the audio stream for an indefinite period of time.
DataAvailable | Notifies the application that the amount of data in the audio-source object's internal buffer had changed.
Overflow | Notifies the application that the internal buffer is full and incoming data has been lost.


IAudioSourceSusp

This interface is supported by the suspending audio-source object.

Member function | Description
AudioSource | Pass the IUnknown for the audio source that inputs to the suspending audio source.
Pause | Pauses the audio.
Resume | Resumes the audio


Interfaces and Member Functions

In addition to the interfaces listed in this section, each object supports IUnknown, which is a common interface supported by all OLE component objects. For information about the IUnknown interface, see the section, "Common Elements of the Speech Application Programming Interface."


IAudio Interface

The IAudio interface allows an audio-source or audio-destination object to manage its internal buffer and to control certain attributes of the audio device it represents, such as the playback or recording level of the device or its wave format.

The IAudio interface is supported by the multimedia audio-destination object and the multimedia audio-source object. It has these member functions:

Claim | Stop
Flush | ToFileTime
LevelGet | TotalGet
LevelSet | UnClaim
PassNotify | WaveFormatGet
PosnGet | WaveFormatSet
Start |


IAudio::Claim

HRESULT Claim(void);

Acquires the multimedia wave device for use by the audio object.

· Returns NOERROR if successful, or one of these values:

E_FAIL
E_OUTOFMEMORY
AUDERR_ALREADYCLAIMED
AUDERR_BADDEVICEID
AUDERR_HANDLEBUSY
AUDERR_INVALIDHANDLE
AUDERR_NEEDWAVEFORMAT
AUDERR_NODRIVER
AUDERR_WAVEDEVICEBUSY
AUDERR_WAVEFORMATNOTSUPPORTED
AUDERR_WAVENOTENABLED

Claim opens the multimedia wave device and generates an IAudioSourceNotifySink::AudioStart notification. The device remains open until the IAudio::UnClaim member function is called.


IAudio::Flush

HRESULT Flush(void);

Clears the object's internal buffer and resets the audio device, but does not stop the playing or recording of audio data.

· Returns NOERROR if successful, or one of these values:

AUDERR_INVALIDHANDLE
AUDERR_NEEDWAVEFORMAT
AUDERR_WAVEDEVICEBUSY

Flush calls the waveOutReset multimedia function for a wave-out device or the waveInReset multimedia function for a wave-in device.

For an audio-destination object, calling Flush automatically stops the audio output because it empties the internal buffer of audio data to be played. However, if audio was playing before Flush was called (and unless the IAudio::Stop member function is called), the audio begins playing again as soon as the internal buffer contains more audio data.

For an audio-source object, calling Flush automatically stops recording audio data. As for an audio-destination object, if recording was in process before Flush was called (and unless Stop is called), recording will resume as soon as more audio data is received.


IAudio::LevelGet

HRESULT LevelGet(DWORD *pdwLevel);

Retrieves the playback or recording level of the audio device.

· Returns NOERROR if successful, or one of these error values:

E_FAIL
E_INVALIDARG
AUDERR_BADDEVICEID
AUDERR_INVALIDHANDLE
AUDERR_NEEDWAVEFORMAT
AUDERR_NODRIVER
AUDERR_NOTSUPPORTED
AUDERR_WAVEFORMATNOTSUPPORTED

pdwLevel

[out] Address of a variable that receives the playback level for an audio destination or the recording level for an audio source. The level is a linear range from 0x0000 for absolute silence to 0xFFFF for the highest playback or recording level. The level for the left channel is in the low word, and the level for the right channel is in the high word of this parameter.

If a device does not support both left and right volume control, the low-order word of pdwLevel contains the mono volume level.

If the device does not provide a level, the function returns an error.


IAudio::LevelSet

HRESULT LevelSet(DWORD dwLevel);

Sets the playback or recording level of the audio device.

· Returns NOERROR if successful, or one of these error values:

E_FAIL
E_INVALIDARG
AUDERR_BADDEVICEID
AUDERR_INVALIDHANDLE
AUDERR_NEEDWAVEFORMAT
AUDERR_NODRIVER
AUDERR_NOTSUPPORTED
AUDERR_WAVEFORMATNOTSUPPORTED

dwLevel

[in] New playback or recording level. The level is a linear range from 0x0000 for absolute silence to 0xFFFF for the highest playback or recording level. The level for the left channel is in the low word, and the level for the right channel is in the high word of this parameter.

If a device does not support both left and right volume control, the low-order word of dwLevel specifies the volume level and the high-order word is ignored.

If the device does not provide a level, the function returns an error.

The change in playback or recording level occurs as soon as possible, although it may take several milliseconds to change because of hardware restrictions. Changing the level does not affect digital-audio data already played or recorded.


IAudio::PassNotify

HRESULT PassNotify(PVOID pNotifyInterface, IID IIDNotifyInterface);

Passes the address of a notification interface to the audio object so that the object can notify the engine when the amount of available data changes or other events occur.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_INVALIDNOTIFYSINK

pNotifyInterface

[in] Address of the notification interface through which the audio object notifies the engine about audio information.

Because passing the address to the audio object does not transfer ownership of the notification interface, the audio object must call the interface's AddRef member function before returning from the call to PassNotify. The audio object must also call the IAudio interface's Release member function when it closes. The calling application must release any reference counts it holds on the notification interface after calling PassNotify, unless it needs the notification object to be valid when the audio object releases it.

IIDNotifyInterface

[in] Interface identifier of the interface used for notification. The identifier must be IID_AudioDestNotifySink for an audio-destination object or IID_AudioSourceNotifySink for an audio-source object.

Initially, an audio object has NULL for a notification sink.

An engine can use the notification interface to get information about the status of audio data, such as whether or not there is space available in the object's internal buffer.

If PassNotify is called more than once, Release is called on the previous notification interface pointed to by pNotifyInterface before AddRef is called on the new interface pointed to by pNotifyInterface.


IAudio::PosnGet

HRESULT PosnGet(QWORD *pqwTimeStamp);

Retrieves the exact byte in the audio stream that is currently being played or recorded (or as exact as possible given the audio device).

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_NEEDWAVEFORMAT

pqwTimeStamp

Address of the byte in the audio stream that is currently being played or recorded. The "time stamp" is actually an index into the number of bytes played/recorded by the audio stream. The time-stamp does not necessarily start at 0. Time stamps only ever increase in value over the life of the audio object, and never decrease in value. Every byte of data that passes through the audio object increases the time stamp. When the audio is UnClaimed the time stamp will not reset to 0. It might increase if the audio object is being time-shared.

The PosnGet function works only for real-time audio destinations, such as audio speakers, or real-time audio sources, such as microphones. It does not work for non-real-time destinations or sources, such as files.

An application can use PosnGet to synchronize actions with events. For example, if an action should occur at a certain interval after a particular byte is played, the application can call PosnGet to get the time that byte was played, add the interval to the time, and synchronize the action with the result.


IAudio::Start

HRESULT Start(void);

Informs an audio object to start playing or recording data.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_ALREADYSTARTED
AUDERR_NEEDWAVEFORMAT
AUDERR_NOTCLAIMED

An audio-destination object starts playing data from its internal buffer, and an audio-source object starts recording digital-audio data.

The internal buffer has a finite size (a minimum of 2 seconds of audio). If the internal buffer of an audio-destination object runs out of data, the user will hear gaps in the audio being played. If the buffer of an audio-source object fills up, data is lost.

For an audio-source object, if the IAudio::Claim member function has not been called, Start makes an implicit call to Claim. If the device is not available, Start continues to try to initiate recording until the IAudio::Stop member function is called. This implementation reduces the work required by the engine in obtaining a device which may not be available immediately. For additional error values that Start may return if it makes an implicit call to Claim, see the description of Claim.


IAudio::Stop

HRESULT Stop(void);

Informs an audio object to stop playing or recording data.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_NEEDWAVEFORMAT

An audio-destination object stops playing data from its internal buffer, essentially pausing the output stream. An audio-source object stops recording data, but does not clear any data that already exists in its internal buffer.

To restart playing or recording, an application can call the IAudio::Start member function. Any data left in the buffer when Stop was called will still be there, and the object will resume playing or recording the data.

If Start made an implicit call to the IAudio::Claim member function for the audio-source object, Stop makes an implicit call to the IAudio::UnClaim member function. For additional error messages that Stop may return if it makes an implicit call to UnClaim, see the description of UnClaim.


IAudio::ToFileTime

HRESULT ToFileTime(PQWORD pqWord, FILETIME *pFT);

Converts a QWORD value representing the time, in bytes, to a Win32 FILETIME structure.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_NEEDWAVEFORMAT

pqWord

[in] Address of a variable that contains the time, in bytes, returned by an engine or by the IAudio::TotalGet or IAudio::PosnGet member function.

pFT

[out] Address of a variable that receives the Win32 FILETIME value of

pqWord.

An application can use ToFileTime to correlate real-time events with a time value returned by an engine. The value returned by the engine is an estimate that should be very accurate within a few seconds of the present, but becomes less accurate farther away from the present, due to time variations in the audio card's clock.


IAudio::TotalGet

HRESULT TotalGet(PQWORD pqWord);

Retrieves the total number of bytes played or recorded by an audio device.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_NEEDWAVEFORMAT

pqWord

Address of a QWORD value that receives the total number of bytes played or recorded by the audio device. The "time stamp" is actually an index into the number of bytes played/recorded by the audio stream. The time-stamp does not necessarily start at 0. Time stamps only ever increase in value over the life of the audio object, and never decrease in value. Every byte of data that passes through the audio object increases the time stamp. When the audio is UnClaimed the time stamp will not reset to 0. It might increase if the audio object is being time-shared.

For an audio-destination object, the value returned by TotalGet includes unplayed bytes still in the internal buffer of an audio-destination object. An application can use this value to track notifications by calling TotalGet and storing the value returned before it passes in audio data to be played. This gives the application a baseline from which to calculate when the audio will be played.

For an audio-source object, the value returned by TotalGet includes the total number of bytes recorded from the audio source, but does not include data in the internal buffer of the audio-source object. This gives the application a way of determining at what time an audio recording occurred.

An application should not assume that the value returned by TotalGet changes only with the IAudioDest::DataSet or IAudioSource::DataGet member function. If the audio device is shared, the value returned by TotalGet includes all of the audio that flows through the device, including audio played or recorded by other applications.


IAudio::UnClaim

HRESULT UnClaim(void);

Releases the multimedia wave device.

· Returns NOERROR if successful, or one of these error values:

E_FAIL
AUDERR_NEEDWAVEFORMAT
AUDERR_NOTCLAIMED

UnClaim generates an AudioStop notification when the multimedia wave device is closed.

In the case of the multimedia audio-destination object, UnClaim releases the device only after all audio buffers have been sent. For this reason, there may be a delay between when UnClaim is called and when the device is actually closed. An application should call UnClaim only after making all calls to the IAudioDest::DataSet member function.


IAudio::WaveFormatGet

HRESULT WaveFormatGet(PSDATA pdWFEX);

Gets the wave format that is expected and used by the audio device.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_NEEDWAVEFORMAT

pdWFEX

[out] Address of an SDATA structure that receives a pointer to a WAVEFORMATEX structure containing wave format information for the audio device.

If the audio-source or destination is attached to file, the WAVEFORMATEX structure is loaded with the file's wave format.

It is up to the calling application to free the memory allocated by the function using the CoTaskMemAlloc and CoTaskMemFree functions.


IAudio::WaveFormatSet

HRESULT WaveFormatSet(SDATA dWFEX);

Sets the wave format for the object for subsequent audio, if this is allowed by the audio device.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_BADDEVICEID
AUDERR_NODRIVER
AUDERR_NOTSUPPORTED
AUDERR_WAVEDEVICEBUSY
AUDERR_WAVEFORMATNOTSUPPORTED

dWFEX

[in] SDATA structure that contains the address of the new wave format for the audio device. The pData member specifies the address of a WAVEFORMATEX structure that contains the wave format information, and the dwSize member indicates the size of the information.

WaveFormatSet specifies the kind of audio data that will be received by the audio source or sent to the audio destination. Most applications use pulse code modulation (PCM), but if the specified format is compressed or encoded audio, the data for the IAudioDest::DataSet and IAudioSource::DataGet member functions will be compressed.

If audio is currently being played, WaveFormatSet has no effect.

Some audio devices allow an application to set their wave format once, after which the device becomes locked. If an audio device does not allow an application to set its wave format, the function returns AUDERR_WAVEFORMATNOTSUPPORTED.


IAudioDest Interface

The IAudioDest interface allows an engine to send information and data to an audio destination, such as a file or an audio wave-out device. When the engine gets the address of IAudioDest for an audio-destination object, that object already represents a particular audio destination. If the audio-destination object represents a multimedia wave-out device, IAudioDest is already open to a particular WAVEFORMATEX structure and device.

The IAudioDest interface is supported by the multimedia audio-destination object. It has these member functions:

BookMark

DataSet

FreeSpace


IAudioDest::BookMark

HRESULT BookMark(DWORD dwMarkID);

Attaches a bookmark to the most recent data in the audio-destination object's internal buffer.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_NEEDWAVEFORMAT

dwMarkID

[in] Unique bookmark identifier.

When the audio-destination object plays the data and encounters the bookmark, it calls the IAudioDestNotifySink::BookMark member function to notify the engine. The engine passes the notification to the application, allowing it to synchronize real-time events with the audio data.

If the audio destination is a file (which is never heard), BookMark returns an error.

If the IAudio::Flush member function is called or the audio-destination object is released, the object sends notifications for all bookmarks that remain in the queue with a flag indicating that they are being flushed. The engine passes the notifications to the application, allowing it to free any memory allocated for a bookmark.


IAudioDest::DataSet

HRESULT DataSet(PVOID pBuffer, DWORD dwSize);

Writes digital-audio data to the end of the object's internal buffer.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_NEEDWAVEFORMAT
AUDERR_NOTCLAIMED
AUDERR_NOTENOUGHDATA

pBuffer

[in] Address of a buffer that contains the data to be copied into the object's internal buffer.

dwSize

[in] Number of bytes of data to copy.

An audio-destination object contains an internal first-in/first-out (FIFO) buffer of digital-audio data. As audio is sent to the audio destination, it is taken from the beginning of the buffer. Each time DataSet is called, the new information is added to the end of the buffer. If the amount of data requested is too large to fit the buffer, DataSet returns an error.

An engine must call the IAudio::Claim member function before calling DataSet. When all buffers have been sent to the audio destination, the engine must call the IAudio::UnClaim member function.


IAudioDest::FreeSpace

HRESULT FreeSpace(DWORD *pdwBytes, BOOL *pfEOF);

Returns the number of bytes that are free in the object's internal buffer.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_NEEDWAVEFORMAT

pdwBytes

[out] Address of a variable that receives the number of bytes of space that are available in the audio-destination object's internal buffer.

pfEOF

Address of a variable that receives TRUE if an end-of-file character has been reached and no more data can be sent to the audio destination or FALSE if more data exists. If the audio destination is a file, this parameter receives TRUE if there is not enough disk space to finish writing the file. If the audio destination is a wave-out device, this parameter always receives FALSE.


IAudioDestNotifySink Interface

The IAudioDestNotifySink interface allows an audio-destination object to notify an engine of changes in the object's internal buffer. Typically, an engine passes the notifications to an application by calling a notification interface provided by the application.

The IAudioDestNotifySink interface has these member functions:

AudioStart

AudioStop

BookMark

FreeSpace


IAudioDestNotifySink::AudioStart

HRESULT AudioStart(void);

Notifies an engine that the audio destination device has been opened.

· Return value is ignored.

AudioStart is called as a result of a call to the IAudio::Claim member function.


IAudioDestNotifySink::AudioStop

HRESULT AudioStop(WORD wReason);

Notifies the engine that no more data will be played on the audio-destination device for an indefinite period of time.

· Return value is ignored.

wReason

[in] Reason that the audio has stopped. This parameter can be one of these values:

Value | Meaning
IANSRSN_EOF | An end-of-file character has been reached.
IANSRSN_INACTIVE | The device has become inactive for an unknown reason.
IANSRSN_NODATA | The engine called the IAudio::UnClaim member function. This notification allows the engine to employ the notification sink to monitor the object's state rather than having to maintain this information itself.
IANSRSN_PRIORITY | Resources were temporarily diverted to a higher-priority task.


IAudioDestNotifySink::BookMark

HRESULT BookMark(DWORD dwMarkID, BOOL fFlush);

Notifies the engine that the audio-destination device has encountered a bookmark in the audio stream.

· Return value is ignored.

dwMarkID

[in] Unique bookmark identifier.

fFlush

[in] TRUE if the bookmark notification has been sent because the engine called the IAudio::Flush member function to clear the object's internal buffer or FALSE if the object has encountered the bookmark while playing back audio data.

Bookmarks can be set by calling the IAudioDest::BookMark member function.


IAudioDestNotifySink::FreeSpace

HRESULT FreeSpace(DWORD dwBytes, BOOL fEOF);

Notifies the engine that the amount of data in the audio-destination object's internal buffer has changed.

· Return value is ignored.

dwBytes

[in] Number of bytes that are currently free in the object's internal buffer.

fEOF

[in] TRUE if an end-of-file character has been reached and no more data can be sent to the audio destination or FALSE if more data exists. If the audio destination is a wave-out device, this parameter is always FALSE.

An engine might respond to the FreeSpace notification by putting more data into the buffer.

This notification is not sent when the amount of data changes as a result of the engine's own actions, such as calling the IAudio::Flush or IAudioDest::DataSet member function.

For best performance, the audio-destination object should not send this notification more than 16 times per second.


IAudioDirect Interface

(New for 3.0)

This interface is supported by the DirectSound audio-destination object. Applications use it to pass in a pointer to the DirectSound object.

Member function | Description
Init | The application passes in the DirectSound objects with this.


IAudioDirect::Init

HRESULT Init(PVOID pInterface, IID iid);

An application should call IAudioDirect::Init as soon as it creates the DirectSound audio source/destination object.

pInterface

[in] Pointer to the interface. The audio object will release this interface when it is finished with it, so the application may wish to AddRef the interface before passing it to Init. The audio object will create a direct sound buffer using the interface.

iid

[in] Interface ID. Use IID_IDirectSound for the audio destination, and IID_IDirectSoundCapture for the audio source.

Before the DirectSound Audio Destination object can play any sound buffers, the application must specify a cooperative level for a DirectSound object by using the IDirectSound::SetCooperativeLevel method.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG


IAudioFile Interface

(New for 3.0)

The IAudioFile interface allows an application to set what files a speech recognition engine will listen to, or where the text-to-speech digital audio will be written.

This interface is supported by the file audio-source and file audio-destination object.

Member function | Description
Add | Adds a file to the queue of files.
Flush | Clears the queue of files.
RealTimeGet | Returns information about how quickly audio is allowed to pass through the object.
RealTimeSet | Specified how quickly audio is allowed to pass through the object.
Register | Passes a pointer to a notification interface to the audio object so the object can notify the application when files are finished, or the current position has changed.
Set | Clears the file queue and sets a file.


IAudioFile::Add

HRESULT Add(LPCWSTR pszFile, DWORD dwID);

Adds a file onto the end of the audio queue. When the file audio-source gets to reading this file, it will send a notification to the application with the dwID. IAudioFile::Add is not supported by the file audio-destination.

pszFile

[in] Unicode file name to read from. This must be a valid .WAV file with the same format as the speech recognition engine uses.

dwID

[in] Identifier that is passed to the application when the file is read, and when all of the data is finished.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_NOTSUPPORTED


IAudioFile::Flush

HRESULT Flush(void);

Clears the queue of all files, effectively pausing speech recognition or text-to-speech.

· Returns NOERROR if successful.


IAudioFile::RealTimeGet

HRESULT RealTimeGet(WORD *pwRealTime);

Returns the allowed data throughput rate for the audio source or destination.

pwRealTime

[out] Filled with the current real-time value. A value of 0x100 means that data is read/written from/to the wave file at real time, so it takes 1 second to process 1 second of audio recording numbers. Higher numbers indicate that processing is faster than real time. 0x200 causes 1 second of audio to be processed in 0.5 second of time. Of course, this only works if the speech recognition or text-to-speech can perform at such accelerated rates given the speech of the CPU.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG


IAudioFile::RealTimeSet

HRESULT RealTimeSet(WORD wRealTime);

Sets the allowed data throughput rate for the audio source or destination.

wRealTime

[in] Sets the current real-time value. A value of 0x100 means that data is read/written from/to the wave file at real time, so it takes 1 second to process 1 second of audio recording numbers. Higher numbers indicate that processing is faster than real time. 0x200 causes 1 second of audio to be processed in 0.5 second of time. Of course, this only works if the speech recognition or text-to-speech can perform at such accelerated rates given the speech of the CPU.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG


IAudioFile::Register

HRESULT Register(PIAUDIOFILENOTIFYSINK pNotifyInterface);

Passes the address of a file notification interface to the audio object so that the object can notify the application when files are opened/closed, and current file position.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pNotifyInterface

[in] Address of the notification interface through which the audio object notifies the engine about audio information.

Because passing the address to the audio object does not transfer ownership of the notification interface, the audio object must call the interface's AddRef member function before returning from the call to Register. The audio object must also call the IAudio interface's Release member function when it closes. The calling application must release any reference counts it holds on the notification interface after calling Register, unless it needs the notification object to be valid when the audio object releases it.


IAudioFile::Set

HRESULT Set(LPCWSTR pszFile, DWORD dwID);

Flushes any files in the queue, and sets the current file. This works for the file audio-source and the file audio-destination.

pszFile

[in] Unicode file name to read from. For the file audio-source, this must be a valid .WAV file with the same format as the speech recognition engine uses.

dwID

[in] Identifier that is passed to the application when the file is read/written, and when all of the data is finished.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG


IAudioFileNotifySink Interface

(New for 3.0)

This interface is supported by the file audio-source and file audio-destination object. Applications provide the notification sink so they are alerted when the current file being listened to has changed or of the current position within the file.

Member function | Description
FileBegin | Sent when a file is opened and processing begins on it.
FileEnd | Sent when a file is closed.
Posn | Sent so the application knows how much data has been read/written.
QueueEmpty | Sent when the file queue is empty.


IAudioFileNotifySink::FileBegin

HRESULT FileBegin(DWORD dwID);

Sent when the audio from the file begins being processed.

dwID

[in] Identifier that is passed to the object on an IAudioFile::Set or IAudioFile::Add call.


IAudioFileNotifySink::FileEnd

HRESULT FileEnd(DWORD dwID);

Sent when the audio from the file is finished being processed.

dwID

[in] Identifier that is passed to the object on an IAudioFile::Set or IAudioFile::Add call.


IAudioFileNotifySink::Posn

HRESULT Posn(QWORD qwProcessed, QWORD qwLeft);

Sent about sixteen times a second so the application knows how much data has been processed.

qwProcessed

[in] Number of bytes that have already been read from the queue, or written to output wave files.

qwLeft

[in] Number of bytes of audio that are left in the queue to be read. The number is only valid for file audio-source objects. This will always be 0 for file audio-destination objects.


IAudioFileNotifySink::QueueEmpty

HRESULT QueueEmpty(void);

Sent when all of the files in the queue have been processed. At this point the speech recognition engine (or text-to-speech engine) is being starved for data.


IAudioMultiMediaDevice Interface

The IAudioMultiMediaDevice interface allows audio-source and audio-destination objects to access features specific to multimedia devices.

The IAudioMultiMediaDevice interface is supported by the multimedia audio-destination and audio-source objects. It has these member functions:

CustomMessage

DeviceNumGet

DeviceNumSet


IAudioMultiMediaDevice::CustomMessage

HRESULT CustomMessage(UINT uMsg, SDATA dData);

Sends a custom message to a multimedia device driver.

· Returns NOERROR if successful, or one of these error values:

E_FAIL
E_INVALIDARG
E_OUTOFMEMORY
AUDERR_NODRIVER

uMsg

[in] Message number to send to the multimedia driver.

dData

[in, out] Address of an SDATA structure that contains information to be sent with the custom message. This structure also receives data returned by the custom message. The data should not contain pointers because it may be marshaled.

An engine or application should send custom messages only after it has verified that the driver supports custom messages.

CustomMessage essentially calls the multimedia function WaveXXXMessage(hWave, uMsg, dData-dwSize, dData-pData). If the wave device resides in another process, the data buffer will be marshaled. It is not possible to reference memory directly in the data because the wave device might be opened by a different process.


IAudioMultiMediaDevice::DeviceNumGet

HRESULT DeviceNumGet(DWORD *pdwDeviceID);

Returns the device identifier of the multimedia device that is represented by the audio-source or audio-destination object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
AUDERR_BADDEVICEID
AUDERR_SYNCNOTALLOWED
AUDERR_WAVEDEVICEBUSY
AUDERR_WAVEDEVNOTSUPPORTED

pdwDeviceID

[out] Address of a variable that receives the device identifier, which ranges from zero to the number of devices that are connected to the system. It can also be WAVE_MAPPER if a specific device is not set.

The device identifier can be used with functions from the Multimedia application programming interface (API) to determine the manufacturer of the device driver and whether or not a digital signal processor (DSP) connection exists.


IAudioMultiMediaDevice::DeviceNumSet

HRESULT DeviceNumSet(DWORD dwDeviceID);

Sets a new device identifier for the multimedia device that is the source of the audio stream.

· Returns NOERROR if successful, or one of these error values:

E_FAIL
AUDERR_BADDEVICEID
AUDERR_SYNCNOTALLOWED
AUDERR_WAVEDEVICEBUSY
AUDERR_WAVEDEVNOTSUPPORTED

dwDeviceID

[in] New device identifier.


IAudioSource Interface

The IAudioSource interface gets information and data from a digital-audio source, such as a file or an audio device. When the engine gets the address of the IAudioSource interface for an audio-source object, that object already represents a particular audio source. If the audio-source object represents a multimedia wave-in device, the IAudioSource interface is already open to a particular WAVEFORMATEX structure and device.

The IAudioSource interface is supported by the multimedia audio-source object. It has these member functions:

DataAvailable

DataGet


IAudioSource::DataAvailable

HRESULT DataAvailable(DWORD *pdwBytes, BOOL *pfEOF);

Retrieves the number of bytes of data in the internal buffer of an audio-source object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_NEEDWAVEFORMAT

pdwBytes

[out] Address of a variable that receives the number of bytes of data in the internal buffer.

pfEOF

[out] Address of a variable that receives TRUE if an end-of-file character has been reached and no more data can be received from the audio source or FALSE if some data might remain. An audio device always returns FALSE.


IAudioSource::DataGet

HRESULT DataGet(PVOID pBuffer, DWORD dwGetSize, DWORD *pdwCopied);

Reads digital-audio data from the internal buffer of an audio-source object, removing data from the buffer as it is read.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_NEEDWAVEFORMAT

pBuffer

[out] Address of a buffer that receives data from the internal buffer.

dwGetSize

[in] Number of bytes of data to write to the buffer.

pdwCopied

[out] Address of a variable that receives the number of bytes written to the buffer.

The audio-source object contains an internal FIFO buffer of digital-audio data. As audio data is received from the audio source, it is added to the end of the buffer. DataGet takes data from the beginning of the buffer.

If the buffer contains fewer than the requested number of bytes, DataGet returns the entire contents of the buffer.

An engine can use the data retrieved with DataGet as input for training dialog boxes or any other dialog boxes that use digital-audio data.


IAudioSourceIntrumented Interface

(New for 3.0)

The IAudioSourceInstrumented interface controls the instrumentation provided by the instrumented audio source object.

Member function | Description
AudioSource | Pass the IUnknown for the audio source that inputs to the instrumented version.
RegistryGet | Returns the registry key that can be used to externally control the instrumentation.
RegistrySet | Changes the registry key that can be used to externally control the instrumentation.
StateGet | Returns the current that the instrumented version is in.
StateSet | Changes the instrumentation mode.
WaveFileReadGet | Gets the wave file that is to be read from.
WaveFileReadSet | Sets the wave file that is to be read from.
WaveFileWriteGet | Gets the wave file that is to be written to.
WaveFileWriteSet | Sets the wave file that is to be written to.


IAudioSourceInstumented::AudioSource

HRESULT AudioSource(PIUNKNOWN pAudioSource);

Passes the address of an audio source object to be instrumented.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_INVALIDNOTIFYSINK

pAudioSource

[in] Address of the IUnknown interface of the audio source object.

This must be called before any other function is in the IAudioInstrumented is called. It can only be called once. The instrumented audio source will rout all calls to IAudio, IAudioSource, and IAudioSourceNotifySink to/from pIAudioSource's versions of the same interfaces. When the instrumented audio object is released it will call release an interfaces to the pAudioSource object.


IAudioSourceInstumented::RegistryGet

HRESULT RegistryGet(WCHAR *pszKey, DWORD dwKeySize,

DWORD *pdwNeeded);

DOES NOT WORK ON WINDOWS95.

Retrieves the name of the registry key in HKEY_CURRENT_USER that will be used for controlling the instrumented version.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszKey

[in/out] Address of a buffer that receives the name of the current key. If the buffer receives an empty string, then the instrumented audio source cannot be controlled by the registry.

dwKeySize

[in] Size, in bytes, of the buffer specified by pszKey. If the specified size is too small, the function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the key string.

The key defaults to an empty string, "", indicating that it cannot be controlled externally by an application changing the registry. If the key string is set then the object will monitor the key (under HKEY_CURRENT_USER) for changes. The values that can be changed are:

· "State" - DWORD - Same value as state set/get.

"WaveFileRead" - STRING - String of the wave file to be read from.

"WaveFileWrite" - STRING - String of the wave file to write to.


IAudioSourceInstumented::RegistrySet

HRESULT RegistrySet(WCHAR *pszKey);

DOES NOT WORK ON WINDOWS95.

Sets the key of the registry controlling the instrumentation.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszKey

[in] Address of a buffer that contains the key. If pszKey is an empty string, "", then the registry will not be used for instrumentation control.

See IAudioSourceInstrumented::RegistryGet.


IAudioSourceInstumented::StateGet

HRESULT StateGet(DWORD *pdwState);

Returns the instrumentation state of the object.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG

pdwState

[out] State of the system. Possible values for the state are:

· IASISTATE_PASSTHROUGH - The audio is being passed through without being recorded. This is the default action of the object upon startup unless the instrumented object is using the registry for control and it specifies a different value.

IASISTATE_PASSNOTHING - Data and notifications of available data are not passed through, preventing any audio information from getting to the speech recognizer.

IASISTATE_READFROMWAVE - Audio data is gotten from the wave file specified by IAudioSourceInstrumented::WaveFileReadSet. If no wave file exists or it is out of data then the pass through acts as though IASISTATE_PASSNOTHING were set.

IASISTATE_WRITETOWAVE - Audio data is passed through and also written to the wave file specified by IAudioSourceInstrumented::WaveFileWriteSet. If no wave file exists or it is out of data then the pass through acts as though IASISTATE_PASSTHROUGH were set.


IAudioSourceInstumented::StateSet

HRESULT StateSet(DWORD dwState);

Sets a new state for the instrumented audio source.

· Returns NOERROR if successful, or one of these error values:

E_FAIL

dwState

[in] New state. See IAudioSourceInstrumented::StateGet.


IAudioSourceInstumented::WaveFileReadGet

HRESULT WaveFileReadGet(WCHAR *pszFileName, DWORD dwFileNameSize,

DWORD *pdwNeeded);

Retrieves the name of the wave file to read from.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszFileName

[in/out] Address of a buffer that receives the name of the file. If the buffer receives an empty string, then no file will be used.

dwFileNameSize

[in] Size, in bytes, of the buffer specified by pszFileName. If the specified size is too small, the function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the key string.

The file name defaults to an empty string unless the registry is used to control the instrumentation and the key is set.


IAudioSourceInstumented::WaveFileReadSet

HRESULT WaveFileReadSet(WCHAR *pszWaveFile);

Sets the file to read from.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_WAVEFORMATNOTSUPPORTED

pszWaveFile

[in] Address of a buffer that contains the file name. If pszFileName is an empty string, "", then no file will be read from. If a file name was already set then the old file will be closed and the new one opened in its place.

If the wave format of the file does not match the wave format of the audio source object then an error will be returned. If no wave format it set then the wave format will be gotten from the file and set. See IAudioSourceInstrumented::WaveFileReadGet.


IAudioSourceInstumented::WaveFileWriteGet

HRESULT WaveFileWriteGet(WCHAR *pszFileName, DWORD dwFileNameSize,

DWORD *pdwNeeded);

Retrieves the name of the wave file to write to.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszFileName

[in/out] Address of a buffer that receives the name of the file. If the buffer receives an empty string, then no file will be used.

dwFileNameSize

[in] Size, in bytes, of the buffer specified by pszFileName. If the specified size is too small, the function returns an error and fills pdwNeeded with the number of bytes needed.

pdwNeeded

[out] Address of a variable that receives the number of bytes needed for the key string.

The file name defaults to an empty string unless the registry is used to control the instrumentation and the key is set.


IAudioSourceInstumented::WaveFileWriteSet

HRESULT WaveFileWriteSet(WCHAR *pszWaveFile);

Sets the file to read from.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY

pszWaveFile

[in] Address of a buffer that contains the file name. If pszFileName is an empty string, "", then no file will be written to. If a file name was already set then the old file will be closed and the new one opened in its place.

See IAudioSourceInstrumented::WaveFileWriteGet.


IAudioSourceNotifySink Interface

The IAudioSourceNotifySink interface allows an audio-source object to notify its user of changes in the object's internal buffer.

The IAudioSourceNotifySink interface has these member functions:

AudioStart

AudioStop

DataAvailable

OverFlow


IAudioSourceNotifySink::AudioStart

HRESULT AudioStart(void);

Notifies the engine that the audio source device has been opened.

· Return value is ignored.

This function is called as a result of a call to the IAudio::Claim member function.


IAudioSourceNotifySink::AudioStop

HRESULT AudioStop(WORD wReason);

Notifies the engine that no more data will be coming into the audio stream for an indefinite period of time.

· Return value is ignored.

wReason

[in] Reason the audio data has stopped. This parameter can be one of these values:

IANSRSN_EOF | An end-of-file character has been reached.
IANSRSN_INACTIVE | The device has become inactive for an unknown reason.
IANSRSN_NODATA | The engine called the IAudio::UnClaim member function. This notification allows the engine to employ the notification sink to monitor the object's state rather than having to maintain this information itself.
IANSRSN_PRIORITY | Resources were temporarily diverted to a higher-priority task.


IAudioSourceNotifySink::DataAvailable

HRESULT DataAvailable(DWORD dwBytes, BOOL fEOF);

Notifies the engine that the amount of data in the audio-source object's internal buffer has changed.

· Return value is ignored.

dwBytes

[in] Number of bytes of data that are currently available in the internal buffer.

fEOF

[in] TRUE if an end-of-file character has been reached and no more data is forthcoming from the audio source or FALSE if some data might remain. An audio device always returns FALSE.

For best performance, this notification should not be called by the audio-source object more than 16 times per second.

This notification is not sent when the amount of data changes as a result of the engine's own actions, such as calls to the IAudio::Flush or IAudioSource::DataGet member function.


IAudioSourceNotifySink::Overflow

HRESULT Overflow(DWORD dwBytes);

Notifies the engine that the internal buffer is full and incoming data has been lost.

· Return value is ignored.

dwBytes

[in] Number of bytes of data that were lost.


IAudioSourceSusp Interface

(New for 3.0)

The IAudioSourceSusp interface controls the pausing/starting provided by the suspeding audio source object.

Member function | Description
AudioSource | Pass the IUnknown for the audio source that inputs to the instrumented version.
Pause | Pause listening.
Resume | Resume listening.


IAudioSourceSusp::AudioSource

HRESULT AudioSource(PIUNKNOWN pAudioSource);

Passes the address of an audio source object to be suspendable.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
AUDERR_INVALIDNOTIFYSINK

pAudioSource

[in] Address of the IUnknown interface of the audio source object.

This must be called before any other function is in the IAudioSourceSusp is called. It can only be called once. The suspending audio source will rout all calls to IAudio, IAudioSource, and IAudioSourceNotifySink to/from pIAudioSource's versions of the same interfaces. When the suspending audio object is released it will call release an interfaces to the pAudioSource object.


IAudioSourceSusp::Pause

HRESULT Pause(void);

Pauses the input and frees up the wave device. (Basically calls IAudio->Stop and IAudio->UnClaim if they were called before).

· Returns NOERROR if successful, or one of these error values:

E_FAIL


IAudioSourceSusp::Resume

HRESULT Resume(void);

Pauses the input and frees up the wave device. (Basically calls IAudio->Start and IAudio->Claim if they were called before).

· Returns NOERROR if successful, or one of these error values:

E_FAIL


Low-Priority Input

The audio source object uses a feature of some sound card drivers, called "low priority." Low priority allows the speech recognition to be listening all of the time, but it allows other applications to play or record sounds. When the other application play or record sounds, the audio to the audio source object is temporarily suspended, and resumed when the other application stops playing/recording.

(New for 3.0)

However, some sound cards have bugs in their low priority support that will cause problems for end users. The audio source object has a list hard-coded in, but new drivers come out every day. An application can override whether or not low priority is used by writing to:

HKCU\Software\Microsoft\SpeechAPI, UseLowPriority=<0 or 1>

A DWORD value of 1 causes low priority to always be used, and 0 to never be used. If there is no value then the object uses its default choice.


Audio-Object Error Values

The following error values may be returned by member functions of the multimedia audio-source and audio-destination objects. If no error occurs, the member functions return NOERROR.

Error value | Description
AUDERR_ALREADYCLAIMED | The IAudio::Claim member function was already called to claim the wave device.
AUDERR_ALREADYSTARTED | The audio object has already started playing or recording data.
AUDERR_BADDEVICEID | A device identifier is invalid.
AUDERR_HANDLEBUSY | The wave device handle is in use on another thread.
AUDERR_INVALIDFLAG | A flag is invalid.
AUDERR_INVALIDHANDLE | A handle is invalid.
AUDERR_INVALIDNOTIFYSINK | An interface type is invalid.
AUDERR_INVALIDPARAM | A parameter is invalid.
AUDERR_NEEDWAVEFORMAT | A wave format that is required to obtain or set the level has not yet been set. The wave format can be set by calling the IAudio::WaveFormatSet member function.
AUDERR_NODRIVER | No driver exists for the audio device.
AUDERR_NOTCLAIMED | The IAudio::Claim member function was not called.
AUDERR_NOTENOUGHDATA | There is insufficient space in the object's internal buffer.
AUDERR_NOTSUPPORTED | The function is not supported.
AUDERR_SYNCNOTALLOWED | Synchronous wave devices, such as the system's speaker driver, are not allowed.
AUDERR_WAVEDEVNOTSUPPORTED
 | The wave device is not supported.
AUDERR_WAVEFORMATNOTSUPPORTED
 | The audio device does not allow the application to set the wave format, or the specified wave format is not supported.
AUDERR_WAVENOTENABLED | The wave device is not enabled.

In addition to the error values described in the preceding list, member functions can return error values generated by calls to the Win32 API. For more information, see "Object Levels and Error Values."


Writing an Audio Source/Destination Object

Applications can write their own audio source or destination objects to rout the audio as they desire. Some typical uses might be:

Speech Recognition Accuracy Test. Write an audio source that gets its audio from a wave file.

Speech Recognition Over the Network. The audio source gets audio from a network pipe rather than a wave device.

Cache Text-to-Speech Output. Write an audio destination that writes its audio to a wave file so that the application can save processing on text-to-speech phrases it speaks a lot.

DirectSound Output. Write an audio destination object that send its audio to the DirectSound driver rather than the wave device.

Echo, Chorus, and other Effects. The audio destination object does some digital effects on the audio before passing it down to another audio destination object.

Text-to-Speech Over the Network. Pipe the audio from the audio destination through a network pipe to another computer.

Get notified ahead of time. If some of the timing notifications come right when the text-to-speech audio is being played then the animation will always lag behind the audio. An application can write an audio destination object that passes all of the data down to another audio destination object except that it re-maps bookmarks so they happen a bit sooner than the text-to-speech engine thinks.

Writing an audio source or destination object is straight forward. Look on the speech SDK for a sample.


Introduction

(New for 3.0)

The Speech Tools objects provide common functionality to make application development quicker.


Microphone Setup Wizard

The Microphone Setup Wizard insures that the user's microphone is plugged in and working. Applications using speech recognition should give the user an opportunity to run this.

To run the Microphone Setup Wizard, the application does a CoCreateInstance on the wizard's object, CLSID_STMicWizard, and queries for the ISTMicWizard interface. It then calls ISTMicWizard->Wizard(), with a set of flags specific to the application.

The Microphone Setup Wizard understands only ISTMicWizard.


ISTMicWizard Interface

(New for 3.0)

The ISTMicWizard interface allows an application to bring up the Microphone Setup Wizard.

Member function | Description
InfoGet | Returns a structure containing information that the Microphone Setup Wizard has learned from the user.
InfoSet | Sets information for the microphone setup wizard.
Wizard | Brings up the Microphone Setup Wizard.


ISTMicWizard::InfoGet

HRESULT InfoGet(PMICWIZARDINFO pInfo);

Retrieves information obtained by the microphone setup wizard. If the wizard has not been run by the user then every element will be unknown.

pInfo

[out] Information structure.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG


ISTMicWizard::InfoSet

HRESULT InfoGet(PMICWIZARDINFO pInfo);

Writes information that the Microphone Setup wizard should use in future.

pInfo

[in] Information structure.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG


ISTMicWizard::Wizard

HRESULT Wizard(HWND hWndParent,

DWORD dwUse,

DWORD dwWaveInDevice,

DWORD dwSamplesPerSec,

DWORD dwFlags);

Brings up the Microphone Setup Wizard. The function returns after the Microphone Setup Wizard had completed.

hWndParent

[in] Parent window to create the wizard off of.

dwUse

[in] What the microphone wizard is intended to be used for. This may change some of the user interface.

Value | Meaning
STMWU_CNC | The wizard is setting the system up for command and control. It will be less strict about audio record quality.
STMWU_DICTATION | The wizard is setting the system up for dictation and will be strict about sound quality.

dwWaveInDevice

[in] Wave-In device to use for recording and to adjust. The application can use WAVE_MAPPER if it doesn't care. If the application uses WAVE_MAPPER and there is more than one sound device, the microphone setup wizard will ask the user which sound device to use. The user's choice can be obtained from ISTMicWizard::InfoGet.

dwSamplesPerSec

[in] Sampling rate that the Microphone Setup Wizard should listen for. If the sampling rate cannot be obtained then it will error out. If an application does not know which sampling rate to use then it should set 16000.

dwFlags

[in] One or more of the following values or-ed together.

Value | Meaning
STMWF_CANSKIP | If the microphone setup wizard has already been run by the user then just return success. An application can run ISTMicWizard->Wizard() every time it starts, using STMWF_CANSKIP, so that the user is guaranteed to have setup the microphone.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG

E_FAIL - The user doesn't have any sound card that can record the audio.


MICWIZARDINFO structure

typedef struct {
BOOL fHasRunMicWizard;
DWORD dwWaveInDevice;
DWORD dwMicrophoneType;
WCHAR szMicString[64];
} MICWIZARDINFO, *PMICWIZARDINFO;

fHasRunMicWizard

TRUE if the current user has run the microphone wizard before, FALSE if they have not.

dwWaveInDevice

Wave input device number they have chosen to use, of WAVE_MAPPER if unknown.

dwMicrophoneType

Microphone type. This can be one of the following:

Value | Meaning
STMWI_UNKNOWN | Unknown
STMWI_CLOSETALK | Close-talk or headset microphone.
STMWI_EARPIECE | Microphone with an element near the ear or cheek bone.
STMWI_HANDSET | Telephone-style handset.
STMWI_CLIPON | Microphone clips onto shirt.
STMWI_DESKTOP | Microphone sits on desktop.
STMWI_HANDHELD | Microphone that is meant to be held in the user's hand, a few centimeters from his/her face.
STMWI_TOPMONITOR | Microphone that sits on top of the computer monitor.
STMWI_INMONITOR | Microphone built into the monitor.
STMWI_KEYBOARD | Microphone in the keyboard.
STMWI_REMOTE | Microphone in the remote control.

szMicString

String specifying the microphone name. The wizard does not currently obtain this information from the user, but when USB microphones become common it will obtain the information from the USB microphone.


Changing Help

The Microphone Setup Wizard ships with a list of mail-order microphones vendors WWW addresses. If you wish a different list, your application should write a new help file and replace $(WINDOWS)\Speech\Speech.hlp and Speech.cnt.

The contexts used are:

232 - Buying a microphone

235 - Adjusting the playback volume

245 - Microphone is too quiet.

246 - Microphone is too loud.


Phoneme Converter

The phoneme converter converts between Unicode IPA and a more human-readable phoneme set. Applications can use the phoneme converter to display pronunciations to end users.

To run the Microphone Setup Wizard, the application does a CoCreateInstance on the wizard's object, CLSID_STPhoneConv, and queries for the ISTPhoneConv interface. It then calls ISTPhoneConv->SetByLang() to specify the character set, and ISTPhoneConv->To/FromIPA() to convert.

The Microphone Setup Wizard understands only ISTPhoneConv.


ISTPhoneConv Interface

(New for 3.0)

The ISTMicWizard interface allows an application to convert between IPA and other phonemic or phonetic character sets.

Member function | Description
FromIPA | Converts from IPA to the phone set.
Get | Gets the text for converting the current version.
SetByLang | Sets the phone set based upon the language.
SetByText | Sets the phone set based upon a text file.
ToIPA | Converts from the phone set into IPA.


ISTPhoneConv::FromIPA

HRESULT FromIPA(LPCWSTR pszFromIPA, DWORD *pdwFirstInvalid, LPWSTR pszToSet, DWORD dwNumChars, DWORD *pdwCharsNeeded);

Converts from IPA to the currently specified phone set. If no phone set is specified, it uses the thread's current language to determine which phone set is used. If no default phone set for that language exists then the English phone set is used.

pszFromIPA

[in] NULL terminated IPA string.

pdwFirstInvalid

[out] Filled with an offset in characters, from the beginning of the string, to the first invalid IPA character. If there is an invalid character, it will be ignored and skipped, but the application can display this information to the end user. Filled with (DWORD) -1 if there are no invalid characters.

pszToSet

[out] Buffer to write the converted phones into. The string will be NULL terminated.

dwNumChars

[in] Number of characters available in the buffer. If there aren't enough characters then E_FAIL is returned.

pdwCharsNeeded

[out] Filled in with the number of characters needed to convert the string. This can be NULL.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_FAIL - Buffer is too small.


ISTPhoneConv::Get

HRESULT Get(SDATA *pData);

Gets the text representation of the rules being used.

pData

[out] pData->pData is filled with a pointer to a NULL-terminated Unicode string allocated with CoTaskMemAlloc(). The application should free the memory. pData->dwSize is filled with the number of bytes in the string (including the NULL termination).

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_FAIL - Buffer is too small.


ISTPhoneConv::SetByLang

HRESULT SetByLang(LANGID lang);

Sets the phone character set to use. If no conversion set for that language exists then it returns E_FAIL.

lang

[in] Language identifier.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_FAIL - Buffer is too small.


ISTPhoneConv::SetByText

HRESULT SetByText(LPCWSTR pszText);

Sets the phone character set to use.

pszText

[in] NULL-terminated string that defines the IPA to character set conversion. See below for the format:

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_FAIL - Buffer is too small.

The text string is a NULL-terminated Unicode string. It contains new-lines (\n) to separate lines. It has a header noted by the string, "[Header]", and then rules for converting "[PhoneToIPA]". The rules are sorted in order of priority, so the first rules take precedence over later rules. Also, rules with longer matches take precedence over rules with shorter matches.

[Header]

Spaces=<1 if spaces occur between phones, 0 if they do not>

CaseSensitive=<1 if the set is case sensitive, 0 if it's case insensitive>

[PhoneToIPA]

<phone #1 text>=<IPA hexsymbol numbers separated by spaces>

<… more phones>

[Examples]

<phone #1 text>=<word using phone, with parentheses surrounding phoneme>

An example conversion might be:

[Header]

Spaces=0

CaseSensitive=0

[PhoneToIPA]

;primary stress

iy1=069 31

iy1=069 2ca

ih1=26a 31

ih1=26a 2ca

ae1=0e6 31

ae1=0e6 2ca

aa1=251 31

aa1=251 2ca

ah1=28c 31

ah1=28c 2ca

ao1=254 31

ao1=254 2ca

ay1=251 26a 31

ay1=251 26a 2ca

ay1=61 26a 31

ay1=61 26a 2ca

ax1=259 31

ax1=259 2ca

ey1=065 31

ey1=065 2ca

eh1=25b 31

eh1=25b 2ca

eh1=268 31

eh1=268 2ca

er1=25a 31

er1=25a 2ca

er1=25d 31

er1=25d 2ca

ow1=06f 31

ow1=06f 2ca

ow1=064 28a 31

ow1=064 28a 2ca

ow1=06f 28a 31

ow1=06f 28a 2ca

aw1=251 28a 31

aw1=251 28a 2ca

aw1=61 28a 31

aw1=61 28a 2ca

oy1=254 26a 31

oy1=254 26a 2ca

oy1=254 069 31

oy1=254 069 2ca

uh1=28a 31

uh1=28a 2ca

uw1=075 31

uw1=075 2ca

;secondary stress

iy2=069 32

iy2=069 2c6

ih2=26a 32

ih2=26a 2c6

ae2=0e6 32

ae2=0e6 2c6

aa2=251 32

aa2=251 2c6

ah2=28c 32

ah2=28c 2c6

ao2=254 32

ao2=254 2c6

ay2=251 26a 32

ay2=251 26a 2c6

ay2=61 26a 32

ay2=61 26a 2c6

ax2=259 32

ax2=259 2c6

ey2=065 32

ey2=065 2c6

eh2=25b 32

eh2=25b 2c6

eh2=268 32

eh2=268 2c6

er2=25a 32

er2=25a 2c6

er2=25d 32

er2=25d 2c6

ow2=06f 32

ow2=06f 2c6

ow2=064 28a 32

ow2=064 28a 2c6

ow2=06f 28a 32

ow2=06f 28a 2c6

aw2=251 28a 32

aw2=251 28a 2c6

aw2=61 28a 32

aw2=61 28a 2c6

oy2=254 26a 32

oy2=254 26a 2c6

oy2=254 069 32

oy2=254 069 2c6

uh2=28a 32

uh2=28a 2c6

uw2=075 32

uw2=075 2c6

;unstressed

iy=069

ih=26a

ae=0e6

aa=251

ah=28c

ao=254

ay=251 26a

ay=61 26a

ax=259

ey=065

eh=25b

eh=268

er=25a

er=25d

ow=06f

ow=064 28a

ow=06f 28a

aw=251 28a

aw=61 28a

oy=254 26a

oy=254 069

uh=28a

uw=075

;can't stress

th=03b8

dh=0f0

sh=283

zh=292

hh=068

nx=14b

ch=2a7

ch=74 283

jh=2a4

jh=64 292

b=062

p=070

d=064

t=074

t=27e

g=067

g=261

k=06b

f=066

v=076

s=073

z=07a

l=06c

l=26b

r=072

r=27b

r=279

y=06a

w=077

m=06d

n=06e

=020

#=020

-=02d

.=02e

?=03f

!=021

,=02c

[Examples]

iy=f(ee)l

ih=f(i)ll

ae=c(a)t

aa=f(a)ther

ah=c(u)t

ao=d(o)g

ay=t(ie)

ax=(a)go

ey=(a)te

eh=p(e)t

er=(ur)n

ow=g(o)

aw=f(ou)l

oy=t(oy)

uh=b(oo)k

uw=t(oo)

th=(th)in

dh=(th)en

sh=(sh)e

zh=a(z)ure

hh=(h)elp

nx=si(ng)

ch=(ch)in

jh=(j)oy

b=(b)ig

p=(p)ut

d=(d)ig

t=(t)alk

g=ta(g)

k=(c)ut

f=(f)ork

v=(v)at

s=(s)it

z=(z)ap

l=(l)id

r=(r)ed

y=(y)acht

w=(w)ith

m=(m)at

n=(n)o

=word-break

#=word-break

-=syllable-break


ISTPhoneConv::ToIPA

HRESULT ToIPA(LPCWSTR pszFromSet, DWORD *pdwFirstInvalid, LPWSTR pszToIPA, DWORD dwNumChars, DWORD *pdwCharsNeeded);

Converts from the currently specified phone set to IPA. If no phone set is specified, it uses the thread's current language to determine which phone set is used. If no default phone set for that language exists then the English phone set is used.

pszFromSet

[in] NULL terminated string with the phonemes.

pdwFirstInvalid

[out] Filled with an offset in characters, from the beginning of the string, to the first invalid text character. If there is an invalid character, it will be ignored and skipped, but the application can display this information to the end user. Filled with (DWORD) -1 if there are no invalid characters.

pszToIPA

[out] Buffer to write the converted IPA phones into. The string will be NULL terminated.

dwNumChars

[in] Number of characters available in the buffer. If there aren't enough characters then E_FAIL is returned.

pdwCharsNeeded

[out] Filled in with the number of characters needed to convert the string. This can be NULL.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_FAIL - Buffer is too small.


Common Information

Coding Conventions


ANSI and Unicode Versions of Speech Interfaces and Structures

In the speech API, every interface and structure that uses strings, and every structure that contains strings, actually has two versions: ANSI and Unicode. Each version is identified by a one-character suffix to the interface or structure name: A for ANSI and W for Unicode. (This is similar to the way Windows NT handles ANSI/Unicode portability.) For example, the two versions of the IAudioSource interface are IAudioSourceA for ANSI and IAudioSourceW for Unicode.

The ANSI version of an interface or structure supports characters (TCHAR) as one-byte ANSI or multibyte characters. The Unicode version supports characters as two-byte Unicode. The two interfaces have different GUIDs.

Typically, an application does not need to specify which version to use, because the base name of the interface or structure is automatically mapped to either ANSI or Unicode as follows:

· The name of an interface or structure is ordinarily mapped to the ANSI version. For example, an application that specifies SRMODEINFO gets SRMODEINFOA.

· If the application declares Unicode as its default (#define _S_UNICODE), the name of the interface or structure is mapped to the Unicode version. In this case, an application that specifies SRMODEINFO gets SRMODEINFOW.

If the application needs the version that is not automatically mapped, it must specify that version explicitly. For more information about Unicode in Windows-based applications, see the Win32 SDK.


Common Pointer Behavior and Rules

NULL pointers and handles are documented for each function where they are acceptable. If NULL is passed to a function where it is not acceptable, the function returns an E_INVALIDARG error value. Invalid handles or non-NULL pointers that point to invalid memory cause undefined behavior.


Object Levels and Error Values

The speech OLE objects are implemented on three different levels. The audio source and destination objects occupy the lowest level, the speech-recognition and text-to-speech objects occupy the middle level, and the voice-command and voice-text objects occupy the top level.

In this manual, the reference information provided for each member function in the speech API includes a list of error values that the function can return if it fails. In addition to these values, the member functions supported by the higher level objects can return error values that were actually generated by a lower level component. For example, an audio-source object can generate error values that can be passed to a speech-recognition object, and then to a voice-command object. This passing of error values applies to all functions in the speech API except for those provided by the audio object, which is the lowest level of the API.

Because speech components are local servers instead of in-process servers, it is possible for a speech function to return an error value that does not map to any of the hexadecimal error values that appear in the header files. This is because a system component has mapped the error value to an HRESULT by using the HRESULT_FROM_WIN32 macro.

If a speech function returns an error value that does not seem to be in the OLE range, you should strip off the bits added by the HRESULT_FROM_WIN32 macro, convert the error value to decimal, and then check for a non-OLE error value that matches the returned error value.


Threads

Speech objects use the OLE apartment threading model, which permits the creation of processes containing more than one thread of execution. Apartment threading rules are simple, but they must be followed carefully:

· Every object should execute on only one thread (within a single apartment).

· The OleInitialize or CoInitialize function must be called to initialize each thread.

· All pointers to objects must be marshaled between apartments (ensuring that all calls to an object are made on its thread).

· Each apartment with objects in it must have a message queue to handle calls from other processes and apartments within the same process.

· Apartment-aware in-process objects must be registered as such in the system registry (that is, the "ThreadingModel = Apartment" entry is required).

· Apartment-aware objects must write dynamic-link library (DLL) entry-point functions carefully.

Multiple objects can execute on a single thread, but no object can execute on more than one thread. In addition, it is necessary to call OleInitialize or CoInitialize on each thread, because each thread requires the services of the OLE library.

The apartment threading rules have the following implications for application and engine developers:

· The application must create all notification sinks on the same thread that creates the audio objects and engines.

· Engines must call all notification sinks on the thread in which they were created.

· Objects (with the exception of the apartment-aware DLL entry-point functions) do not need to be thread safe because they will always be called from the thread in which they were created.

For more information about OLE and OLE COM, see the Win32 SDK.


Avoiding Remote Procedure Call Exceptions

An application can cause an RPC_E_CANTCALLOUT_ININPUTSYNCCALL exception if it calls OLE functions from within a message handler. This exception occurs when the message handler calls a local server while processing a message that was generated by another process. The message can be the result of the process calling the SendMessage function directly, or calling a function that caused the system to send the message. For example, the RPC_E_CANTCALLOUT_ININPUTSYNCCALL exception would occur if an application calls the IVCmdMenu::Activate member function while handling the WM_ACTIVATE message.

To avoid this problem, an application should use its message handler to post itself a private message (WM_USER + n), and then call OLE functions from the code that handles the private message. The following example illustrates this technique.

LRESULT CALLBACK MyWindowProc (HWND, hwnd, UINT uMsg, WPARAM wParam,
LPARAM lParam)

{

switch (uMsg) {

case WM_ACTIVATE:

.

. // perform typical WM_ACTIVATE tasks

.

// Post a private message instead of calling OLE functions.

PostMessage (hwnd, WM_MYACTIVATE, wParam, lParam);

return 0;

case WM_MYACTIVATE:

pVoiceInterface->Activate(hwnd, NULL);

return 0;

.

. //..process other messages

.

}

}

A slight variation of this technique is implemented in the voice object (VOICEOBJ.CPP and VOICEOBJ.H) that is part of the VOICEMDI sample in the Microsoft Speech SDK. See especially the CVoiceMenuObject::Activate, CVoiceMenuObject::ActivateMsg, CVoiceMenuObject::Deactivate, and CVoiceMenuObject::DeactivateMsg functions.


Interfaces

This section describes the interfaces, functions, structures, and constants that are common to two or more speech objects. It also describes the UseSpeech registry entry, which all speech applications should check when starting up to determine if the user wants to use speech features or to determine if the system supports speech capabilities. Finally, this section describes voice-navigation applications and their relationship to the UseSpeech registry entry.


Enumerator Interfaces

A frequent OLE programming task is to recurrently traverse a sequence of items. OLE supports such enumerations through the use of interfaces provided by enumeration objects. The speech application programming interface (API) includes several enumeration objects that, together, provide four interfaces: IVCmdEnum, ISREnum, ISREnumSRShare, and ITTSEnum.

Each enumeration interface includes functions that are specific to the kind of item being enumerated. In addition, each interface includes the following core functions.

Interface | Description
Clone | Returns another object containing the same enumeration state as the current one.
Next | Retrieves the specified number of items in the enumeration sequence.
Reset | Resets the enumeration sequence back to the beginning.
Skip | Skips over the specified number of items in the enumeration sequence.

For more information about enumeration objects and the core member functions, see the Microsoft Win32 Software Development Kit (SDK). For more information about the IVCmdEnum interface, see the section, "Voice Commands." For more information about the ISREnum and IEnumSRSHARE interfaces, see the section, "Low-Level Speech Recognition API." For more information about the ITTSEnum interface, see the section, "Low-Level Text-to-Speech API."


Pronunciation Lexicons

Both speech-recognition and text-to-speech engines can make use of a pronunciation lexicon, which is a database of pronunciations for words and phrases to be recognized or played. Some engines allow an application or the user to edit the pronunciation lexicon to add new pronunciations or to replace incorrect pronunciations with correct ones.

An engine can give the user direct access to its pronunciation lexicon by providing a Lexicon dialog box. The Lexicon dialog box displays entries in the pronunciation lexicon and allows the user to edit them. An application causes a speech-recognition engine to display its Lexicion dialog box by calling the ISRDialogs::LexiconDlg or IVCmdDialogs::LexiconDlg member function. Similarly, an application calls the ITTSDialogs::LexiconDlg or IVTxtDialogs::LexiconDlg member function to cause a text-to-speech engine to display its Lexicon dialog box.

If an engine supports the optional ILexPronounce interface, an application can use it to query and control the engine's pronunciation lexicon. The ILexPronounce::Get member function retrieves the pronunciation and part of speech of a given word, the ILexPronounce::Add member function adds the pronunciation and part of speech of a word to the lexicon, and the ILexPronounce::Remove member function removes the pronunciation and part of speech of the given word from the lexicon.

For information about how to use the ILexPronounce interface to adjust a pronunciation, see Appendix C, "Unicode and the International Phonetic Alphabet."


IUnknown Interface

The IUnknown interface is a common interface supported by all OLE objects and, therefore, by all speech recognition and text-to-speech objects. This interface allows an application to obtain the addresses of the other interfaces supported by an object and manage the interface pointers that are obtained.

All interfaces contain the IUnknown member functions QueryInterface, AddRef, and Release. This means that once you have the address of any interface for a particular object, it is not necessary to call the IUnknown::QueryInterface member function to obtain the address of another of the object's interfaces.

For more information about the IUnknown interface and its member functions, see the Win32 SDK.


Interfaces and Member Functions

The following interfaces and member functions are common to two or more speech objects:

· The IEnumX member functions Clone, Next, Reset, and Skip are included in the following interfaces: IVCmdEnum, ISREnum, IEnumSRSHARE, and ITTSEnum.

· The ILexPronounce interface is supported by the speech-recognition and text-to-speech engine objects. This interface is optional.

· The IUnknown interface is supported by all speech objects.


IEnumX

The speech API includes a number of enumeration interfaces, including IVCmdEnum, ISREnum, IEnumSRSHARE, and ITTSEnum. Each enumeration interface includes member functions that are specific to the type of item being enumerated. However, all enumeration interfaces include the following member functions:

Clone

Next

Reset

Skip


IEnumX::Clone

HRESULT IEnumX::Clone(IEnumX **ppenum);

Retrieves another enumerator containing the same enumeration state as the current one.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
E_UNEXPECTED

ppenum

[out] Address of a variable that receives the cloned enumerator. The type of this parameter is the same as the enumerator name. For example, if the enumerator name is IEnumFORMATETC, this parameter is of the IEnumFORMATETC type.

Using Clone, it is possible to record a particular point in the enumeration sequence and then return to that point at a later time. The enumerator returned is of the same interface type as the one being cloned.


IEnumX::Next

HRESULT IEnumX::Next(ULONG celt, IUnknown **rgelt, ULONG *pceltFetched);

Retrieves the specified number of items in the enumeration sequence.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
E_UNEXPECTED
S_FALSE
S_OK

celt

[in] Number of elements to retrieve. If the number of elements requested is more than remains in the sequence, only the remaining elements are retrieved.

rgelt

[out] Address of an array that receives the elements. If an error value is returned, no entries in the array are valid.

pceltFetched

[out] Address of a variable that receives the number of array elements actually copied to the array. This parameter cannot be NULL if celt is greater than one. If this parameter is NULL, celt must be one.


IEnumX::Reset

HRESULT IEnumX::Reset(void);

Resets the enumeration sequence back to the beginning.

· Returns NOERROR if successful, or one of these error values:

S_FALSE
S_OK


IEnumX::Skip

HRESULT IEnumX::Skip (ULONG celt);

Skips over a specified number of elements in the enumeration sequence.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
E_UNEXPECTED
S_FALSE
S_OK

celt

[in] Number of elements to be skipped.


ILexPronounce Interface

The ILexPronounce interface allows an application to query and control the pronunciation lexicon for a speech-recognition or text-to-speech engine. If a speech-recognition engine supports ILexPronounce, it also supports the International Phonetic Alphabet (IPA) character set. ILexPronounce may be used by other objects in the system, such as a lexicon object. This interface is optional, and may or it may not be supported by an engine.

The ILexPronounce interface is supported by a speech-recognition engine and a text-to-speech engine. It has these member functions:

Add

Get

Remove


ILexPronounce::Add

HRESULT Add(VOICECHARSET CharSet, TCHAR *pszText, TCHAR *pszPronounce,
VOICEPARTOFSPEECH PartOfSpeech, PVOID pEngineInfo,
DWORD dwEngineInfoSize);

Adds the pronunciation, the part of speech, and optional engine information of the given word to the lexicon.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
LEXERR_ALREADYINLEX
LEXERR_INVALIDPRONCHAR
LEXERR_INVALIDTEXTCHAR
LEXERR_OUTOFDISK

CharSet

[in] Character set to use for the pronunciation.

pszText

[in] Address of a string that contains the word to pronounce.

pszPronounce

[in] Address of a null-terminated string that contains the phonetic or phonemic transcription of

pszText.

PartOfSpeech

[in] Value that indicates the part of speech of the word. For a list of possible values, see the description of the VOICEPARTOFSPEECH enumeration.

pEngineInfo

[in] Address of a buffer that contains additional information about the word to be added. If an engine can maintain additional information about a word and the application can provide the information in the format required by the engine, it can pass that information to the engine. Otherwise, this parameter can be NULL.

dwEngineInfoSize

[in] Size, in bytes, of the data pointed to by pEngineInfo.

If the lexicon already has a sense of the word with the specified pronunciation and part of speech, the pronunciation lexicon is not changed.


ILexPronounce::Get

HRESULT Get(VOICECHARSET CharSet, TCHAR *pszText, WORD wSense,
TCHAR *pszPronounce, DWORD dwPronounceSize,
DWORD *pdwPronounceNeeded, VOICEPARTOFSPEECH *pPartOfSpeech,
PVOID pEngineInfo, DWORD dwEngineInfoSize,
DWORD *pdwEngineInfoNeeded);

Retrieves the pronunciation, part of speech, and optional engine information of the given word from the lexicon.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
LEXERR_ENGBUFTOOSMALL
LEXERR_INVALIDSENSE
LEXERR_INVALIDTEXTCHAR
LEXERR_PRNBUFTOOSMALL

CharSet

[in] Character set to use for the pronunciation.

pszText

[in] Address of a string that contains the word.

wSense

[in] Alternate pronunciation or meaning of the word. Sense numbers start at zero and increment sequentially. If the sense does not exist, the function returns an error.

pszPronounce

[out] Address of a buffer that receives the phonetic or phonemic transcription of pszText. If the buffer is too small, no information is copied.

dwPronounceSize

[in] Number of bytes available in pszPronounce. If this number is too small, the function returns an error, and pdwPronounceNeeded receives the number of bytes needed.

pdwPronounceNeeded

[out] Address of a variable that receives the size, in bytes, that the buffer specified by pszPronounce needs to be to receive the transcription of pszText.

pPartOfSpeech

[out] Address of a variable that receives a value indicating the part of speech for the word. For a list of possible values, see the description of the VOICEPARTOFSPEECH enumeration.

pEngineInfo

[out] Address of a buffer that receives additional information about the word. This parameter can be NULL if the engine does not maintain additional information or if the application does not need to receive it.

dwEngineInfoSize

[in] Number of bytes of data pointed to by pEngineInfo. If this number is too small, the function returns an error and fills pdwEngineInfoNeeded with the number of bytes needed (unless pdwEngineInfoNeeded is NULL).

pdwEngineInfoNeeded

[out] Address of a variable that receives the size, in bytes, that the buffer specified by pEngineInfo needs to be to receive additional information about the word. If this parameter is NULL, the function does not return the size.

If dwPronounceSize and dwEngineInfoSize are zero, pdwPronounceNeeded and pdwEngineInfoNeeded are filled in and pszPronounce and pEngineInfo are ignored.

The pronunciation lexicon may have several versions of the same word, each representing a different pronunciation and part of speech.


ILexPronounce::Remove

HRESULT Remove(TCHAR *pszText, WORD wSense);

Removes the pronunciation, part of speech, and optional engine information of the given word from the lexicon.

· Returns NOERROR if successful, or one of these error values:

E_INVALIDARG
E_OUTOFMEMORY
LEXERR_INVALIDTEXTCHAR
LEXERR_INVALIDSENSE
LEXERR_NOTINLEX

pszText

[in] Address of a string that contains the word to remove.

wSense

[in] Alternate pronunciation or meaning of the word to remove. Sense numbers start at zero and increment sequentially. Removing one sense causes the remaining senses to be renumbered.


IUnknown Interface

The IUnknown interface is a common interface supported by all OLE objects and, therefore, by all speech objects. An application uses IUnknown to obtain pointers to other interfaces supported by an object and to manage the interface pointers after obtaining them.

The IUnknown interface has these member functions:

AddRef

QueryInterface

Release


IUnknown::AddRef

ULONG IUnknown::AddRef(void)

Increments a reference count for every new copy of an interface pointer to a specified interface on a particular object.

· Returns the value of the reference count.

When an interface is fully released, the reference count is zero. You should use this information only for diagnostics and testing.


IUnknown::QueryInterface

HRESULT IUnknown::QueryInterface (REFIID riid, LPVOID FAR *ppvObj);

Retrieves the address of a specified interface on a particular object so that an application can query an object to determine what interfaces it supports.

· Returns NOERROR if successful, or one of these error values:

CO_E_OBJNOTCONNECTED
E_NOINTERFACE
E_OUTOFMEMORY
E_INVALIDARG
E_UNEXPECTED
REGDB_E_IIDNOTREG

riid

[in] Interface identifier of the interface to be retrieved.

ppvObj

[out] Address of a variable that receives the address of the specified interface on the object. If the interface specified in riid is not supported by the object, the function returns E_NOINTERFACE. All errors set *ppvObj to NULL.


IUnknown::Release

ULONG IUnknown::Release(void)

Decrements the reference count for the specified interface on a particular object.

· Returns the value of the reference count.

When an interface is fully released, the reference count is zero. You should use this information only for diagnostics and testing.

If the object reference count goes to zero as a result of calling Release, the object is freed from memory.

If the IUnknown::AddRef member function has been called on this object's interface n times and this is the n+1th call to Release, the interface pointer frees itself. An object frees itself if the released pointer is the only pointer and if the object supports multiple interfaces through the IUnknown::QueryInterface member function.


Structures and Data Types

This section describes the structures and data types used by various speech functions.


LANGUAGE

typedef struct { // lang
LANGID LanguageID;
TCHAR szDialect [LANG_LEN];
} LANGUAGE, FAR *PLANGUAGE;

Contains the name of a language and dialect. Language and dialect strings are not case-sensitive.

LanguageID

Language identifier as specified for the Win32 API. Bits 0 through 9 identify the primary language, such as English, French, Spanish, and so on. Bits 10 through 15 indicate the sublanguage, which is essentially a locale setting. For example, Portuguese has two sublanguages, Brazilian and Standard. For a list of primary and sublanguage identifiers, see the documentation for the MAKELANGID macro in the Win32 SDK.

szDialect

Dialect specific to the language — for example, "Texas" or "New York City." This member can be "Standard" (or its localized equivalent) for the standard dialect used by the television and radio news media.


QWORD

typedef unsigned _int64 QWORD, * PQWORD;

An 8-byte unsigned integer typically used to pass a time stamp for a particular byte of audio data.


SDATA

typedef struct { // sdata
PVOID pData;
DWORD dwSize;
} SDATA, *PSDATA;

Contains the address of a data buffer and the amount of data in the buffer. An application uses the SDATA structure to pass data to various speech API functions.

pData

Address of a buffer that contains the data.

dwSize

Size, in bytes, of the data in the buffer specified by pData.


Constants

This section describes constants used by speech member functions.


VOICECHARSET

typedef enum {

CHARSET_TEXT = 0,

CHARSET_IPAPHONETIC = 1,

CHARSET_ENGINEPHONETIC = 2

} VOICECHARSET;

Indicates which character set is being used by a member function for a string.

CHARSET_TEXT

ANSI or Unicode character set.

CHARSET_IPAPHONETIC

IPA character set. This set requires a wide character (WCHAR).

CHARSET_ENGINEPHONETIC

Engine-specific phonetic character set.


VOICEPARTOFSPEECH

typedef enum _VOICEPARTOFSPEECH {

VPS_UNKNOWN = 0,

VPS_NOUN = 1,

VPS_VERB = 2,

VPS_ADVERB = 3,

VPS_ADJECTIVE = 4,

VPS_PROPERNOUN = 5,

VPS_PRONOUN = 6,

VPS_CONJUNCTION = 7,

VPS_CARDINAL = 8,

VPS_ORDINAL = 9,

VPS_DETERMINER = 10,

VPS_QUANTIFIER = 11,

VPS_PUNCTUATION = 12,

VPS_CONTRACTION = 13,

VPS_INTERJECTION = 14,

VPS_ABBREVIATION = 15,

VPS_PREPOSITION = 16

} VOICEPARTOFSPEECH;

Indicates the part of speech of a word.

VPS_UNKNOWN

Part of speech is unknown, or the application or engine does not store this information.

VPS_NOUN

Word that names a person, place, thing, or abstraction — for example, "astronaut," "New York," "printer," or "history."

VPS_VERB

Word that expresses an act or a state of being — for example, "eat" or "be."

VPS_ADVERB

Word that modifies a verb, an adjective, or another adverb — for example, "quickly" describes speed in "quickly run."

VPS_ADJECTIVE

Word that modifies a noun — for example, "green" describes color in "green grass."

VPS_PROPERNOUN

Noun that names a specific person or place — for example, "Katherine" or "Cleveland."

VPS_PRONOUN

Word that substitutes for a noun — for example, "it" in "it fell apart."

VPS_CONJUNCTION

Word used to join sentences, clauses, or phrases — for example, "dogs and cats."

VPS_CARDINAL

Number used in simple counting — for example, "one," "two," or "three."

VPS_ORDINAL

Number used to show order — for example, "first," "second," or "third."

VPS_DETERMINER

Limiting noun modifier that precedes an adjective modifying the same noun — for example, "her" in "her new house."

VPS_QUANTIFIER

Limiting noun modifier that expresses quantity and precedes an adjective modifying the same noun — for example, "four" in "the four ripe strawberries."

VPS_PUNCTUATION

Marks that clarify the meaning and structure of writing — for example, a period (.) indicates the end of a sentence.

VPS_CONTRACTION

Shortened form of one or two words with the omitted letter or letters indicated by an apostrophe — for example, "can't" is the contraction for "cannot."

VPS_INTERJECTION

Word or phrase that expresses emotion — for example, "indeed" or "heavens."

VPS_ABBREVIATION

Shortened form of a word — for example, "hr." is an abbreviation for "hour."

VPS_PREPOSITION

Word that precedes a noun or pronoun to form an adjectival or adverbial phrase — for example, "aboard" in "aboard the ship."


Common Error Values

The following errors may be returned by common member functions. If no error occurs, the member functions return NOERROR.

Error value | Description
CO_E_OBJNOTCONNECTED | The handler was created, but the object application is not running. The object application must be running to determine whether it supports an interface.
E_FAIL | The function failed.
E_INVALIDARG | One or more arguments are invalid.
E_NOINTERFACE | The interface is not supported by the object; *ppvObj is set to NULL.
E_OUTOFMEMORY | There is no more memory.
E_UNEXPECTED | An unexpected error occurred.
LEXERR_ALREADYINLEX | The lexicon already contains information for the given word.
LEXERR_ENGBUFTOOSMALL | The buffer that is to receive the additional information about the given word is too small.
LEXERR_INVALIDPRONCHAR | The string that contains the phonetic or phonemic transcription of the given word contains an invalid character.
LEXERR_INVALIDSENSE | The value that indicates the alternate pronunciation or meaning of the word is invalid.
LEXERR_INVALIDTEXTCHAR | The string that contains the word to pronounce contains an invalid character.
LEXERR_NOTINLEX | The given word is not in the lexicon.
LEXERR_OUTOFDISK | The disk does not contain enough space to save the information to be added to the lexicon.
LEXERR_PRNBUFTOOSMALL | The buffer that is to receive the phonetic or phonemic transcription for the given word is too small.
REGDB_E_IIDNOTREG | The object application is running, but riid cannot be remoted because no information exists in the registration database to indicate how it is to be done.
S_FALSE | For the Next function: Fewer elements were returned than requested by celt. In this case, unused slots in the enumeration are not set to NULL and *pceltFetched holds the number of valid entries, even if zero is returned. For the Reset function: The enumeration sequence was not reset to the beginning. For the Skip function: The number of elements skipped is fewer than celt.
S_OK | For the Next function: The requested number of elements were returned; *pceltFetched is set if the number is non-NULL. All requested entries are valid. For the Reset function: The enumeration sequence was reset to the beginning. For the Skip function: The number of elements skipped is equal to celt.

In addition to the error values described in the preceding list, member functions can return error values generated by lower-level component objects or calls to the Win32 API. For example, an IVoiceText member function might return an AUDERR_ error value generated by the audio-destination object. For more information, see "Object Levels and Error Values."


Registry Entries


Speech-Recognition Engines

When the user installs the speech-recognition engine, the Setup program should write the following registry entries:

1. A class ID entry for each OLE Component Object Model (COM) object under the HKEY_CLASSES_ROOT key. Also, apartment-aware in-process objects must be registered as such by specifying the ThreadingModel = "Apartment" entry under the InProcServer32 key. For example:

HKEY_CLASSES_ROOT
CLSID
{ClassID}
InProcServer32
(Default) = "PathToDLL"
ThreadingModel = "Apartment"

2. An engine ID entry under the following key:

HKEY_LOCAL_MACHINE\Software\Voice\SpeechRecognition\Engine

The engine ID entry should be in the following format:

EngineName : REG_SZ : EngineGUID

In an application, the speech-recognition enumerator enumerates all of the strings under this key and allows applications to access the engines' speech-recognition modes.

3. Registry entries for any custom marshaling code.

For information about class IDs and registry entries for marshaling code, see the Microsoft Win32 Software Development Kit (SDK).

The speech-recognition engine is responsible for handling all user preferences that cannot be set with the standard API. Registry values should be stored under HKEY_CURRENT_USER\Software\CompanyName.


Text-to-Speech Engines

When the user installs the text-to-speech engine, the Setup program should write the following registry entries:

1. A class ID entry for each OLE COM object under the HKEY_CLASSES_ROOT key. Also, apartment-aware in-process objects must be registered as such by specifying the ThreadingModel = "Apartment" entry under the InProcServer32 key. For example:

HKEY_CLASSES_ROOT
CLSID
{ClassID}
InProcServer32
(Default) = "PathToDLL"
ThreadingModel = "Apartment"

2. An engine ID entry under the following key:

HKEY_LOCAL_MACHINE\Software\Voice\TextToSpeech\Engine

The engine ID entry should be in the following format:

EngineName : REG_SZ : EngineGUID

In an application, the text-to-speech enumerator enumerates all of the strings under this key and allows applications to access the engines' text-to-speech modes.

3. Registry entries for any custom marshaling code.

For information about class IDs and registry entries for marshaling code, see the Win32 SDK.

The text-to-speech engine is responsible for handling all user preferences that cannot be set by the standard API. Registry values should be stored under HKEY_CURRENT_USER\Software\CompanyName.


UseSpeech Registry Entry

Some users do not wish to have the computer speak or to talk to the computer. These users can indicate their preferences by using a voice-navigation application, which records their preferences in the registry. Microsoft Voice is an example of a voice-navigation application. An application can check the value of this entry before trying to load its voice-command or voice-text code and can save memory by not loading the code if it is not needed.

An application that uses voice commands or voice text on the local computer (not necessarily over the telephone) should check the following registry entry.

HKEY_CURRENT_USER\Software\Voice\UseSpeech

UseSpeech is a DWORD value under the Voice key that is TRUE if the user has turned on speech recognition or text-to-speech using the installed voice-navigation application. An application should assume that the value is FALSE if the UseSpeech entry is not in the registry.

If the value of UseSpeech is FALSE when the application starts, it should not call the CoCreateInstance function for the CLSID_VCmd or CLSID_VTxt class identifier.

An application can use the following code to obtain the value of UseSpeech from the registry.

HKEY hKey;

DWORD dwValue, dwType = REG_DWORD, dwSize = sizeof(DWORD);

if (RegOpenKeyEx(HKEY_CURRENT_USER, "Software\\Voice", 0, KEY_READ,

&hKey) == ERROR_SUCCESS) {

RegQueryValueEx(hKey, "UseSpeech", 0, &dwType, (LPBYTE) &dwValue,

&dwSize);

RegCloseKey(hKey);

}

Whenever speech is turned on or off, the voice-navigation application sends the WM_SPEECHSTARTED or WM_SPEECHENDED message to all top-level windows in the system. An application can use these messages to determine when to dynamically create or release instances of its voice-command or voice-text objects. Although highly recommended, the use of these messages by applications is optional. These messages are not defined in the SPEECH.H header file. You can use the RegisterWindowMessage function to obtain the message identifiers to use in your application.


Voice-Navigation Applications

Speech-recognition engines and text-to-speech engines provide dialog boxes that enable the user to configure the engine. An application can direct an engine to display one of its dialog boxes by using the IVCmdDialogs or IVTxtDialogs interface. However, if a voice-navigation application, such as Microsoft Voice, is running, an application does not need to use these interfaces because the voice-navigation application gives the user access to the engine's dialog boxes.

At a minimum, a voice-navigation application has the following characteristics:

· Allows the user to turn voice commands and voice text on and off and allows access to engine dialog boxes and attribute settings, such as the awake and enabled states.

· Provides feedback that tells the user whether the voice-command or voice-text capability is on or off and, optionally, the current state of the voice-command and voice-text objects.

An application should assume that a voice-navigation application is always present in the system. An application can determine if a voice-navigation application is running by checking the value of the UseSpeech registry entry located under HKEY_CURRENT_USER\Software\Voice key. UseSpeech is TRUE if a voice-navigation application is running. If UseSpeech if FALSE or not present in the registry, there is no voice-navigation application. (The UseSpeech entry is not present if no speech software is installed in the system.)

When the user turns speech on or off, the voice-navigation application must change the value of the UseSpeech entry and broadcast the WM_SPEECHSTARTED or WM_SPEECHENDED message to all top-level windows. Changing the entry and broadcasting the messages is best accomplished by using a control panel application.

If UseSpeech is FALSE when an application starts, but the application requires speech to be useful, it should display a dialog box asking the user to turn on speech and then exit. An application that is useful even without speech features should just start without loading its speech code.

If UseSpeech is TRUE when the application starts, it should use the current attribute settings of its voice-command and voice-text sites and defer changing settings to the voice-navigation application. For sites other than the system speaker, the application should provide an interface that allows the user to adjust the attributes.

You can provide your own voice-navigation application, or you can rely on one supplied by a third party for voice-navigation features. Applications that only use telephony features or send speech output to a device other than the primary sound card can ignore the UseSpeech entry.


International Phonetic Alphabet

The International Phonetic Alphabet (IPA) is a standard system for indicating specific sounds, or phonemes. The Unicode character set contains all single symbols and diacritics in the most recently published IPA revision, as well as a few symbols no longer in use.

If an engine supports the IPA, an application can adjust the pronunciation of words by passing the engine the IPA phonemes that represent the correct pronunciation, expressed as Unicode values.

This appendix lists the Unicode values for common IPA phonemes. An application developer can use the information in this appendix to adjust pronunciations used by a speech-recognition or text-to-speech engine that supports the IPA.


Determining Whether an Engine Supports the IPA

To determine whether a text-to-speech engine supports the IPA, check the dwFeatures member of the TTSMODEINFO structure that contains information about a text-to-speech mode of that engine. If dwFeatures includes TTSFEATURE_IPAUNICODE, the mode supports the IPA.

To determine whether a speech-recognition engine supports the IPA, check whether it supports the ILexPronounce interface. If so, it also supports the IPA, because the interface requires it.


Adjusting an Engine's Pronunciation of a Word

Typically, it should not be necessary to adjust the pronunciations of words in the engine's native language. However, engines are likely to mispronounce foreign names or invented words (such as those that commonly occur in game software). An application can adjust an engine's pronunciation of a word in any of the following ways:

· If the engine has a pronunciation lexicon, the correct pronunciation can be added to the lexicon and the incorrect pronunciation (if any) removed. For more information, see the ILexPronounce interface in this section.

· The application can provide a text-to-speech engine with phonemes instead of ASCII text to be spoken. For more information, see the ITTSCentral::TextData member function in the section, "Low-Level Text-to-Speech API."

For example, suppose a text-to-speech engine mispronounces a word that is not in the engine's lexicon. Correcting the pronunciation would involve the following steps:

1. Determine what phonemes the engine is using by calling the ITTSCentral::Phoneme member function with the text of the word. For example:

pITTSCentral->Phoneme(CHARSET_IPAPHONETIC, //IPA character set
NULL, // text is not tagged
dtext, // text of the word
pdPhoneme); // function fills in pointer to phonemes

The function fills in the SDATA structure pointed to by the pdPhoneme parameter with a pointer to a WCHAR array of Unicode values, which contains the phonemes used to pronounce the word.

2. Check the phonemes to determine which ones should be changed.

3. Store the correct pronunciation in the engine's lexicon by calling the ILexPronounce::Add member function, or pass the corrected phonemes to the engine by calling the ITTSCentral::TextData member function and specifying CHARSET_IPAPHONETIC for the eCharacterSet parameter.

If an engine doesn't support the IPA or the ILexPronounce interface, it may still provide a dialog box that allows an end-user to adjust pronunciation, typically by supporting the LexiconDlg function of the ISRDialogs interface (for a speech-recognition engine) or the ITTSDialogs interface (for a text-to-speech engine). For more information, see the section, "Low-Level Speech Recognition API" or "Low-Level Text-to-Speech API."


Phonetic Alphabets

Every text-to-speech and speech-recognition engine uses its own phonetic character set, because standardized character sets are usually not as complete as necessary for the vendor's purposes. The vendor defines the way engine-specific phonemes map to standardized character sets.

The International Phonetic Alphabet (IPA) is a standard phonetic character set that is already supported in Unicode. Although the IPA is not used by any specific engine, it is an international standard. At least a rudimentary conversion between the engine's character set and the IPA set is possible, although the conversion may lose some characters. (It is unlikely that any engine will use this set internally.)

The IPA character set is defined in the Unicode standard. It occupies 0x0250 through 0x02AF for the sounds and some Latin characters, and 0x02B0 through 0x02FF for the modifier characters. If an engine's flags indicate that it understands the IPA, an application can use IPA characters with that engine.

Because of the size of the IPA character set, the only way to send IPA characters to or receive them from an engine is through Unicode. There is no 8-bit method of using a standard phonetic set. You can use the Phoneme Conversion object in Speech Tools to display an 8-bit phoneme set that is language specific.


Unicode Values for IPA Characters

The IPA standard considers its characters to be a separate alphabet that includes the lowercase Latin characters "a" through "z" and other symbols. IPA Latin characters are not duplicated in Unicode, so many IPA characters are found outside the standard phonetic and modifier blocks.

Because the IPA is primarily used by linguists to capture spoken language in print, IPA characters in the standard phonetic block are arranged in order of their resemblance to the Latin characters "a" through "z" and not their phonetic similarity. As a result, adjacent Unicode values may bear no relation to each other phonetically.

The following table lists groups of IPA characters and the Unicode blocks in which they can be found. The U+ prefix is a convention that identifies Unicode; they are 16-bit hexadecimal values.

IPA characters | Unicode block
Standard Latin | U+0041 – U+00FF
European and Extended Latin | U+0010 – U+01F0
Standard phonetic characters | U+0250 – U+02AF
Modifier letters (spacing) | U+02B0 – U+02FF
Diacritical marks (nonspacing) | U+0300 – U+036F

The symbols used for American English phonemes are listed below. Each phoneme symbol is accompanied by an example, as well as the IPA description, the Unicode name for the glyph shape used in the IPA standard phonetic charts, and the Unicode value. Some phonemic labels are described as dipthongs or affricate clusters. For these, it may be preferable to rely on the MS labels, rather than the Unicode clusters of their component phonemes, since some TTS engines will provide single combined data points for these phonemes, rather than synthesize them as combinations of separately modeled phonemes. In the Unicode names, 'LATIN' means 'LATIN SMALL LETTER' and 'GREEK' means 'GREEK SMALL LETTER'.

MS | Example | IPA Description | Unicode name | Unicode
iy | feel, eve, me | front close unrounded | LATIN I | U+0069
ih | fill, hit, lid | front close unrounded (lax) | LATIN CAPITAL I | U+026A
ae | at, carry, gas | front open unrounded (tense) | LATIN AE | U+00E6
aa | father, ah, car | back open unrounded | LATIN ALPHA | U+0251
ah | cut, bud, up | open-mid back unrounded | LATIN TURNED V | U+028C
ao | dog, lawn, caught | open-mid back round | LATIN OPEN O | U+0254
ay | tie, ice, bite | dipthong with quality: aa + ih |  |
ax | ago, comply | central close mid (schwa) | LATIN SCHWA | U+0259
ey | ate, day, tape | front close-mid unrounded (tense) | LATIN E | U+0065
eh | pet, berry, ten | front open-mid unrounded | LATIN OPEN E | U+025B
er | turn, fur, meter | central open-mid unrounded rhoticized | LATIN SCHWA W/HOOK | U+025A
ow | go, own, tone | back close-mid rounded | LATIN O | U+006F
aw | foul, how, our | dipthong with quality: aa + uh |  |
oy | toy, coin, oil | dipthong with quality: ao + ih |  |
uh | book, pull, good | back close-mid unrounded (lax) | LATIN UPSILON | U+028A
uw | tool, crew, moo | back close round | LATIN U | U+0075
b | big, able, tab | voiced bilabial plosive | LATIN B | U+0062
p | put, open, tap | voiceless bilabial plosive | LATIN P | U+0070
d | dig, idea, wad | voiced alveolar plosive | LATIN D | U+0064
t | talk, sat | voiceless alveolar plosive & | LATIN T | U+0074
 | meter | alveolar flap | LATIN R W/FISHHOOK | U+027E
g | gut, angle, tag | voiced velar plosive | LATIN SCRIPT G | U+0067
k | cut, oaken, take | voiceless velar plosive | LATIN K | U+006B
f | fork, after, if | voiceless labiodental fricative | LATIN F | U+0066
v | vat, over, have | voiced labiodental fricative | LATIN V | U+0076
s | sit, cast, toss | voiceless alveolar fricative | LATIN S | U+0073
z | zap, lazy, haze | voiced alveolar fricative | LATIN Z | U+007A
th | thin, nothing, truth | voiceless dental fricative | GREEK THETA | U+03B8
dh | then, father, scythe | voiced dental fricative | LATIN ETH | U+00F0
sh | she, cushion, wash | voiceless postalveolar fricative | LATIN ESH | U+0283
zh | genre, azure | voiced postalveolar fricative | LATIN EZH | U+0292
l | lid | alveolar lateral approximant | LATIN L | U+006C
 | elbow, sail | velar lateral approximant | LATIN L W/MIDDLE TILDE | U+026B
r | red, part, far | retroflex approximant | LATIN R | U+0072
y | yacht, onion, yard | palatal sonorant glide | LATIN Y | U+006A
w | with, away | labiovelar sonorant glide | LATIN W | U+0077
hh | help, ahead, hotel | voiceless glottal fricative | LATIN H | U+0068
m | mat, amid, aim | bilabial nasal | LATIN M | U+006D
n | no, end, pan | alveolar nasal | LATIN N | U+006E
nx | sing, anger, drink | velar nasal | LATIN ENG | U+014B
ch | chin, archer, march | voiceless alveolar affricate: t + sh |  | U+02A7
jh | joy, agile, edge | voiced alveolar affricate: d + zh |  | U+02a4

The following symbols can be used to construct phoneme strings and phonetic input to a TTS engine.

The precise effects may vary in different TTS engines.

MS | Description | Unicode name | Unicode | Usage/Effect
- | syllable boundary | HYPHEN-MINUS | U+002D | separates syllables
# | word boundary | NUMBER SIGN | U+0023 | separates words
(space) | word boundary | SPACE | U+0020 | separates words
_ | silence | UNDERLINE | U+005f | indicates silent period
1 | primary stress | DIGIT ONE | U+0031 | follows affected vowel
2 | secondary stress | DIGIT TWO | U+0032 | follows affected vowel
(blank) | unstressed | DIGIT ZERO | U+0030 | follows affected vowel
. | period | FULL STOP | U+002E | pitch fall, pause
? | question mark | QUESTION MARK | U+003F | pitch rise, pause
! | exclamation | EXCLAMATION MARK | U+0021 | raised pitch range, pause
, | comma | COMMA | U+002C | continuation rise, pause

Use the Prn control tag to indicate how to pronounce text by passing the phonetic equivalent to the engine. For information about Prn, see Appendix A, "Text-to-Speech Control Tags."

The rest of this appendix provides tables and diagrams of common phonemes and their Unicode values. For more information about IPA characters and Unicode, see the following publications:

· The Unicode Standard, Volumes 1 and 2 (Addison-Wesley).

· The Unicode Standard, Version 1.1 (The Unicode Consortium). This document can be found in the Microsoft Development Library.

· Phonetic Symbol Guide (Pullum, Geoffrey K. and Ladusaw, William A., 1986, University of Chicago Press).


Relationships Between Phonemes

The following tables and diagrams illustrate the relationships between phonemes (consonants and vowels) with Unicode values and some examples in English. Phonemes without examples are typically used in languages other than English.

If a phoneme needs to be adjusted, it will typically be to an adjacent cell in a table or node in a diagram.

Pulmonic Obstruents

A pulmonic obstruent is a phonetic stop or stop articulation in which the breath is obstructed completely or to the point of producing friction.

Plosives:

 | Voiceless sounds |  | Voiced sounds
Articulation | Value | Example | Value | Example
Bilabial | U_0070 | pig | U+0062 | big
Alveolar | U+0074 | talk | U+0064 | dig
Retroflex | U+0288 | - | U+0256 | -
Palatal | U+0063 | - | U+025F | -
Velar | U+0068 | cut | U+0261 | gut
Uvular | U+0071 | - | U+0262 | -
Glottal | U+0294 | button ("bu_n") | n/a | -

Fricatives (bilabial through palato-alveolar):

 | Voiceless sounds |  | Voiced sounds
Articulation | Value | Example | Value | Example
Bilabial | U+0278 | - | U+03B2 | -
Labiodental | U+0066 | fork | U+0076 | vat
Interdental | U+03B8 | with | U+00F0 | then
Alveolar | U+0073 | sit | U+-007A | zap
Retroflex | U+0282 | - | U+0290 | -
Palato-alveolar | U+0283 | cushion | U+0292 | measure

Fricatives (alveolo-palatal through glottal):

 | Voiceless sounds |  | Voiced sounds
Articulation | Value | Example | Value | Example
Alveolo-palatal | U+0255 | - | U+0291 | -
Palatized | U+00E7 | - | U+006A | -
Velar | U+0078 | - | U+0263 | -
Uvular | U+03C7 | - | U+0281 | -
Pharyngeal | U+0127 | - | U+0295 | -
Glottal | U+0068 | help | U+0266 | -

Doubly articulated fricatives:

 | Voiceless sounds |  | Voiced sounds
Articulation | Value | Example | Value | Example
Palatized palato-alveolar | U+0286 | - | U+0293 | -
Velar and palato-alveolar | U+0267 | - | n/a | -
Labial-velar | U+028D | - | n/a | -

Affricates:

 | Voiceless sounds |  | Voiced sounds
Articulation | Value | Example | Value | Example
Dental | U+02A3 | - | U+02A6 | -
Postalveolar | U+02A4 | judge | U+02A7 | chip
Alveolo-palatal | U+02A5 | - | U+02A8 | -

Pulmonic Resonants

A pulmonic resonant is a phonetic stop or stop articulation that is articulated without occlusion or friction.

Nasals:

Articulation | Value | Example
Bilabial | U+006D | mouse
Labiodental | U+0271 | -
Alveolar | U+006E | new
Retroflex | U+0273 | -
Palatal | U+0272 | -
Velar | U+014B | sing
Uvular | U+0274 | -

Laterals:

Articulation | Value | Example
Voiced alveolar approximant | U+006C | look
Voiceless alveolar fricative | U+026C | -
Voiced alveolar fricative | U+026E | -
Retroflex approximant | U+026D | -
Palatal approximant | U+028E | -

Median Approximants:

Articulation | Value | Example
Labiodental | U+028B | -
Post-alveolar | U+0279 | rock
Retroflex | U+027B | -
Palatal | U+006A | ya
Velar | U+0270 | -
Labial-palatal | U+0265 | -
Labial-velar | U+0077 | with

Taps, Flaps, and Trills:

Articulation | Value | Example
Alveolar trill | U+0072 | -
Alveolar fricative trill | U+027C | -
Alveolar tap or flap | U+027E | bottle
Alveolar lateral flap | U+027A | -
Retroflex flap | U+027D | -
Uvular flap or trill | U+0280 | -

Non-pulmonic Consonants

A non-pulmonic consonant is uttered with a constriction or closure at one or more points in the breath channel but without the passage of breath.

Glottalic Consonants:

 | Implosives | Ejectives
Articulation | Value | Value
Bilabial | U+0253 | U+0070 U+02BC
Alveolar | U+0257 | U+0074 U+02BC
Velar | U+0260 | U+006B U+02BC

Velaric Consonants (Stops):

Articulation | Value
Bilabial median click | U+0298
Dental or alveolar median click | U+0287
Postalveolar, retroflex, or palatal lateral click | U+0297
Alveolar or postalveolar lateral click | U+0296

Rounded Vowels

A rounded vowel is pronounced with the lips making an approximately oval opening.

Unrounded Vowels

An unrounded vowel is pronounced without rounding the lips.


Glossary


A

ACM (Audio Compression Module)

Code typically used by an engine that converts PCM data to a different format.

active voice menu

A set of voice commands that can be recognized.

archiving

Storing copies of programs and data to ensure against loss.

asleep state

The state in which an application listens to each sound, but responds only to commands on the sleep menu. See also awake state.

audio destination

A device such as an audio speaker or the telephone over which text is played as speech. An audio-destination object is an OLE COM object that supports audio communication interfaces in common with a text-to-speech engine.

audio signal

An electrical signal with varying voltage that becomes sound when amplified and converted to vibrations played by an audio speaker.

audio source

A device such as a microphone or telephone that provides audio data for speech recognition. An audio-source object is an OLE COM object that supports audio communication interfaces in common with a speech-recognition engine.

awake state

The state in which an application recognizes and executes commands on active voice menus. See also asleep state.


B

bookmark

A marker embedded in an audio recording that can be used to locate and play back an audio segment.


C

complete-phrase value

The number of milliseconds that the engine waits before regarding a phrase as complete after the user has stopped speaking.

component object

An object defined according to the OLE Component Object Model (COM). A component object has a set of interfaces that communicate with the object, data associated with an instance of the object at run-time, and the ability to support multiple instances of the object running at the same time.

COM (Component Object Model)

See OLE Component Object Model.

context-free grammar

Uses rules that predict the words that might follow the word just spoken, reducing the number of candidates that need to be evaluated to recognize the next word.

continuous speech

A continuous utterance without pauses between words. Some speech-recognition engines can recognize continuous speech.


D

degradation

A reduction in quality or performance of a communications channel.

deterioration

The gradual loss of data stored by a speech-recognition results object. The information in a results object can occupy a significant amount of memory, so an engine developer may permit the object to discard data automatically as time passes.

dictation grammar

Defines a context for the speaker by identifying the subject of the dictation, the expected style of language, and what dictation has already been done.

digital-audio format

Audio format controlled by binary or numeric data.

digital-audio stream

Continuous audio data received from or sent to an audio device.

Digital Signal Processor (DSP)

A general-purpose multiprocessor tailored to a particular type of operation. Applications involving communications, compression, and audio are more efficiently performed on a DSP than on the host computer.

diphone

A sound consisting of two phonemes: one that leads into the sound and one that finishes the sound. For example, the word "hello" consists of these diphones: silence-h h-eh eh-l l-oe oe-silence.

diphone concatenation

The text-to-speech engine concatenates short digital-audio segments and performs intersegment smoothing to produce a continuous sound.

discrete speech

Every word must be isolated by a pause before and after the word—usually about a quarter of a second—in order for the engine to recognize it.

DTMF (Dual Tone Multi-Frequency)

Touch-tone or push-button dialing. Pushing a button on a telephone keypad generates a sound that is a combination of two tones, one high frequency and the other low frequency.


E

echo canceling

A method of controlling echoing on communication lines, in which the sender checks the inbound channel for a slightly delayed duplicate of its own transmission. In echo canceling, the sender adds an appropriately modified, reversed version of its transmission to the path on which it receives information. The result is to erase the echo electronically but leave incoming data intact.

energy floor

See noise floor.

engine

A program that does the actual work of recognizing speech or translating text into speech. Most speech-recognition engines convert incoming audio data to engine-specific phonemes, which are then translated into text for use by an application. A text-to-speech engine performs the same process, only in reverse. An engine object is an OLE COM object that represents a mode of a speech-recognition or text-to-speech engine.

engine enumerator

Enumerates the speech-recognition or text-to-speech modes supported by a particular engine.

engine-specific phoneme character set

A character set that describes phonemes, pauses, and so on, and that is specific to a text-to-speech engine.


F

frequency

The rate of vibration or oscillation, measured in hertz (Hz). The normal human ear can detect sounds ranging from 20 Hz to 20,000 Hz.


G

gain

The increase in signalling power, measured in decibels (dB), that occurs as the signal is boosted by an electronic device.

global voice menu

A menu that is active all of the time regardless of which window is in the foreground.

grammar

A set of words and phrases that can be recognized by an engine. A grammar object is an OLE COM object that an application uses to control how an engine uses the grammar to recognize speech.

GUID

Globally unique identifier used by an interface or object for identification.


I

incomplete-phrase value

The number of milliseconds that the speech-recognition engine waits before discarding an incomplete phrase because the user has stopped speaking.

interface

A set of semantically related functions that an application can call to perform the actions defined for that interface.

interference

Noise or other external signals that affect the performance of a communications channel; also, the electromagnetic signals generated by electronic devices, such as computers, that can disturb radio or television reception.

IPA (International Phonetic Alphabet)

A standard system for indicating specific sounds, first introduced in 1886. The Unicode character set includes all single symbols and diacritics in the most recent revision of the IPA, which occurred in 1989, as well as a few IPA symbols no longer in use.


L

lexicon

See pronunciation lexicon.

limited-domain grammar

Provides a set of words to recognize without using strict syntax structures. A limited-domain grammar is a hybrid between a context-free grammar and a dictation grammar.

localization

Adaptation of a software package from English to the needs of a foreign country.


M

marshaling

If an instance uses a separate process space from that of the application which invokes it, its data must be marshaled across the process boundary. Each interface contains marshaling code that allows its parameters to be transmitted across process boundaries.

matching techniques

The methods by which the engine matches a detected word to known words in its vocabulary.


N

node

A word or phoneme on a recognition path in a recognition/alternative graph generated by an engine.

noise

Any interference that affects the operation of a device. In communications, noise consists of random electronic signals, produced either naturally or by the circuitry, that degrade the quality or performance of a communications channel.

noise floor

The noise value in the signal-to-noise (SNR) ratio for an environment. In general, the higher the noise floor, the more sensitive the engine will be to background noise.

notification sink

Similar to a callback function, except the sink is implemented as an interface with a set of functions rather than as a single function.


O

OLE Component Object Model (COM)

A specification that defines a binary standard for OLE object implementation independent of programming language.


P

PCM (pulse code modulation)

The most common method of encoding an analog voice signal into a digital bit stream. First, the amplitude of the voice conversation is sampled. Then, the sample is coded into binary data which can then be switched, transmitted, and stored digitally.

perplexity

The number of choices at a given node in a recognition path.

phoneme

The smallest structural unit of sound in any language that can be used to distinguish one word from another.

phrase

An ordered list of words that are spoken in the same utterance.

pitch

The tone of a sound, which generally is determined by the sound's frequency. A high-pitched sound has a higher frequency; a low pitched sound has a lower frequency.

pronunciation lexicon

A database of pronunciations maintained by a speech-recognition or text-to-speech engine. An engine may allow an application to collect new or corrected pronunciations from the end-user.

pronunciation rule

A rule followed by a text-to-speech engine to convert text into phonemes.

prosody

The inflection, timing, and accent of speech.


R

recognition mode

Each speech-recognition engine supports one or more recognition modes that conform to a different code set or data set. For example, each language (or dialect) supported by the engine will have a different mode.

recognition path

A sequence of words or phonemes that an engine analyzed while attempting to recognize an utterance.

recognition rule

A rule followed by a speech-recognition engine using a context-free grammar to recognize speech.

recognition/alternative graph

A graph generated by a speech-recognition engine that depicts the recognition paths explored by the engine in recognizing an utterance.

recursion

The number of levels of rules in a context-free grammar.

registry

The database in which configuration information is stored. The database takes the place of most configuration and initialization files for Windows and new Windows-based programs.

results object

See speech-recognition results object.

rules

See pronunciation rule, and recognition rule.


S

SNR (signal-to-noise ratio)

The amount of power, measured in decibels (dB), by which a signal exceeds the amount of channel noise at the same point of transmission. It provides an indication of the clarity or accuracy with which communication can take place.

speaker

The end-user who utters the speech to be recognized by an application. Training performed by a speaker may be stored in a speaker profile.

speaker-adaptive

The engine trains itself to recognize the user's voice while the user performs ordinary tasks.

speaker-dependent

The engine requires the user to train it to recognize his or her voice.

speaker-independent

The engine does not require training. Speaker-independent engines typically start with an accuracy above 95 percent for most users (those who speak without accents).

speaker profile

All of the information the engine has about the speaker, such as a data header, languages for which training has been done, known patterns of speech and the language model, how specific words are pronounced, phonetic training, speaker ID, and speaker preferences.

speech recognition

The ability of a computer to understand the spoken word for the purpose of receiving command and data input from the speaker.

speech-recognition engine

An OLE Component Object Model dynamic-link library (DLL) or executable file (.exe) that performs recognition from a digital-audio stream. Speech-recognition engines are supplied by vendors who specialize in the software.

speech-recognition enumerator

Enumerates the engines that are available to an application.

speech-recognition mode

An engine typically provides an assortment of modes that can be used to recognize speech in different languages, dialects, and audio-sampling rates.

speech-recognition results object

Provides detailed information about a speech-recognition event.

speech-recognition sharing object

Enumerates shared engine-audio source pairs, or creates new ones.

subword matching

The engine looks for subwords—usually phonemes—and then performs further pattern recognition on those.

synthesis

The text-to-speech engine synthesizes the glottal pulse from human vocal cords and applies various filters to simulate throat length, mouth cavity, lip shape, and tongue position.


T

tags

See text-to-speech control tags.

text-to-speech

Technologies for converting textual (ASCII) information into synthetic speech output. Used in voice-processing applications requiring production of broad, unrelated, and unpredictable vocabularies, such as products in a catalog or names and addresses. This technology is appropriate when system design constraints prevent the more efficient use of speech concatenation alone.

text-to-speech control tags

Instructions that can be embedded in text sent to a text-to-speech engine to improve the prosody of the spoken text.

text-to-speech engine

An OLE Component Object Model dynamic-link library (DLL) or executable file (.exe) that provides functionality for converting text to digital-audio speech. Text-to-speech engines are supplied by vendors who specialize in the software.

text-to-speech enumerator

Enumerates the text-to-speech modes provided by all of the engines available to the application.

text-to-speech mode

Analogous to voice quality or personality. Every text-to-speech mode is different, and each allows for different properties such as timbre, accent, language, and digital-audio sampling rate.

threshold

The point below which an utterance is rejected as unrecognized.

training

The process of speaking a series of preselected phrases for the engine. This provides the engine with more information about the voice of the speaker and can improve speech recognition.


U

Unicode

A 16-bit character set that replaces ASCII and allows any character from any language to be represented in a text string. The Unicode character set contains a subset for International Phonetic Alphabet (IPA) phonemes.

utterance

Anything heard by the engine as a finite series of sounds that the engine attempts to recognize as speech.


V

vocabulary

A set of words used in a grammar. A speech-recognition engine typically supports several different sizes of vocabulary, which determine the words that the engine can recognize in a given state.

voice command

A word or phrase associated with a voice menu. When an engine recognizes a voice command, it notifies the application that owns the voice menu containing the command.

voice-command site

A speech-recognition mode and audio source that together serve as a source of voice-command input.

voice menu

A list of voice commands to which an application can respond. A voice menu must be active before an engine can recognize its commands.

voice-text site

A text-to-speech mode and an audio destination that together serve as a destination for voice-text output.

VU (Volume Units) Meter

An indicator that displays the volume of sound being received by the microphone or through the line-in port. Optimum reception is achieved when the meter registers in the middle area.


W

whole-word matching

The engine compares the incoming digital-audio signal against a prerecorded template of the word.

word

An atomic Unicode text string. A "word" can have several related vernacular words (such as "Los Angeles") within it because the vernacular words are always used in common.

word separation

The degree of isolation between words required for the engine to recognize a word.

word spotting

A series of words may be spoken in a continuous utterance, but the engine recognizes only one word or phrase.


Developing a Speech Engine

This appendix contains information for developers of speech-recognition and text-to-speech engines. It describes how to develop an engine that is compatible with the speech application programming interface (API).

Before reading this document, you should read the section, "Low-Level Speech Recognition API" and "Low-Level Text-to-Speech API." You should also be familiar with the objects, interfaces, and member functions that support speech-recognition and text-to-speech.


Developing a Speech-Recognition Engine

This section contains information to help you develop a speech-recognition engine that is compatible with the speech API.


Porting an Existing Speech-Recognition Engine

Typically, an engine developer ports the code for an existing speech-recognition engine from 16-bit Windows, UNIX®, Macintosh®, or OS/2®, rather than developing a completely new engine. Porting a speech-recognition engine to the speech API usually involves the following steps:

1. Port the engine code to Win32 by using the API that is already in the code. This should be straightforward, but be aware that this may uncover old bugs because Win32 is more stringent about memory usage.

2. Move the engine code into a dynamic-link library (DLL).

3. Implement the major interfaces for speech recognition first to get the engine running with the speech API as quickly as possible.

4. Redesign the engine's audio input and output architecture to use streams. Using streams makes the engine consistent with the multimedia audio source and multimedia audio-destination objects, which also use streams. These objects are similar to multimedia wave-in and wave-out drivers.

5. Design the engine's interface architecture to support the interfaces defined for the speech-recognition engine object.

6. Implement the speech-recognition interfaces and functions for the engine.

7. Write additional code to enable the engine to support multiple instances. The time required to do this varies depending on the engine's internal architecture.

The engine must provide the engine enumerator, engine object, and grammar object, and may optionally provide a speech-recognition results object. The following table lists the required and optional interfaces for each object.

The engine should support both ANSI and Unicode implementations of each interface.

Object | Interface
Engine enumerator | ISREnum ISRFind (optional)
Engine | ILexPronounce (optional) ISRAttributes ISRCentral ISRDialogs ISRSpeaker (optional)
Grammar | ISRGramCFG (context-free grammar) ISRGramCommon ISRGramDictation (dictation grammar)
Speech recognition results (optional) | ISRResAudio (optional) ISRResBasic (optional) ISRResCorrection (optional) ISRResEval (optional) ISRResGraph (optional) ISRResMemory (optional) ISRResMerge (optional) ISRResSpeaker (optional)

The engine should also support the following notification sinks to pass information back to the application.

· ISRNotifySink. Used by an engine object to notify the application of interference, noise, and other factors that affect speech recognition.

· ISRGramNotifySink. Used by a grammar object to notify the application about a particular recognition.


Registry

When the user installs the speech-recognition engine, the setup program should write the following registry entries:

1. A class identifier entry for each OLE Component Object Model (COM) object.

2. An engine identifier entry under the following key:

HKEY_LOCAL_MACHINE\Software\Voice\SpeechRecognition\Engine

The engine identifier entry should be in the following format:

EngineName : REG_SZ : EngineGUID

In an application, the speech-recognition enumerator enumerates all of the strings under this key and allows applications to access the engines' speech-recognition modes.

3. Registry entries for any custom marshaling code.

For more information about class identifiers and registry entries for marshaling code, see the Microsoft Win32 Software Development Kit (SDK).


Supporting Multiple Engine Instances

It is important that an engine support multiple instances, both within the same process and within multiple processes, for the following reasons:

· If multiple instances are not supported, one application may preclude another from using speech recognition.

· Speech recognition may be occurring simultaneously from multiple sources, such as the user's computer, one of several telephone lines, and multiple files.

The speech-recognition engine is an OLE COM object based on a DLL. This means that it uses the process and memory of the executable module that calls it.

In general, each engine object should be independent of any other objects created by the DLL. However, some data should be shared to reduce memory consumption.

To minimize memory usage and start-up times, all instances should share common databases, such as lexicons and speaker-independent models. In addition, because the instances are in the same process space, they can share memory. However, make sure the instances cooperate when they share data, especially if they modify it. Note also that the instances may be running under separate threads, so mutual exclusion calls will be necessary.

You should employ the following protection schemes:

· Changing a run-time attribute for one instance should not affect other instances.

· Enabling, disabling, pausing, or resuming one instance should not affect other instances.

· One system user (as logged on) should not be able to affect the engine preferences for another system user.

After all of the engine objects are released, the DLL should free any shared memory used among the instances.

For information about sharing a speech-recognition engine among multiple applications, see the section, "Low-Level Speech Recognition API."


Supporting Multiple Languages

To support multiple languages, an engine should:

· Provide one or more speech-recognition modes for each language, each with its own globally-unique identifier (GUID). If the engine can support multiple languages at one time, all of the modes should have the same engine GUID.

· Load the modes for a language when the application uses that particular language.

· Do not provide a mode for a language if the application's setup program does not install a mode for that language.

In some cases, one or more applications will need to have several languages active and simultaneously recognizing speech from the same audio source. This is especially true in multilingual countries or countries that do not have localized versions of all products.


Suggested Waveform-Audio Formats to Support

It is strongly recommended that a speech-recognition engine support the following waveform-audio formats:

· 11 kilohertz (kHz), 8-bit mono pulse code modulation (PCM). Every sound card can support this format.

· 11 kHz, 16-bit mono PCM. Most sound cards can support this waveform-audio format.

· 8 kHz, 8/16-bit mono PCM. Telephony cards can output this format.

An engine can also support the following waveform-audio formats:

· 16 kHz or 22 kHz, 8/16-bit mono PCM. The higher fidelity increases accuracy.

· Custom formats. Supporting one or more custom formats allows an engine to take advantage of digital signal processing (DSP) hardware that has waveform-audio drivers that support the custom formats.


Using a DSP to Process Information

Microsoft highly recommends that an engine be capable of recognizing speech without requiring any assisted hardware. But it should be capable of taking advantage of such hardware if it is present to

A speech-recognition engine can use a DSP in any of the following ways, singly or in combination:

1. The DSP board has a wave-in driver written for it, supporting a custom waveform-audio format that represents the normal output of the DSP (some sort of fast Fourier transform (FFT), for example). All engines call the IAudio::WaveFormatSet member function one or more times when they start, usually going through different PCM types, such as different sampling rates and bits per sample. The engine could also query to see if the waveform-audio device supports the "custom" format (FFT, for example) and use that data format if the device supports it. This may not provide enough of a communication path between the DSP and engine, and it requires that the wave-in driver be written by the engine vendor.

2. The DSP board has a wave-in driver written for it. The engine can query the IAudioMultiMediaDevice::DeviceNumGet member function to determine the waveform-audio device to which it is connected. If the device number of that waveform-audio device is known to be a friendly DSP board, the engine can begin communicating with the DSP directly, or it can go through the wave-in custom messages. The disadvantage of this is that it requires the wave-in driver be written by the engine vendor.

3. The DSP board provides an API that takes in PCM data and asynchronously outputs FFT data, for example. If the engine gets PCM data, it tries to pass the PCM data down to the DSP to have it do the processing. The disadvantage of this is that a lot of data passes back-and-forth across the bus, but it allows the DSP to be used independent of the origin of the audio.

4. The engine vendor writes a custom audio-source object that communicates directly with the computer. This allows maximum flexibility, but it prevents the engine from being used by applications not specifically written for the engine.


Preventing the Exclusive Capture of the Wave-In Device

The speech-recognition engine should have the audio source active only while it needs audio, so that if it's attached to a wave-in device, other applications that need the device will not be prevented from using it. To do this:

· Call the IAudio::Start member function only when a grammar is active and listening.

· Call the IAudio::Stop member function when all of the grammars have been deactivated or have stopped listening.


Handing Echo Cancellation

If an engine is recognizing speech input from the computer's microphone while speaking output over an audio speaker on that computer, the engine may recognize its own output as input. The same problem occurs over telephone lines. However, the engine can act as if the computer's output signal has been removed and echo-cancellation has been done on any digital-audio data passed to the engine.

The audio driver that handles the output to the attached speakers and the input from the microphone prevents this echoing problem by removing the output signal (referred to as echo cancellation). Echo cancellation is best left to the audio driver because:

· The speech-recognition engine should be hardware-independent. If the engine performed its own echo cancellation, it would need to receive the digital-audio stream from the computer's speaker or telephone line.

· Every application that plays digital audio cannot be expected to forward all audio through the speech-recognition engine because multimedia drivers do not guarantee synchronization of the digital input with the digital output. Only the driver can guarantee the synchronization of the analog-to-digital converter (ADC) and the digital-to-analog converter (DAC).

· The sampling rate of the output wave may differ from the input source. The volume level and waveform-audio format can also differ.

· Even if these issues were resolved, there are other ways to produce audio from a computer, such as with music synthesis.

The following mechanisms for echo canceling exist:

· Assume that the audio source is already echo canceled and that the computer will not be hearing itself.

· Because many engines are already working in the frequency domain, some engines may be able to do their own echo canceling if they know what's being summed with the input channel. Engines that can do this should invent a new waveform-audio format and register it with Multimedia Systems. The waveform-audio format should contain two channels, one with the input signal and the other with the signal being summed/output to the channel.

When an engine starts, it tries to call the IAudio::WaveFormatSet member function with the custom format. If the hardware drivers that the engine is connecting to can handle the format, the drivers can return the echo information. Audio sources that cannot provide echo-canceling features return an AUDERR_WFNOTSUPPORTED error. The engine can then attempt to open the device with a more traditional format, like PCM mono.


Supporting Scaleable Platforms

It is highly recommended that a speech-recognition engine be scalable so that it can work with different processor speeds and memory configurations. This not only broadens the engine's market, but it provides experienced users with the ability to make tradeoffs between recognition accuracy and demands for processor resources.

When an engine is first installed or loaded, it should analyze the machine and make certain tradeoffs for optimal performance. To run effectively on slower machines, an engine should accept a higher error rate in recognitions.To take advantage of faster machines, the engine should:

· Use threads so the engine runs on symmetric multi-processor (SMP) machines that are running with Windows NT.

· Take advantage of a DSP if one is available.


Supporting Recognition Over the Telephone

A speech-recognition engine should support recognition over the telephone line as well as from a wave-in audio device connected to the computer. Here are a few suggestions:

· Supply at least one speech-recognition mode that supports an 8-kHz sampling rate.

· Support multiple instances of the engine while recognizing input from different sources. This allows recognition to occur over the computer and the telephone line at the same time, or several telephone lines can be connected to the same machine, all of them active at the same time.

· Set the flag in the SRMODEINFO structure that indicates that the mode is designed for the telephone.


Connecting Directly to Audio Hardware

An existing engine may connect directly to audio hardware, usually to a circuit board with a hardware assist. If necessary, an engine can connect directly to the hardware and still be ported to the speech API. Be aware, however, that the engine will work only with applications specifically designed to cooperate with that engine, which limits the usefulness of the engine.

To enable an engine to connect directly to audio hardware:

1. Create a custom OLE interface — for example, ICustom — that provides direct control of the hardware.

2. Provide a custom OLE object — for example, "custom audio source" — that supports the ICustom interface.

To enable an application to recognize audio directly from the hardware:

1. Create an instance of the custom audio-source object and pass the instance to the engine.

2. When the application passes a pointer to an IUnknown interface to the engine, the engine calls the IUnknown::QueryInterface member function for the ICustom interface.

3. If the ICustom interface is found, the engine uses ICustom to control and configure the hardware.

4. If the ICustom interface is not found, the engine uses the digital-audio data supplied by the IAudio and IAudioSource interfaces.


Managing User Preferences

One system user should not be able to affect the preferences, lexicon, and other settings of another user. This is especially true for an engine running on Windows NT.

The speech-recognition engine is responsible for handling all user preferences that cannot be set with the standard API. To do this, an engine should store registry values under the HKEY_CURRENT_USER key.


Maintaining a Database of Compiled Grammars

The speech API was written with the assumption that applications, rather than the engine, will store compiled grammars for the following reasons:

· The application can determine when the compiled grammars are no longer needed and can free them.

· The engine doesn't need to store a database.

· The compiled grammar can be transferred to another machine.

It may be preferable for some engines to store compiled grammars, especially if they include user-specific information in the grammar. To store compiled grammars, the engine can maintain a database of grammars. Instead of filling in the native grammar with a representation of the grammar, it can store a handle to the grammar.

When the application eventually passes back the handle of the native grammar, the engine uses the handle to access its database. This handle should be robust enough to be transferred to another machine and recognized as foreign by the engine on the other machine. In addition, the engine should be able to determine whether the handle was deleted because the engine's database was too large.


Defining Additional Interfaces

An engine can provide additional functionality to an application by supporting custom interfaces. These interfaces can expose features that the speech API does not yet support. To support a custom interface:

· Define an interface that follows the OLE COM and that contains all of the functions necessary to support the additional functionality.

· Provide marshaling for the interface so the engine can run in a different process than the calling application.

· Enable the engine to return a pointer to the interface when the engine receives a call of the IUnknown::QueryInterface member function for that interface.

· Provide documentation for the interface so that application developers can take advantage of it.


Distributing Processing Over a Network

An engine might want to do some of its processing on a different machine than the one the application is running on. While the Microsoft Speech API provides no official way of implementing the distributed processing, it's fairly easy for an engine to distribute processing.

To distribute processing, split the engine into two parts.

The first part is basically a stub that takes all of the calls to the speech API and marshals the data over the network, to the second part. The engine stub supports all of the interfaces that the full engine supports, although it does almost none of the processing. In order to reduce network usage, the stub might do feature extraction on the host.

The second part of the engine lies on a server. It accepts the marshaled data from the stub and then processes it, sending back notifications to the stub as recognitions occur.

The engine might also support a custom interface, IMyDistributed. The IMyDistributed interface allows an application that knows about remote processing to control which machine the processing is on, and to finely control what kind of data is transmitted over the LAN (or other communication method) so that unneeded data (perhaps results objects) aren't transmitted.


Getting a Fake Engine from Microsoft

Developers that are converting their engine to the Microsoft Speech API can write to MSSpeech@Microsoft.Com for the source code to a "fake" speech recognition engine that works with the API.


Debugging the Engine in Applications that Use Voice Commands/Dictation

To debug the engine in applications using voice text:

Open Visual C (or whatever debugging environment you're using) with the engine DLL.

Set the application to run as "$(WINDOWS)\Speech\VCMD.EXE -Embedding". $(WINDOWS) should be replaced with your windows directory.

Set a break-point in the engine's class factory so that you know that the debugging is working.

Start the debugging session. This will run VCMD.EXE. The debugger will be running although the engine won't be loaded yet.

Run the application that uses Voice Commands or Voice Dictation.

Your engine's breakpoint should bit hit.


Low-Level API Calls used by the Speech Recognition Enumerator

The speech recognition enumerator uses the following interfaces and functions of the low-level API.

ISREnumW

Next

Skip

Reset

Clone

Select


Low-Level API Calls used by Voice Commands

The high-level Voice Commands API works with any speech recognizer that supports continuous context-free grammars. It uses the following interfaces and functions of the low-level API

ISRAttributesW

AutoGainEnableGet/Set

MicrophoneGet/Set

SpeakerGet/Set

ThresholdGet/Set

TimeOutGet/Set

ISRCentralW

GrammarLoad - Context free grammars. Some "words" have numbers and other symbols.

ModeGet

Register

Resume

UnRegister

ISRDialogsW

AboutDlg

GeneralDlg

LexiconDlg

TrainGeneralDlg

TrainMicDlg

ISREnumW

Next

Skip

Reset

Clone

Select

ISRGramCommonW

Activate

Archive

Deactivate

DeteriorationGet

DeteriorationSet

TrainDlg

ISRGramCFGW

ListAppend

ListGet

ListQuery

ListRemove

ListSet - Some list entries may have multiple words.

ISRGramNotifySinkW

PhraseFinish

PhraseStart

Training

ISRNotifySinkW

AttribChanged

Interference

Sound

UtteranceBegin

UtteranceEnd

VUMeter

ISRResAudioW

GetWAV

ISRResBasicW

FlagsGet

Identify

PhraseGet

TimeGet

ISRResXXX

Results objects can be exported to the application, so any results object can be accessed.


Low-Level API Calls used by Voice Dictation

The high-level Voice Dictation API works with any speech recognizer that supports discrete/continuous dictation or limited-domain grammars. It uses the following interfaces and functions of the low-level API

ILexPronounceA/W

Add

Get

Remove

ISRAttributesW

AutoGainEnableGet/Set

EchoGet/Set

EnergyFloorGet/Set

MicrophoneGet/Set

RealTimeGet/Set

SpeakerGet/Set

ThresholdGet/Set

TimeOutGet/Set

ISRCentralW

Pause

GrammarLoad

ModeGet

PosnGet

Register

Resume

ToFileTime

UnRegister

ISRDialogsW

AboutDlg

GeneralDlg

LexiconDlg

TrainGeneralDlg

TrainMicDlg

ISREnumW

Next

Skip

Reset

Clone

Select

ISRGramCommonW

Activate

Archive

Deactivate

DeteriorationGet

DeteriorationSet

TrainDlg

ISRGramDictationW

Context

Hint

Words

ISRGramNotifySinkW

PhraseFinish

PhraseStart

ReEvaluate

Training

UnArchive

ISRNotifySinkW

AttribChanged

Interference

Sound

UtteranceBegin

UtteranceEnd

VUMeter

ISRResAudioW

GetWAV

ISRResBasicW

FlagsGet

Identify

PhraseGet

TimeGet

ISRResCorrectionW

Correction

Validate

ISRResMemory

Free

Get

LockGet

LockSet

ISRSpeakerA/W

Delete

Enum

Merge

New

Query

Read

Revert

Select

Write


How a dictation engine passes up punctuation and numbers...

Generally, the Voice Dictation module will transcribe exactly what the engine passes up to it. However if it sees a backslash in a "word" then it puts whatever is to the left of the back-slash in the text buffer. This would be used for punctuation, like:

.\Period

;\Semi-colon

\\Back-Slash

If a number is passed up (because the recognizer supports continuous digits) then the number is added to the text buffer without alteration.


Developing a Text-to-Speech Engine

This section contains information to help you develop a text-to-speech engine that is compatible with the speech API.


Porting an Existing Text-to-Speech Engine

Typically, an engine developer ports the code for an existing text-to-speech engine from 16-bit Windows, UNIX, Macintosh, or OS/2, rather than develop a completely new engine. Porting a text-to-speech engine to the text-to-speech API usually involves these steps:

1. Port the engine code to Win32 by using the API that is already in the code. This should be straightforward, but be aware that this may uncover old bugs because Win32 is more stringent about memory.

2. Move the engine code into a DLL.

3. Implement the major interfaces for text-to-speech translation first to get the engine running with the speech API as quickly as possible.

4. Redesign the engine's audio input and output architecture to use streams. Using streams makes the engine consistent with the multimedia audio source and multimedia audio-destination objects, which also use streams. These objects are similar to multimedia wave-in and wave-out drivers.

5. Design the engine's interface architecture to support the interfaces defined for the text-to-speech engine object.

6. Implement the text-to-speech interfaces and functions for the engine.

7. Write additional code to enable the engine to support multiple instances. The time required to do this varies depending upon the engine's internal architecture.

The engine must provide the engine enumerator and the engine object. The following table lists the required and optional interfaces for each object.

The engine should support both ANSI and Unicode implementations of each interface.

Class | Interface
Engine enumerator | ITTSEnum ITTSFind (optional)
Engine | ILexPronounce (optional) ITTSAttributes ITTSCentral ITTSDialogs

The engine should also support the following notification sinks to pass information back to the application.

· ITTSBufNotifySink. Notify the application of changes to the buffer that contains the text being spoken.

· ITTSNotifySink. Notify the application of the phoneme being spoken or that audio has started or stopped.


Registry

When the user installs the text-to-speech engine, the setup program should write the following registry entries:

1. A class identifier entry for each OLE COM object.

2. An engine identifier entry under the following key:

HKEY_LOCAL_MACHINE\Software\Voice\TextToSpeech\Engine

The engine identifier entry should be in the following format:

EngineName : REG_SZ : EngineGUID

In an application, the text-to-speech enumerator enumerates all of the strings under this key and allows applications to access the engines' text-to-speech modes.

3. Registry entries for any custom marshaling code.

For more information about class identifiers and registry entries for marshaling code, see the Win32 SDK.


Supporting Multiple Engine Instances

It is important that engines support multiple instances, both within the same process and within multiple processes, for the following reasons:

· If multiple instances are not supported, one application may preclude another from using text-to-speech.

· Text-to-speech may be occurring over multiple destinations at the same time, such as the user's computer, one of several telephone lines, and multiple files.

The text-to-speech engine is an OLE COM object based on a DLL. This means that it uses the process and memory of the executable that calls it.

In general, each engine object should be independent of any other objects created by the DLL. However, some data should be shared to reduce memory consumption.

To minimize memory usage and start-up times, all instances should share common databases, such as dictionaries and phoneme models. In addition, because the instances are in the same process space, they can share memory, or the data can be shared between processes by using shared memory files. However, make sure the instances cooperate when they share data, especially if they modify it. Note also that the instances may be running under separate threads, so mutual exclusion calls will be necessary.

You should employ the following protection schemes:

· Changing a run-time attribute for one instance should not affect other instances.

· Enabling, disabling, pausing, or resuming one instance should not affect other instances.

· One system user (as logged on) should not be able to affect the engine preferences for another system user.

After all of the engine objects are released, the DLL should free any shared memory used among the instances.


Buffering Text with ITTSCentral::TextData

For buffering and playing back text, Microsoft suggests the following approach, which uses the ITTSCentral::TextData, ITTSCentral::AudioPause, and ITTSCentral::AudioResume member functions. More complex schemes using threads can be easily derived from this scheme.

1. When the text-to-speech engine object is first created, it is in a playback state. Calling the ITTSCentral::AudioPause member function pauses the engine; calling the ITTSCentral::AudioResume member function returns it to the playback state.

2. The engine maintains an internal linked list of text buffers. Initially, the list is empty.

3. When the application calls the ITTSCentral::TextData member function with new data, the engine allocates memory for the data, copies the text into the memory, and adds the memory to the linked list. If the buffer was empty and the engine is in the playback state, the engine calls TextToAudio, an internal function that processes the data.

4. Reading from a position at the top of the linked list, TextToAudio converts the list to phonemes and then into digital audio. The engine then calls the IAudioDest::DataSet member function to pass the data to the audio-destination object, continuing until the digital-audio object cannot take any more data. Note that if the engine is paused, the IAudioDest::DataSet member function does nothing.

5. In the background, the audio-destination object's internal buffer gradually empties as the digital audio is played over the speaker. When the status of the internal buffer changes, the audio-destination object notifies the engine, which then calls TextToAudio to convert more data.

6. When the application calls the ITTSCentral::AudioPause member function on the engine, the engine is paused and it calls the IAudio::Stop member function on the audio-destination object.

7. When the application calls the ITTSCentral::AudioResume member function on the engine, the engine resumes playback and calls the IAudio::Start member function on the audio-destination object.


Synchronizing Notifications

Engines need to provide accurate synchronization for some of the notifications sent to the ITTSBufNotifySink interface. To do this:

1. Allocate memory to store the notification information. Copy the notification information into the memory.

2. While the digital-audio output is passed to the IAudioDest::DataSet member function, call the IAudioDest::BookMark member function with the pointer to the memory used as the bookmark identifier.

3. When the audio surrounding the bookmark is played, the audio-destination object calls the IAudioDestNotifySink::BookMark member function with a bookmark identifier. Convert this to a pointer to notification information.

4. Reference the notification information and call the appropriate member function of the ITTSBufNotifySink interface.

5. Free the notification information.


Supporting Multiple Languages

To support multiple languages, an engine should:

· Provide one or more text-to-speech modes for each language, each with its own GUID. If the engine can support multiple languages at one time, all of the modes should have the same engine GUID.

· Load the modes for a language when the application uses that particular language.

· Do not provide a mode for a language if the application's setup program does not install a mode for that language.

In some cases, one or more applications will need to speak to the same audio destination in several languages. This is especially true in multilingual countries or countries that do not have localized versions of all products.


Suggested Waveform-Audio Formats to Support

It is strongly recommended that a text-to-speech engine support the following waveform-audio formats:

· 11 kHz, 8-bit mono PCM. Every sound card can handle this waveform-audio format.

· 11 kHz, 16-bit mono PCM. Most sound cards can handle this waveform-audio format.

· 8 kHz, 16-bit mono PCM. Telephony cards can accept this format.

An engine may also support the following waveform-audio formats:

· 16 kHz or 22 kHz, 8/16-bit mono PCM. The higher fidelity improves sound quality.

· Custom formats. Supporting one or more custom formats allows an engine to take advantage of DSP hardware that have waveform-audio drivers that support the custom formats.


Managing User Preferences

One system user should not be able to affect the preferences, lexicon, and other settings of another user. This is especially true for Windows NT.

The text-to-speech engine must handle all user preferences that cannot be set by the standard API. To do this, store registry values under the HKEY_CURRENT_USER key.


Using a DSP to Process Information

Microsoft highly recommends that an engine be capable of speaking text without requiring any assisted hardware, but be capable of taking advantage of such hardware if it is present to reduce the engine's processing requirements or to increase playback accuracy.

A text-to-speech engine can use a DSP in any of the following ways, singly or in combination:

1. The DSP board has a wave-out driver written for it, supporting a custom waveform-audio format that represents the normal output to the DSP (filter coefficients and pitch information, for example). All engines have a "dialog" with the IAudio::WaveFormatSet member function when they start up, usually going through different PCM types, such as different sampling rates and bits per sample. The engine could also query to see if the waveform-audio device supports the "custom" format (such as filter coefficients) and use that data format if the device supports it. This may not provide enough of a communication path between the DSP and engine, and it requires that the wave-out driver be written by the engine vendor.

2. The DSP board has a wave-out driver written for it. The engine can query the IAudioMultiMediaDevice::DeviceNumGet member function to determine the waveform-audio device to which it is connected. If the device number of that waveform-audio device is known to be a friendly DSP board, the engine can start a conversation with the DSP directly, or go through the wave-out custom messages. The disadvantage of this is that it requires the wave-out driver be written by the engine vendor.

3. The DSP board provides a function that takes in filter-coefficient and pitch data (for example) and asynchronously outputs PCM data. The disadvantage of this is that a lot of data passes back-and-forth across the bus, but it allows the DSP to be used independent of the destination of the audio.

4. You provide a custom audio-destination object that communicates directly with the PC. This allows maximum flexibility, but it prevents the engine from being used by applications not specifically written for the engine.


Supporting Text-to-Speech Over the Telephone

A text-to-speech engine should support output over the telephone line as well as to a wave-out audio device connected to the computer. Here are a few suggestions:

· Supply at least one text-to-speech mode that supports an 8-kHz sampling rate.

· Support multiple instances of the engine while sending output to different destinations so that text-to-speech can occur over the PC and the telephone line at the same time or several telephone lines can be connected to the same machine.

· Set the flag in the TTSMODEINFO structure that indicates that the mode is designed for the telephone.


Supporting Scalable Platforms

It is highly recommended that the text-to-speech engine be scalable so that it can work with different processor speeds and memory configurations. It not only broadens the engine's market, but it provides experienced users with the ability to trade off processor consumption with sound quality.

When an engine is first installed or loaded, it should analyze the machine and make certain tradeoffs for optimal performance. To run effectively on the slower machines, an engine should accept lower voice quality. To take advantage of faster machines, the engine should:

· Use threads so the engine runs on SMP machines with Windows NT.

· Take advantage of a DSP if one is available.


Reducing Clicks

If a machine is not fast enough to handle text-to-speech conversion, or if it becomes overburdened, the text-to-speech engine may fail to produce digital-audio data quickly enough. The following techniques can be used to resolve this problem:

· Maintain an internal cache of up to 1 second of digital audio, making notes where a word or a phrase breaks. Do not send digital-audio buffers until enough cached data exists for a whole word or phrase. If the engine falls behind, the pauses will occur at word or phrase breaks where they will be less noticeable.

· If the engine is running on a machine that is too slow, incorporate operations that reduce computation time at the expense of quality.


Connecting Directly to Audio Hardware

An existing engine may connect directly to audio hardware, usually to a board with a hardware assist. If necessary, an engine can connect directly to the hardware and still be ported to the speech API. Be aware, however, that the engine will work only with applications specifically designed to cooperate with that engine, which limits the usefulness of the engine.

To enable an engine to connect directly to audio hardware:

1. Create a custom OLE interface — for example, ICustom — that provides direct control of the hardware.

2. Provide a a custom OLE object — for example, "custom audio destination" — that supports the ICustom interface.

To enable an application to play audio directly over the hardware:

1. Create an instance of the custom audio-destination object and pass the instance to the engine.

2. When the application passes a pointer to an IUnknown interface to the engine, the engine calls the IUnknown::QueryInterface member function for the ICustom interface.

3. If the ICustom interface is found, the engine uses ICustom to control and configure the hardware.

4. If the ICustom interface is not found, the engine uses the digital-audio data supplied by the IAudio and IAudioDest interfaces.


Defining Additional Interfaces

An engine can provide additional functionality to an application by supporting custom interfaces. These can be used to expose features not yet supported by the text-to-speech API. To do this:

· Define an interface that follows the OLE COM and that contains all of the functions necessary to support the additional functionality.

· Provide marshaling for the interface so the engine can run in a different process than the calling application.

· Enable the engine to return a pointer to the interface when the engine receives a call of the IUnknown::QueryInterface member function for that interface.

· Provide documentation for the interface so that application developers can take advantage of it.

The main thread of any engine and audio object runs at the priority of the thread that created the engine or object. Following are four general rules that an engine should follow with regards to thread priorities:

1. An engine should not adjust the priority of the thread that created the engine.

2. A multithreaded engine may run one or more threads at a higher priority than its main thread.

3. An engine may check the priority of the calling thread for hints about what priority to use for the threads that the engine creates.

4. An engine should avoid boosting priorities except under the circumstances described in documentation included with the Microsoft Win32 Software Development Kit (SDK).


Distributing Processing Over a Network

An engine might want to do some of its processing on a different machine than the one the application is running on. While the Microsoft Speech API provides no official way of implementing the distributed processing, it's fairly easy for an engine to distribute processing.

To distribute processing, split the engine into two parts.

The first part is basically a stub that takes all of the calls to the speech API and marshals the data over the network, to the second part. The server portion will perform the synthesis and send digital audio data back to the stub. The engine stub supports all of the interfaces that the full engine supports, although it does almost none of the processing. In order to reduce network usage, the stub might accept a "compressed" audio format from the server and decompress on the host.

The second part of the engine lies on a server. It accepts the marshaled data from the stub and then processes it, sending back notifications to the stub as digital audio is synthesized.

The engine might also support a custom interface, IMyDistributed. The IMyDistributed interface allows an application that knows about remote processing to control which machine the processing is on, and to finely control what kind of data is transmitted over the LAN (or other communication method) so that unneeded data (such as mouth synchronization information) isn't transmitted.


Getting a Fake Engine from Microsoft

Developers that are converting their engine to the Microsoft Speech API can write to MSSpeech@Microsoft.Com for the source code to a "fake" text-to-speech engine that works with the API.


Debugging the Engine in Applications that Use Voice Text

To debug the engine in applications using voice text:

Open Visual C (or whatever debugging environment you're using) with the engine DLL.

Set the application to run as "$(WINDOWS)\Speech\VCMD.EXE -Embedding". $(WINDOWS) should be replaced with your windows directory.

Set a break-point in the engine's class factory so that you know that the debugging is working.

Start the debugging session. This will run VCMD.EXE. The debugger will be running although the engine won't be loaded yet.

Run the application that uses Voice Text.

Your engine's breakpoint should bit hit.


Low-Level API Calls used by the Text-to-speech Enumerator

The text-to-speech recognition enumerator uses the following interfaces and functions of the low-level API.

ITTSEnumW

Next

Skip

Reset

Clone

Select


Low-Level API Calls used by Voice Text

The high-level Voice Text API works with any text-to-speech engine. It uses the following interfaces and functions of the low-level API

ITTSAttributesW

PitchGet/Set

SpeechGet/Set

VolumeGet/Set

ITTSBufNotifySinkW

BookMark

TextDataDone

TextDataStarted

AudioPause

AudioReset

AudioResume

ModeGet

Register

TextData - Tagged text. The only required tag is \mrk=\

UnRegister

ITTSDialogsW

AboutDlg

GeneralDlg

LexiconDlg

TranslateDlg

ITTSEnumW

Clone

Next

Reset

Select

Skip

ITTSNotifySinkW

AttribChanged

AudioStart

AudioStop

Visual


Topic Not Found

No matching topic was found. Some possible reasons and remedies are as follows:

  • The file (.chm) that contains the destination topic is not in the expected location.

    • Remedy: Rerun the MSDN Library Setup.

  • The file (.chm) that contains the current topic or destination topic is out-of-date and the link is no longer valid.

    • Remedy: Install the latest version of the MSDN Library.

  • The MSDN Library CD is not in the CD-ROM drive.

    • Remedy: Insert the MSDN Library CD into the CD-ROM drive.