| 2 | | = '''Voice Import Tools Tutorial : How to build a HMM-based voice for the MARY 4.0 (beta) platform''' = |
| 3 | | |
| 4 | | For creating HMM-based voices we use a version of the speaker dependent training scripts provided by [http://hts.sp.nitech.ac.jp/ HTS] that was adapted to the MARY 4.0 beta platform. The steps for building a HMM voice for the MARY platform can be summarised in:[[BR]] |
| 5 | | |
| 6 | | I) Checking the necessary programs and files[[BR]] |
| 7 | | II) Data preparation[[BR]] |
| 8 | | III) Training of HMM models[[BR]] |
| 9 | | IV) Adding a new HMM voice in the Mary system.[[BR]] |
| 10 | | V) Creating other voice in German (''to train a HMM voice with another speech database'').[[BR]] |
| 11 | | VI) (''NEW'') Creating other voice in a language different from German or English (US). |
| 12 | | |
| 13 | | The previous steps will be explained below creating a HMM voice using the HTS '''speaker dependent training demo''' adapted to the MARY 4.0 beta platform.[[BR]] |
| | 2 | = '''Voice Import Tools Tutorial : How to build a HMM-based voice for the MARY 4.0 platform''' = |
| | 3 | |
| | 4 | For creating HMM-based voices we use a version of the speaker dependent training scripts provided by [http://hts.sp.nitech.ac.jp/ HTS] that was adapted to the MARY 4.0 platform. The steps for building a HMM voice for the MARY platform can be summarised in:[[BR]] |
| | 5 | |
| | 6 | I) Download MARY TTS including Voice import tools[[BR]] |
| | 7 | II) Check necessary programs and files[[BR]] |
| | 8 | III) Check data: audio and text files[[BR]] |
| | 9 | IV) Run the Voice import tools [[BR]] |
| | 10 | V) Creating other voice in a language different from German or English (US). |
| 21 | | === I) Checking the necessary programs and files: === |
| 22 | | ''' |
| | 18 | == I) Download MARY TTS including Voice import tools == |
| | 19 | ''' |
| | 20 | |
| | 21 | Click on the latest MARY release [http://mary.dfki.de/download/4.0%20beta/openmary-standalone-install-4.0beta.jar MARY download] or download the file and run it with: |
| | 22 | {{{ |
| | 23 | java -jar openmary-standalone-install-4.0beta.jar |
| | 24 | }}} |
| | 25 | |
| | 26 | |
| | 27 | ''' |
| | 28 | == II) Check the necessary programs and files: == |
| | 29 | ''' |
| | 30 | |
| | 31 | To facilitate the checking and installation of the necessary external programs, once installed MARY TTS open a command line shell in your voice building directory and run: |
| | 32 | {{{ |
| | 33 | $MARY_BASE/lib/external/download_install_external_programs.sh |
| | 34 | }}} |
| | 35 | |
| | 36 | With the option '''-check''', this script will check if the necessary programs and versions are installed (that is, the programs can be found in the PATH or in the paths provided by the user); with the option '''-install''' this script will try to download and install the necessary programs in: $MARY_TTS/lib/external/bin (if problems, it will suggest how to install manually the programs). |
| | 37 | |
| | 38 | If you have already installed some of the required programs, you can provide the paths (if they are not in the PATH), for example: |
| | 39 | {{{ |
| | 40 | $MARY_BASE/lib/external/download_install_external_programs.sh -check /your/path/to/htk/bin /your/path/to/Festival/festvox/src/ehmm/bin |
| | 41 | }}} |
| | 42 | |
| | 43 | The necessary programs that this script checks are:[[BR]] |
| 48 | | The HTS demo for MARY 4.0 beta, includes a shell script "check_programs.sh" that will help you to check if all the previous programs are installed. |
| 49 | | |
| 50 | | |
| 51 | | ''' |
| 52 | | === II) Data preparation (): === |
| 53 | | |
| 54 | | ''' |
| 55 | | |
| 56 | | Where to start? There are three options a, b and c: |
| 57 | | |
| 58 | | '''a-''' If you would like to try the HTS-demo_CMU-ARCTIC-SLT for MARY 4.0 beta from scratch:[[BR]] |
| 59 | | Download the [http://mary.dfki.de/download/4.0%20beta/HTS-demo_CMU-ARCTIC-SLT_for_MARY-4.0-beta.tar.gz HTS-demo_CMU-ARCTIC-SLT_for_MARY-4.0-beta] (92MB), unpack the file and go to that directory: |
| 60 | | {{{ |
| 61 | | tar -zxvf HTS-demo_CMU-ARCTIC-SLT_for_MARY-4.0-beta.tar.gz |
| 62 | | cd HTS-demo_CMU-ARCTIC-SLT_for_MARY-4.0-beta |
| 63 | | }}} |
| 64 | | |
| 65 | | '''b-''' If you have already created a unit selection voice for MARY, with the CMU-ARCTIC-SLT data, and want to build a HMM-based voice for that, |
| 66 | | copy the $MARY_BASE/lib/hts/HTS-demo_for_MARY-4.0-beta.tar.gz (112K) in your unit selection voice creation directory and unpack the file: |
| 67 | | {{{ |
| 68 | | tar -zxvf HTS-demo_for_MARY-4.0-beta.tar.gz |
| 69 | | }}} |
| 70 | | If you have already created a unit selection voice for this data, most probably you have already created phonefeatures, phonelab and a mary/features.txt file for that, so you can run steps 1-3, skip steps 4-11 and continue with section III HMM models training. |
| 71 | | |
| 72 | | '''c-''' If you want to create a HMM voice in other language please see the section V or VI below.[[BR]] |
| 73 | | |
| 74 | | |
| 75 | | Once you have unpacked the HTS demo for MARY 4.0 beta, follow the steps: |
| 76 | | |
| 77 | | '''1-''' Check if all the required programs (and versions) are available in your system, you can run the shell script: |
| 78 | | {{{ |
| 79 | | ./check_programs.sh |
| 80 | | }}} |
| 81 | | This is a simple shell script that will check which programs are available in the PATH and report what is missing. You can provide the paths where you have installed the required programs if they are not found in the PATH. The script will check minimal requirements for programs, versions, suggest how to install missing programs, etc. |
| 82 | | If all the necessary programs are installed correctly you can continue with step 2. |
| 83 | | |
| 84 | | '''2-''' Run the Voice Import Tools program |
| 85 | | |
| 86 | | The Voice Import Tools programs can be started from: '''Applications -> OpenMary -> Voice import tools'''[[BR]] |
| 87 | | |
| 88 | | When starting the voice import tools, go to your working directory (the directory where you have unpacked the HTS demo for MARY 4.0 beta) and provide information for: |
| 89 | | {{{ |
| 90 | | db.gender = female |
| 91 | | db.locale = en_US |
| 92 | | db.marybase = /path/to/$MARY_BASE/ |
| 93 | | db.voicename = slt-hsmm |
| 94 | | }}} |
| 95 | | If you are not familiar or have problems with the Voice Import Tools program, please read the instructions in the Tutorial: [http://mary.opendfki.de/wiki/VoiceImportToolsTutorial VoiceImportToolsTutorial] |
| 96 | | |
| 97 | | Please remember that whenever you are in doubt about the settings of a particular component you can check its corresponding help for a description of the meaning |
| 98 | | (and possible values) of each variable. |
| 99 | | |
| 100 | | After starting the Voice Import Tools check the global settings of the voice, make sure that the allophones file is provided and exist: |
| | 67 | |
| | 68 | |
| | 69 | |
| | 70 | ''' |
| | 71 | == III) Check data: audio and text files[[BR]] == |
| | 72 | ''' |
| | 73 | |
| | 74 | In your voice building directory execute the step-by-step procedure in [http://mary.opendfki.de/wiki/VoiceImportToolsTutorial VoiceImportToolsTutorial] to make |
| | 75 | sure that the data, sound (wav) and text files are in the correct place and format.[[BR]] |
| | 76 | |
| | 77 | As a result of this step your voice building directory should contain a wav and text directories. |
| | 78 | |
| | 79 | |
| | 80 | ''' |
| | 81 | == IV) Run the Voice Import tools == |
| | 82 | ''' |
| | 83 | |
| | 84 | In your voice building directory run the voice import tools: |
| | 85 | {{{ |
| | 86 | export MARY_BASE="/your/path/to/MARY TTS/" |
| | 87 | java -Xmx1024m -jar $MARY_BASE/java/voiceimport.jar |
| | 88 | }}} |
| | 89 | |
| | 90 | After starting the Voice Import Tools check the global settings of the voice, make sure that the allophones file is provided and exists: |
| 105 | | '''3-''' Run the HMMVoiceDataPreparation of the HMM Voice Trainer group to check if text, wav or raw files are available and in the correct paths. |
| 106 | | If just raw is provided, the program will do the conversion. If no text files are available but utts in festival format, the program will do the conversion as well. |
| 107 | | |
| 108 | | '''4-''' Run the AllophonesExtractor of the Automatic Labeling group to create the '''prompt_allophones''' directory required in the next step. This component requires the MARY server. [[BR]] |
| 109 | | |
| 110 | | '''5-''' Run the EHMMlabeler component of the Automatic Labeling group to label automatically the wav files using the corresponding transcriptions. This procedure might |
| | 95 | |
| | 96 | And run the following components: |
| | 97 | |
| | 98 | |
| | 99 | '''1-''' Run the HMMVoiceDataPreparation of the HMM Voice Trainer group to set up the environment to create a HMM voice and check if required external programs and text and wav files are available and in the correct paths. |
| | 100 | |
| | 101 | '''2-''' Run the AllophonesExtractor of the Automatic Labeling group to create the '''prompt_allophones''' directory required in the next step. This component requires the MARY server. [[BR]] |
| | 102 | |
| | 103 | '''3-''' Run the EHMMlabeler component of the Automatic Labeling group to label automatically the wav files using the corresponding transcriptions. This procedure might |
| 117 | | '''6-''' Run the LabelPauseDeleter component of the Automatic Labeling group. Please use the settings editor of this component to set the variable: |
| 118 | | {{{ |
| 119 | | LabelPauseDeleter.threshold = 10 |
| 120 | | }}} |
| 121 | | |
| 122 | | '''7-''' Run the TranscriptionAligner component of the Label-Transcript Alignment group. This program will create the '''allophones''' directory. |
| 123 | | |
| 124 | | '''8-''' Run the PhoneUnitLabelComputer component of the Label-Transcript Alignment group. This procedure has as input the '''lab''' directory created with the EHMMLabeler and will create as an output the '''phonelab''' directory. |
| 125 | | |
| 126 | | '''9-''' Run the FeatureSelelection component of the Feature Extraction group. This program will create a '''mary/features.txt''' file, it requires the MARY server running. Select here all the features and save the file. |
| 127 | | |
| 128 | | '''10-''' Run the PhoneUnitFeatureComputer component of the Feature Extraction group to extract context feature vectors from the text data. This procedure will create a '''phonefeatures''' directory. For running this component the MARY server should be running as well. |
| 129 | | |
| 130 | | '''11-''' Run the PhonelabelFeatureAligner component of the Verify Alignment group. This procedure will verify alignment between "phonefeatures" and "phonelabels".[[BR]] |
| | 116 | '''5-''' Run the TranscriptionAligner component of the Label-Transcript Alignment group. This program will create the '''allophones''' directory. |
| | 117 | |
| | 118 | '''6-''' Run the PhoneUnitLabelComputer component of the Label-Transcript Alignment group. This procedure has as input the '''lab''' directory and will create as an output the '''phonelab''' directory. |
| | 119 | |
| | 120 | '''7-''' Run the FeatureSelelection component of the Feature Extraction group. This program will create a '''mary/features.txt''' file, it requires the MARY server running. Select here all the features and save the file. |
| | 121 | |
| | 122 | '''8-''' Run the PhoneUnitFeatureComputer component of the Feature Extraction group to extract context feature vectors from the text data. This procedure will create a '''phonefeatures''' directory. For running this component the MARY server should be running as well. |
| | 123 | |
| | 124 | '''9-''' Run the PhonelabelFeatureAligner component of the Verify Alignment group. This procedure will verify alignment between "phonefeatures" and "phonelabels".[[BR]] |
| 136 | | |
| 137 | | |
| 138 | | ''' |
| 139 | | === III) HMM models training: === |
| 140 | | ''' |
| 141 | | |
| 142 | | '''12-''' Run the HMMVoiceConfigure component of the HMM Voice trainer group. The default setting values of this component are already fixed for the HTS-demo_CMU-ARCTIC-SLT voice, although some setting depends on your installation, please provide paths for: |
| 143 | | {{{ |
| 144 | | HMMVoiceConfigure.htsPath = /yourpath/htk-hts2.1/bin |
| 145 | | HMMVoiceConfigure.htsEnginePath = /yourpath/hts_engine_API-1.01/bin |
| 146 | | HMMVoiceConfigure.sptkPath = /yourpath/SPTK-3.2/bin |
| 147 | | HMMVoiceConfigure.tclPath = /yourpath/ActiveTcl-8.6/bin |
| 148 | | HMMVoiceConfigure.soxPath = /yourpath/usr/bin |
| 149 | | }}} |
| | 130 | - MARY_BASE/external/externalPaths.txt |
| | 131 | |
| | 132 | |
| | 133 | ''' |
| | 134 | === HMM models training: === |
| | 135 | ''' |
| | 136 | |
| | 137 | '''10-''' Run the HMMVoiceConfigure component of the HMM Voice trainer group. The default setting values ar already fixed for the arctic slt voice, some setting depends on your installation, and willbe taken from MARY_BASE/external/externalPaths.txt |
| 242 | | === V) Creating other voice in German === |
| 243 | | ''' |
| 244 | | |
| 245 | | - If you are creating the HMM-based voice for German from scratch it will be necessary: [[BR]] |
| 246 | | * NLP components for German, those should be available with MARY 4.0 |
| 247 | | * a wav or raw directory with the speech files you will use for training the German voice. [[BR]] |
| 248 | | * transcriptions of the files, one text file per speech file, or transcriptions in festival format if available. [[BR]] |
| 249 | | then copy the $MARY_BASE/lib/hts/HTS-demo_for_MARY-4.0-beta.tar.gz file in the directory where you have your wav and transcription data and unpack the file: |
| 250 | | {{{ |
| 251 | | tar -zxvf HTS-demo_for_MARY-4.0-beta.tar.gz |
| 252 | | }}} |
| 253 | | |
| 254 | | Once you have unpacked the HTS demo for MARY 4.0 beta, follow the instructions as normal from step 1. Provide general settings for: |
| 255 | | {{{ |
| 256 | | db.gender = male (or female) |
| 257 | | db.locale = de |
| 258 | | db.marybase = /path/to/mary/base/ |
| 259 | | db.voicename = german_voice |
| 260 | | }}} |
| 261 | | |
| 262 | | |
| 263 | | - If you have already created a German unit selection voice for MARY and want to build a HMM-based voice for that, copy the $MARY_BASE/lib/hts/HTS-demo_for_MARY-4.0-beta.tar.gz (112K) in your unit selection voice creation directory and unpack the file: |
| 264 | | {{{ |
| 265 | | tar -zxvf HTS-demo_for_MARY-4.0-beta.tar.gz |
| 266 | | }}} |
| 267 | | |
| 268 | | If you have already created a unit selection voice for German, most probably you have already created phonefeatures, phonelab and a mary/features.txt file for that, so you can run steps 1-3, skip steps 4-11 and continue with section III HMM models training. |
| 269 | | |
| 270 | | |
| 271 | | ''' |
| 272 | | === VI) Creating other voice in a language different from German or English (US). === |
| | 230 | === V) Creating other voice in a language different from German or English (US). === |
