Context Navigation

Changes between Version 1 and Version 2 of HMMVoiceCreationAdapt

Timestamp:: 05/09/08 14:53:26 (17 years ago)
Author:: marcela_charfuelan
Comment:: --

Legend:

: Unmodified
: Added
: Removed
: Modified

HMMVoiceCreationAdapt

-                      v1
+                      v2
 = '''(Draft) Voice Import Tools Tutorial : How to build an adapted HMM-based voice for the MARY platform''' =
+= '''Voice Import Tools Tutorial : How to build an adapted HMM-based voice for the MARY platform''' =
 An adapted HMM-based voice is a voice created after adapting the (generally) small corpus of a particular voice to another voice that has been trained with
 …
 .1) download and un-zip, un-tar the latest speaker adaptation/adaptive training demo for English.
 Here it is used: http://hts.sp.nitech.ac.jp/archives/2.0.1/HTS-demo_CMU-ARCTIC-ADAPT.tar.bz2 for HTS-2.0.1
+http://hts.sp.nitech.ac.jp/archives/2.0.1/HTS-demo_CMU-ARCTIC-ADAPT.tar.bz2 for HTS-2.0.1
 .2) download and unzip the adaptation/adaptive patch file for using MARY instead of Festival as text analyser.
 …
 https://mary.opendfki.de/repos/trunk/lib/hts/HTS-2.0.1-demo_CMU-ARCTIC-ADAPT_for_Mary-3.6.patch.zip  [[BR]]
+apply the patch to the HTS-demo_CMU-ARCTIC-ADAPT directory:  [[BR]]
+apply the patch to the HTS-demo_CMU-ARCTIC-ADAPT directory:
+{{{
    patch -p1 -d . < HTS-2.0.1-demo_CMU-ARCTIC-ADAPT_for_Mary-3.6.patch
+}}}
 .3) create a wav directory.
 …
 .4) Run the VoiceImport program
+First of all you need to set your MARY_BASE directory:  [[BR]]
+First of all you need to set your MARY_BASE directory and then run the program:
+{{{
    export MARY_BASE="/dir/to/openmary"
-then you can run:  [[BR]]
    java -jar -Xmx1024m  $MARY_BASE/java/voiceimport.jar
+}}}
 If you are not familiar or have problems with the VoiceImport program, please read and follow the instructions in the Voice Import Tools
 …
 If you want to create another adapted voice in German or English please see the section V below.
+Please remember that whenever you are in doubt about the settings of a particular component you can check its corresponding help for a description of the meaning (and possible values) of each variable.
 '''
 …
 HTS-demo_CMU-ARCTIC-ADAPT voice.
 IMPORTANT: the names of the files should contain a label that identifies the data set and a label that identifies the voice.
+IMPORTANT: the names of the files contain a label that identifies the data set (cmu_us_arctic) and another label that identifies the voice (awb).
 This is important because the training scripts require a mask to differentiate the data from one user to another.
+The spkrMask label should reflect the data set and the voice name labels. For example for the CMU_US_ARTIC data the settings are:
+Another important configuration setting is the f0Ranges, that is the set of F0 ranges for all the voices. The format of this setting is:[[BR]]
+spkr1 lowerF01 upperF01  spkr2 lowerF02 upperF02  ... . The voice order of appearance is first the trainSpkr names and then the adaptSpkr names.
+For example the spkrMask and f0Ranges settings for the CMU_US_ARTIC data are:
 {{{
 HMMVoiceConfigureAdapt.dataSet     = cmu_us_arctic
 HMMVoiceConfigureAdapt.trainSpkr   = awb bdl clb jmk rms
+HMMVoiceConfigureAdapt.trainSpkr   = 'awb bdl clb jmk rms'   (please use quotes if there is more than one name)
 HMMVoiceConfigureAdapt.adaptSpkr   = slt
 HMMVoiceConfigureAdapt.spkrMask    = */cmu_us_arctic_%%%_*
                                     (here the voice name is exactly 3 letters, so all the voice names should be 3 letters long)
+}}}
+The file names for CMU_US_ARCTIC have the fromat:
+HMMVoiceConfigureAdapt.f0Ranges    = 'awb 40 280  bdl 40 280  clb 80 350  jmk 40 280  rms 40 280  slt 80 350'
+                                     (please leave two spaces after each set)
+}}}
+The file names for CMU_US_ARCTIC have the format:
 {{{
 awb --> cmu_us_arctic_awb_*.*
 …
 slt --> cmu_us_arctic_slt_*.*
 }}}
 Using the setting of this component you can also change other variables like using LSP instead og MGC, sampling frequency, etc.,
 the same as you would do when running "make configure" with the original HTS scripts.
+- Run HMMVoiceMakeData component of the HMM Voice trainer group to run the HTS procedure "make data". This procedure is the same as in the original scripts with additional sections for calculating strenghts (for mixed exitation), global variance, and handling of MARY context features.
+Particular procedures can be repeated isolated, fixing the particular settings for this component. For example, if the procedure that creates strengths (str directory) has to be repeated with a different set of filters (data/filters/), please set:
+{{{
+  HMMVoiceMakeData.makeSTR       1
+  HMMVocieMakeData.makeCMPMARY   1
+}}}
+all the other variables in 0, and run again the component. (In this case you need to run as well makeCMPMARY because you need to compose again the vectors mgc+lf0+str).
+The procedures can be repeated manually as well, going to the data directory and running "make data" or "make str", as is normally done with the original HTS scripts.
+Note: the Makefile in data/ includes a gv: section copied from HTS-2.1alpha version to calculate global variance files. In MARY, these files are generated little endian and contain a header of size one short to indicate the size of the vectors it contains. In the case of adapted voices, the gv variance is calculated from the adapted corpus.
+- Run the HMMVoiceMakeData component of the HMM Voice trainer group to execute the HTS procedure "make data". This procedure is the same as in the original scripts with additional sections for calculating strengths (for mixed excitation), global variance, and handling of MARY context features.
+NOTE: the Makefile in data/ includes a gv: section copied from HTS-2.1alpha version to calculate global variance files. In MARY, these files are generated little endian and contain a header of size one short to indicate the size of the vectors it contains. In the case of adapted voices, the gv variance is calculated from the adapted corpus for each adapted voice.
 - Run the HMMVoiceMakeVoiceAdapt component of the HMM Voice trainer group, here again particular training steps can be repeated selecting them (setting in 1, all the others in 0) from the settings of this component. This is equivalent to run again:
 …
 This component will generate general information about the execution of the training steps. Detailed information about the training status can be found in the logfile in the current directory.
 The training procedure can take several hours (or days), check the log file time to time to check progress.
+The adaptive training procedure can take several hours (or days), check the log file time to time to check progress.
 …
 '''
 - Run HMMVoiceInstaller component of the Install Voice group. This step is similar to the the speaker dependent demo, but please set the appropriate directories
 for the gv and voices data, which should be in the directory of the adapted voice, for example for the adapted slt voice:
+- Run the HMMVoiceInstaller component of the Install Voice group. This step is similar to the the speaker dependent demo, but the gv and voice directories
+have to be fixed according to the adapted voice. For example for the adapted slt voice please set:
 {{{
 HMMVoiceInstaller.Fgva data/gv/slt/gv-mag-littend.pdf
 …
 }}}
+If more than one voice is adapted, this procedure should be repeated setting the appropriate directories for gv and voice.
 '''
 …
 If using German:
 For creating a new German voice it is necessary: [[BR]]
+For creating a new adapted German voice it is necessary: [[BR]]
   * a wav or raw directory with the speech files you will use for training the German voice. [[BR]]
   * transcriptions of the files, one text file per speech file, or transcriptions in festival format if available. [[BR]]
+Please be aware of the names format for the adaptive scripts. Since it is used a mask for the names it is better if the names of your files have a particular format.
+For example we experiment adapting a neutral voice to different styles with the male German PAVOQUE database. For this database the file names have the format:
+{{{
+neutr --> pavoque_neutr_*.*    training data, big corpus, male voice with neutral style.
+obadi --> pavoque_obadi_*.*    data for adaptation, small corpus, the same male voice but with depressed style.
+poppy --> pavoque_poppy_*.*    data for adaptation, small corpus, the same male voice but with happy style.
+spike --> pavoque_spike_*.*    data for adaptation, small corpus, the same male voice but with angry style.
+}}}
+Having this distribution of files, our settings for configureAdapt looked like:
+{{{
+HMMVoiceConfigureAdapt.dataSet     = pavoque
+HMMVoiceConfigureAdapt.trainSpkr   = neutr
+HMMVoiceConfigureAdapt.adaptSpkr   = 'obadi poppy spike'
+HMMVoiceConfigureAdapt.spkrMask    = */pavoque_%%%%%_*
+                                    (here the voice names are exactly 5 letters long, it can not be a voice name with more that 5 letters!)
+HMMVoiceConfigureAdapt.f0Ranges    = 'neutr 40 280  obadi 40 280  poppy 40 280  spike 40 280'
+                                     (please leave two spaces after each set)
+}}}
 Then we use as a base the original HTS-demo_CMU-ARCTIC-SLT directory:
 …
 - Move your transcription files to this directory, if you have a text directory containing the transcription of each file in separate files, this should be copied in the current directory (german_voice/text). If you have transcriptions in festival format please copy this directory in the data/utts directory (german_voice/data/utts/).
+- Please be aware of the names format for the adaptive scripts. Since it is used a mask for the names it is bettr if the names of your files have a particular format.
+For example for our PAVOQUE database the file names have the format:
+{{{
+neutr --> pavoque_neutr_*.*    training data, big corpus, male voice and sound effect neutral.
+obadi --> pavoque_obadi_*.*    adapted voice, small corpus, the same male voice but sound effect depressed.
+poppy --> pavoque_poppy_*.*    adapted voice, small corpus, the same male voice but sound effect happy.
+spike --> pavoque_spike_*.*    adapted voice, small corpus, the same male voice but sound effect angry.
+}}}
+Having this distribution of files, our setting for configure will look like:
+{{{
+HMMVoiceConfigureAdapt.dataSet     = pavoque
+HMMVoiceConfigureAdapt.trainSpkr   = neutr
+HMMVoiceConfigureAdapt.adaptSpkr   = obadi poppy spike
+HMMVoiceConfigureAdapt.spkrMask    = */pavoque_%%%%%_*
+                                    (here the voice names are exactly 5 letters long, it can not be a voice name with more that 5 letters!)
+}}}
+- Now run the VoiceImport program and follow the instructions as normal. Provide settings for locale must be "de", path to mary_base and name of the voice.
+- Now run the VoiceImport program and follow the HMMVoiceCreationAdapt instructions as normal. Provide settings for locale, if German it must be "de", path to mary_base and name of the voice.
+NOTE: In our PAVOQUE example we generate three adapted voices, so the name of the voice at the beginning (during training) can be a general one, but during installation, please set the name according to the adapted voice you are going to install.
 [[BR]]
 [[BR]]
+Marcela Charfuelan DFKI
+Thu May  8 17:26:44 CEST 2008
+Marcela Charfuelan [[BR]]
+DFKI - Fri May  9 14:35:27 CEST 2008