Niseko normalization pipeline

Description and stages

Niseko is a pipeline used for the normalization of the user’s utterance through the execution of the following normalizers:

• PunctuationNormalizer
• SplitPunctNormalizer
• SpaceNormalizer
• CurrencyNormalizer
• UnicodeNormalizer
• LowercaseNormalizer
• CardinalityNormalizer
• PunctuationNormalizer
• SpaceNormalizer
• StopWordsFromFileNormalizer
• WordReplacerFromFileNormalizer

Configuration

This stage requires the following configuration in the nlp.json configuration file:

For the specific language and channel, in the nlp field of this JSON file, the key normalizer_pipeline_class must be filled in with the value: auracog_pipelines.pipelines.normalization.niseko.NisekoPipeline

{
  "es-es": {
    "mp": {
         "nlp": {
         "normalizer_pipeline_class": "auracog_pipelines.pipelines.normalization.niseko.NisekoPipeline"
      }
    }
  }
}