Giter VIP home page Giter VIP logo

strasak's Introduction

SDL Trados Studio Automation Kit (STraSAK)

Introduction

In localization engineer's daily work, many projects use more-or-less fixed process and structure:

  • master TMs are stored in a fixed location
  • individual handoffs use fixed (sub)folders structure (location of the prepared source files is known, location where the Trados Studio project should be created is known, etc...)
  • Trados Studio projects naming follows certain pattern, e.g. {client_name}_{current_date}_{handoff_name}
  • source language is (almost) always the same... target languages are also often the same...
  • and so on...

So... it should not be necessary to enter manually all these parameters required for Trados Studio operations over and over again... some script should be able to use all this known data and do everything automatically.

And that's the purpose of this automation kit – to allow engineers to automate Trados-related operation using simple scriptable commands:

Command Description
New-Project Create new project – optionally based on project template or another project – in specified location, using specified source and target languages and TMs from specified location. Get source files from specified location and automatically convert them to translatable format and copy them to target languages. Optionally also pre-translate and analyze the files, saving results to Trados 2007-formatted log.
PseudoTranslate Pseudo-translate specified project, using specified pseudo-translation options. Optionally also export the pseudo-translated target files.
Export-Package Create translation packages from specified project, using specified package options, and save them to specified location
Import-Package Import return packages from specified location in a specified project
ConvertTo-TradosLog Convert Studio XML-based report to Trados 2007-formatted log
Export-TargetFiles Export target files from specified project to specified location
Update-MainTMs Update main translation memories of specified project
New-FileBasedTM Create new translation memory in specified location, using specified options
Export-TMX Export one or more Trados Studio translation memories to TMX, optionally applying a filter
Import-TMX Import content from TMX file in a specified TM, optionally applying a filter

Technical info

Automation is based on SDL PowerShell Toolkit (https://github.com/sdl/Sdl-studio-powershell-toolkit) with own extensive enhancements and customizations. The kit consists of:

  • PowerShell modules containing the actual automation
  • Windows batch wrapper/launcher script for easy invocation of the PowerShell functions from command line
  • PDF manual with detailed description of functions, their parameters and usage examples

Installation and setup

  1. Pre-requisite: Windows PowerShell 4.0 or newer installed
    This may be required only for Windows 7 and 8.
    Windows 8.1 has PowerShell 4.0 already built-in, Windows 10 has PowerShell 5.0 already built-in.
    Download Powershell 4.0: https://www.microsoft.com/en-us/download/details.aspx?id=40855
    Download Powershell 5.1: https://www.microsoft.com/en-us/download/details.aspx?id=54616
  2. Create WindowsPowerShell subfolder in your Documents folder
    (i.e. the result will be C:\Users\<YourProfile>\Documents\WindowsPowerShell)
    NOTE: If you moved your Documents folder to another location, create the subfolder in that location.
  3. Copy the entire Modules folder (including the folder structure) into the created WindowsPowerShell folder.
  4. Put the TS2015.cmd (TS2017.cmd, TS2019.cmd) wrapper script to any preferred location and add the location to your PATH environment variable, so that you can run the script without specifying its full path.
    See https://www.java.com/en/download/help/path.xml for more information about PATH variable and how to edit its content in different operating systems.
    Note: Make sure to add the location to the existing value of PATH (separated by semicolon)... do not replace the existing value, otherwise some applications or your system may stop working properly!
    Optionally you can put the wrapper script to a location which is already listed in the PATH variable (e.g. C:\WINDOWS)... but that may be uncomfortable, depeding on particular system setup, etc.

That's all... the kit is now ready for use!

Usage

In Windows batch script

Call the wrapper script with desired action command and its parameters as command line arguments:
call TS2017 New-Project -Name "My project" -Location "D:\My project" ...

If the wrapper script's location is not listed in PATH environment variable, you need to use full path to script:
call "C:\My scripts\TS2017.cmd" New-Project -Name "My project" -Location "D:\My project" ...

In Powershell script

Call the desired PowerShell function with corresponding parameters directly from your PowerShell script:
New-Project -Name "My project" -Location "D:\Projects\My project" ...

In other scripting languages

Use the language's appropriate method to call either the batch wrapper, or the PowerShell function.

Usage Examples

Here is a few Windows batch scripts as examples of automation implementation. Target languages list for "CreateProject", "ExportPackages" and "ExportFiles" scripts is passed as script parameter, e.g.:
03_CreateProject.cmd "de-DE fr-FR it-IT" (language codes can be separated by space, comma or pipe)

== 03_CreateProject.cmd ==

@echo off
set TARGETLANGUAGES=%~1
for %%D in ("%CD%") do set "PROJECTNAME=%%~nxD"
call TS2017 New-Project ^
     -Name "%PROJECTNAME%" ^
     -SourceLocation "02_Prep" ^
     -ProjectLocation "03_Studio" ^
     -LogLocation "04_ForTrans" ^
     -TargetLanguages "%TARGETLANGUAGES%" ^
     -TMLocation "X:\Projects\My Project\_TMs" ^
     -ProjectTemplate "X:\Projects\My Project\_Template\MyProject.sdltpl" ^
     -Pretranslate -Analyze

== 04_ExportPackages.cmd ==

@echo off
set TARGETLANGUAGES=%~1
call TS2017 Export-Package ^
     -ProjectLocation "03_Studio" ^
     -PackageLocation "04_ForTrans" ^
     -TargetLanguages "%TARGETLANGUAGES%" ^
     -IncludeMainTMs -IncludeTermbases

== 05_ImportPackages.cmd ==

@echo off
call TS2017 Import-Package ^
     -ProjectLocation "03_Studio" ^
     -PackageLocation "05_FromTrans"

== 06_ExportFiles.cmd ==

@echo off
set TARGETLANGUAGES=%~1
call TS2017 Export-TargetFiles ^
     -ProjectLocation "03_Studio" ^
     -ExportLocation "06_Post" ^
     -TargetLanguages "%TARGETLANGUAGES%"

Known issues

"log4net:ERROR: XmlConfigurator..." message displayed each time automation is started

This is Trados Studio API bug. It's just a cosmetic issue and does not influence automation functionality.

Out Of Memory error during analysis or package creation

This seems to be caused by some weird memory leak in Studio API if a huge TM is used for analysis or for creating Project TM (either during project creation, or when a "Create new TM" package creation option is used), which may cause Out Of Memory exception.

As a workaround you can try the -PerLanguage parameter of the New-Project function. Or you can try to use smaller/less TMs. Or both...

Automation fails if folder- or file name contains square brackets

This is PowerShell bug and there is currently no reasonable workaround... except for removing there characters from folder/file names.

You can vote for fixing this bug at following links:
https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/15976273-powershell-fails-to-start-if-path-contains-bracket

https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/11088702-start-process-doesn-t-work-if-in-a-directory-name

Version history

STraSAK v1.8       (2021-01-13)

  • Added support for Studio 2021

STraSAK v1.7       (2019-10-11)

  • Improved detection of default project template (it works now correctly with Studio 2019 SR1 and newer, which broke the way it worked reliably for years :( )
  • Import-TMX command now displays separate progress information for each import phase

STraSAK v1.6       (2019-09-15)

  • Import-TMX and Export-TMX commands can now apply filter during import/export
  • fixed Multiple ambiguous overloads found errors and no progress information displayed during TMX import/export with Studio 2017 SR1 CU13 and newer
  • progress information is now displayed also for Pre-translate Files and Analyze Files automatic tasks
  • New-FilebasedTM now optionally returns created TM object(s) using -PassThru switch
  • TM helper functions (Get-TM...) now accept also TM object as input, apart from TM file
  • internal refactoring

STraSAK v1.5       (2019-07-03)

  • New-FileBasedTM command can now create multiple TMs at once

STraSAK v1.4       (2018-06-29)

  • Added support for Studio 2019

STraSAK v1.3       (2018-02-04)

  • New-Project: fixed v1.2 bug causing source language code missing in created Trados log file names (e.g. Analyze Files de-DE.log instead of Analyze Files en-US_de-DE.log)

STraSAK v1.2       (2018-01-29)

  • Export-Package: fixed bug causing Project TM (and analysis report) not being included in created translation package when "Create new project TM" option was used
  • Export-Package: added Recompute and IncludeExisting aliases for RecomputeAnalysis and IncludeExistingReports parameters
  • Export-TargetFiles: added Export alias for ExportLocation parameter, to be consistent with pseudotranslation parameters

STraSAK v1.1       (2018-01-17)

  • Added PseudoTranslate command

STraSAK v1.0 FINAL       (2017-12-29)

  • added New-Project command features:
    • can also apply PerfectMatch during project creation (with automatic matching of bilingual files)
    • can also save analysis logs to Excel format
    • can run in special "per-language mode" when tasks (pretranslation, analysis, etc.) are run separately for each language – see more info in manual
  • added/changed ConvertTo-TradosLog command features:
    • can now convert all logs in folder (incl. subfolders, if needed)
    • can now specify output file name and location when converting single file
  • new Update-MainTMs command which does exactly what it says ;-)
  • new New-FileBasedTM command for quick creation of empty TMs
  • new Import-TMX command for quick TMX import to TM
  • new Export-TMX command for quick TMX export
    • can export all TMs in folder, incl. subfolders if needed
  • changed Import-Package command feature
    • now does not search input folder recursively by default
    • separate -Recurse switch added to search for packages also in subfolders
  • improved progress messages

STraSAK v1.0 beta hotfix 1       (2016-12-02)

  • internal fuzzies now correctly included in Trados 2007 textual logs

STraSAK v1.0 beta       (2016-11-21)

  • Studio project creation, including pre-translation, analysis and textual logs export
  • translation packages creation
  • return packages import
  • target files export

STraSAK Technical Preview       (2016-10-10)

Basic project creation functionality

strasak's People

Contributors

cromica avatar evzenp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

strasak's Issues

Support for Studio 21

Hi there, I am using the new Studio 21 trial version but I get a ton of errors when using the scripts of the repo. Do you plan on releasing support for this latest update? Thanks!

Can't use Machine Translation

Hi,

All functions of STraSAK are working perfectly fine, but when it comes to using Machine Translation, I get the following error:

Microsoft Windows [Version 10.0.18363.719]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Windows\system32>Call "C:\My Scripts\TS2017.cmd"  New-Project -Name "Sample_Project" -PrjLoc "\\Mac\Home\Desktop\Path\Project" -SrcLoc "\\Mac\Home\Desktop\Path\Source" -SrcLng "en-US" -TrgLng "nl-NL" -TMLoc "\\Mac\Home\Desktop\Path\Working TM" -LogLocation "\\Mac\Home\Desktop\Path\Log Files" -PrjTpl "\\Mac\Home\Desktop\Path\Template\Machine_Translation.sdltpl" -Pretranslate -Analyze

Creating new project...
log4net:ERROR Failed to find configuration section 'log4net' in the application's .config file. Check your .config file for the <log4net> and <configSections> elements. The configuration section should look like: <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net" />

Assigning TMs to project...
\\Mac\Home\Desktop\Path\Working TM\FileBased_TM_NL.sdltm added to project

Adding source files...
Done

Running preparation tasks...
Task Scan
   50%  Started
Information: There are multiple matching file type settings found: Graphics, XML: Any v 1.2.0.0
  100%  Completed
Task Scan successfully completed.
Task Convert to Translatable Format
  100%  Completed
Task Convert to Translatable Format successfully completed.
Task Copy to Target Languages
  100%  Completed
Task Copy to Target Languages successfully completed.
Done

Running automatic tasks...
    0%  Started
Error: Unexpected exception when initializing task 'Analyze Files': Failed to create an instance of translation provider 'modernmt:///?sm=20629&k=NntgyUTgrPStYcXvlxKGMYcw8DDbklfVqhWUB+jf+WKaICIdBBrR39KfkEfJgTmodp7jIh0/sK0E1+gsOKD81d4MQXQeIoAMF8zIeRHmfmlDP222MhYuoTIoCNYZ1EVRoJzF7ituY4/4m8jyue+Cfxe6N7Rmu49LCJ8hMm+NxBk='..
Sdl.ProjectAutomation.Core.ProjectAutomationException: Unexpected exception when initializing task 'Analyze Files': Failed to create an instance of translation provider 'modernmt:///?sm=20629&k=NntgyUTgrPStYcXvlxKGMYcw8DDbklfVqhWUB+jf+WKaICIdBBrR39KfkEfJgTmodp7jIh0/sK0E1+gsOKD81d4MQXQeIoAMF8zIeRHmfmlDP222MhYuoTIoCNYZ1EVRoJzF7ituY4/4m8jyue+Cfxe6N7Rmu49LCJ8hMm+NxBk='.. ---> Sdl.ProjectApi.ProjectApiException: Unexpected exception when initializing task 'Analyze Files': Failed to create an instance of translation provider 'modernmt:///?sm=20629&k=NntgyUTgrPStYcXvlxKGMYcw8DDbklfVqhWUB+jf+WKaICIdBBrR39KfkEfJgTmodp7jIh0/sK0E1+gsOKD81d4MQXQeIoAMF8zIeRHmfmlDP222MhYuoTIoCNYZ1EVRoJzF7ituY4/4m8jyue+Cfxe6N7Rmu49LCJ8hMm+NxBk='.. ---> Sdl.ProjectAutomation.Core.ProjectAutomationException: Failed to create an instance of translation provider 'modernmt:///?sm=20629&k=NntgyUTgrPStYcXvlxKGMYcw8DDbklfVqhWUB+jf+WKaICIdBBrR39KfkEfJgTmodp7jIh0/sK0E1+gsOKD81d4MQXQeIoAMF8zIeRHmfmlDP222MhYuoTIoCNYZ1EVRoJzF7ituY4/4m8jyue+Cfxe6N7Rmu49LCJ8hMm+NxBk='. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentNullException: Value cannot be null.
Parameter name: element
   at System.Attribute.GetCustomAttributes(Assembly element, Type attributeType, Boolean inherit)
   at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType, Boolean inherit)
   at System.Reflection.CustomAttributeExtensions.GetCustomAttribute[T](Assembly element)
   at ModernMT.Core.PluginInfo..ctor()
   at ModernMT.Plugin.ModernMTProviderFactory..ctor()
   --- End of inner exception stack trace ---
   at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& bNeedSecurityCheck)
   at System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
   at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(Type type, Boolean nonPublic)
   at System.Activator.CreateInstance(Type type)
   at Sdl.Core.PluginFramework.DefaultObjectResolver.CreateObject(Type objectType, XElement attributeElement)
   at Sdl.Core.PluginFramework.Implementation.Extension.CreateInstance()
   at Sdl.Core.PluginFramework.ObjectRegistry`2.CreateObjects()
   at Sdl.LanguagePlatform.TranslationMemoryApi.TranslationProviderManager.GetTranslationProviderFactories()
   at Sdl.LanguagePlatform.TranslationMemoryApi.TranslationProviderManager.CreateTranslationProvider(Uri translationProviderUri, String translationProviderState, ITranslationProviderCredentialStore credentialStore)
   at Sdl.ProjectApi.TranslationProviderCache.CreateTranslationProvider(Uri translationProviderUri, String translationProviderState, ITranslationProviderCredentialStore credentialStore)
   at Sdl.ProjectApi.TranslationProviderCache.GetTranslationProvider(Uri translationProviderUri, String translationProviderState, ITranslationProviderCredentialStore credentialStore, Boolean performUpdate, Boolean refreshCache)
   at Sdl.ProjectApi.Helpers.ProjectCascadeFactory.GetTranslationProviderLanguageDirection(TranslationProviderItem translationProviderItem, LanguagePair languageDirection, Boolean performUpdate, Boolean refreshCache)
   --- End of inner exception stack trace ---
   at Sdl.ProjectAutomation.FileBased.Internal.AutomationServerEvents.HandleTranslationProviderException(TranslationProviderItem translationProviderItem, Exception exception)
   at Sdl.ProjectApi.Helpers.ProjectCascadeFactory.GetTranslationProviderLanguageDirection(TranslationProviderItem translationProviderItem, LanguagePair languageDirection, Boolean performUpdate, Boolean refreshCache)
   at Sdl.ProjectApi.Helpers.ProjectCascadeFactory.CreateProjectCascadeEntries(IList`1 projectCascadeEntryDataList, LanguagePair languagePair, Boolean readOnly, Boolean refreshCache)
   at Sdl.ProjectApi.Helpers.ProjectCascadeFactory.CreateCascade(ProjectCascadeSettings projectCascadeSettings, Boolean refreshCache)
   at Sdl.ProjectApi.Helpers.LanguageDirectionHelper.CreateCascade(ILanguageDirection languageDirection, ProjectCascadeEntryDataFilterFunction filter, IComparer`1 sort, Boolean readOnly, Boolean refreshCache)
   at Sdl.ProjectApi.Helpers.LanguageObjectsCache.<>c__DisplayClass5_0.<GetRecognizers>b__0()
   at Sdl.ProjectApi.Helpers.LanguageObjectsCache.GetOrCreateObject(String key, Func`1 createObject)
   at Sdl.ProjectApi.Helpers.LanguageObjectsCache.GetRecognizers(ILanguageDirection languageDirection)
   at Sdl.ProjectApi.Helpers.LanguageObjectsCache.GetLanguageTools(ILanguageDirection languageDirection)
   at Sdl.ProjectApi.AutomaticTasks.Analysis.AnalysisDataCollectorX.CreateAnalysisDataStorage(ILanguageDirection languageDirection, AnalysisTaskSettings analysisSettings, TranslationMemorySettings translationMemorySettings, ILanguageObjectsCache objectsCache)
   at Sdl.ProjectApi.AutomaticTasks.Analysis.AnalysisDataCollectorX.Initialize(IList`1 files)
   at Sdl.ProjectApi.AutomaticTasks.Analysis.AnalysisTask.CreateAnalysisDataCollectors()
   at Sdl.ProjectApi.AutomaticTasks.Analysis.AnalysisTask.InitializeTask(IExecutingAutomaticTask task)
   at Sdl.ProjectApi.Implementation.TaskExecution.ContentProcessingTaskImplementation.InitializeTaskImplementations()
   --- End of inner exception stack trace ---
   at Sdl.ProjectApi.Implementation.TaskExecution.ContentProcessingTaskImplementation.InitializeTaskImplementations()
   at Sdl.ProjectApi.Implementation.TaskExecution.ContentProcessingTaskImplementation.Execute()
   at Sdl.ProjectApi.Implementation.AutomaticTaskExecuter.Execute()
   --- End of inner exception stack trace ---
Task Pre-translate Files+Analyze Files failed.
Unexpected exception when initializing task 'Analyze Files': Failed to create an instance of translation provider 'modernmt:///?sm=20629&k=NntgyUTgrPStYcXvlxKGMYcw8DDbklfVqhWUB+jf+WKaICIdBBrR39KfkEfJgTmodp7jIh0/sK0E1+gsOKD81d4MQXQeIoAMF8zIeRHmfmlDP222MhYuoTIoCNYZ1EVRoJzF7ituY4/4m8jyue+Cfxe6N7Rmu49LCJ8hMm+NxBk='..

Saving logs...
Done

Saving project...
Done

C:\Windows\system32>

Tried with Studio 2019, but the same.
I'm using ModernMT, but I guess it's the same for all other machine translation providers.
The strange part is that even if I try to create a project without specifying the MT, I still get the same error. Only after I uninstall the plugin I don't get this error.

Not sure if this is a plugin error as creating the same manually works fine.

Any idea how to solve this?

Thanks a lot,
Sergei

Is it possible to use this with plugins?

Hello. No issue, but a question. Can I use your toolkit in combination with a Trados Studio plugin? Specifically for the StarTransit plugin to import PPF files? This plugin creates a Ribbon button to do the import.
Thanks for your time!

Update TMs with SDLXLIFF files

New feature:
There're two good features about STraSAK:

  1. Create new file based translation memories
  2. Export the Trados Studio translation memories to TMX files

Is there something that can be added in the middle to update the created translation memories with their respective SDLXLIFF files from the main project? This avoids using third party apps like SDLXliff2Tmx (then exporting TMs to TMXs can do the trick).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.