It is necessary in order to avoid people referring to the tracker issues instead of using some more specific issue that can be closed when changes are landed.

For example it might be a good idea to put the TM search and the TM list in separate pages.

They don't feel right. Code will be cleaner and easier to understand if we just don't use global at all.

For example Inkscape TM can be tagged as "graphics", and LibreOffice and Apache OpenOffice can be tagged as "office".

Even if all SC-related HTML files are including some jQuery-dependent scripts (like the cookiechecker), they don't include the jQuery library, what generates an error when the pages are displayed in the browser.

This is required by #16

https://pypi.python.org/pypi/pep8

Examples of use in Pootle repo, among others.

Including the svg/ directory.

In the README it says "python 2.7 or higher".

But I have serious doubts about if this code can work on Python 3 (I am thinking on stuff like print "some string" where print is treated like a keyword instead of a function).

Also I am curious why this is not supposed to work on Python 2.6 (maybe it is because of format?).

Move embedded HTML in Python code to standalone templates (first step towards using Flask).

This means, for example, converting all the prints with HTML code in https://github.com/Softcatala/translation-memory-tools/blob/57fa66e1/web/web_search.py to something like https://github.com/Softcatala/translation-memory-tools/blob/57fa66e1/web/index-creation.py#L37-L39 that renders a template with all that embedded HTML code.

It is not clear which coding style is used for the various shell scripts present in this repository. Having some clear guidelines, like how many characters are used for indentation, if then appears in the same line as if or not, and such will be helpful to create consistent code easier to read by humans.

Background

This ticket collects all the architecture improvements needed to fully the new set of requirements and address the limitations that we learnt until September 2014.

Downloading, converting and building translation memories

• Decouple the download process #63
• Create a configuration subdirectory where every project has its own json file #62

Solution: Consider moving to TMX as native format for the tool. Some considerations:

• The limitation here is that there are not out of the box tools for merging (msgcat) catalogs.
• Probably we will need to build something and consider contributing it to translate toolkit.
• Many tools that convert from other formats like TS, strings, etc, they do convert to PO (ts2po) not to TMX. We need need to think if we are OK converting from these formats to PO and then to TMX or we need native conversors.
• This will require rewritting the index creator, terminology analysis and other tools since all the of they relay on PO files as source format

Limitation: The conversion from any format to PO format is limited. The problems observed are:

• Currently we are using the file extensions to identify the formats. In the case of INI or strings files you need sometimes to be more specific since these can have different variations.

Solution: By the default, as today, we have conversors associated to extensions. However, also having some kind of pattern matching in projects.json where you can specify per project which conversors to use.

Web application

Limitation: Currently all the Softcatalà application is tightly coupled with with the backends.
Solution: The Softcatalà application, and any other front ends, should be independent applications that different teams maintain that use APIs to interact with the system. In github, we should have a simple agonistic web application to show the APIs work (instead of Softcatalà one). We should provide 3 APIs:

• API to search the text index (#28)
• API to access the translation memory downloads created (date, file, etc)
• API to access the terminology items created (glossaries)

Limitation: The web application is written using CGI
Solution: Write the application using MVC (#24)

Text Search engine

Potential limitation: We are currently using Whoosh as full text search engine. We are not sure how this will scale if we add 50 languages and 50 projects more for example.
Solution: See (#24)

Integration with Translation Memory servers

The vision here is not to implement a Translation Memory server. Implement #23 to integrate with (Amagrama) https://github.com/translate/amagama.

Because sometimes you might need to narrow your search for specifically searching in docs or in interface translations.

This means keep TMs for old releases, like GNOME 2.28 or GNOME 2.30, available for checking how the translation evolves.

These old TMs shouldn't be queried in the default search.

I suggest using the Pootle Python coding style (which extends the TTK one which extends PEP8): http://docs.translatehouse.org/projects/pootle/en/latest/developers/styleguide.html

I suggest using the Pootle CSS coding style: http://docs.translatehouse.org/projects/pootle/en/latest/developers/styleguide.html#css

I am talking about maybe:

• https://github.com/Softcatala/translation-memory-tools/tree/master/source-pos/recull
• https://github.com/Softcatala/translation-memory-tools/blob/master/web/index.html
• and perhaps other stuff

2 or 3 paragraphs intro, and move the rest to the docs/ directory.

This includes plain HTML files and any file used as template or snippet parsed by any templating system (Jinja, Mustache or whatever).

Flask has been suggested to be such framework

• Simplify the root README (2 or 3 paragraphs intro, and move the rest to the docs/ directory) (#51)
• Include this project docs in Read the Docs (#52)
• Remove all README (or other notes files) in the repository, but the root README (#57)
• Add docs about how to deploy (#53)
• Add docs about how to import translations (#54)
• Add docs about how to help in development (#55)
• Add coding style for Python (#37)
• Add coding style for CSS (#38)
• Add coding style for JavaScript (#39)
• Add coding style for HTML (#40)
• Add coding style for shell scripts (#8)

This means moving unittests/ directory and integration-tests/ directory to a new tests/ directory.

I can't believe this has no license or copying file.

Like jmas home or other similar paths.

I know it is going to be really hard.

This is feeding the tool with more translations :)

They can be converted into issues or moved to docs/.

This is necessary because some projects update few times in long periods of time, and some other projects are just dead. Also not all the projects use a translation format that can be easily converted to PO or TMX, at least without human intervention.

Acceptable frequencies can be:

• never (this TM was created manually and is not meant to be recreated automatically)
• yearly
• monthly
• weekly (most important FLOSS projects)

In a future it must be possible to specify different frequencies by language.

This issue has to be split into several smaller ones, i.e. this is a tracker issue.

This is for providing a way to quickly find all the ways a given English word is translated to a source language.

For example if we want to know all the ways the "window" word is translated to Galician language we can start by specifying "xanela", then "fiestra"... and that hides the results that match and counting the occurrences by project. This useful when discussing terminology.

This can be achieved with a bit of JavaScript on a special search page that returns all the results for the specified query.

• Translate comments in Catalan to English (#30)
• Add license (#35)
• Softcatalà has a grave accent (so à not á) (#68)
• Add docs (#14)
• Include building docs on Travis (#69)
• Include run of integration tests on Travis (#70)
• Remove specific references in code to jmas home (and stuff like this) (#56)
• Move images to img/ directory (#9)
• Move CSS files to css/ directory (#10)
• Move templates to templates/ directory (#11)
• Move SoftCatalá specifics out of repo (#27)
• Move all tests code to tests/ directory (#31)
• Perform a cleaning sweep on all the code to ensure we have valid HTML5 (#48)
• Perform a cleaning sweep on all the code to ensure we have valid CSS (#49)
• Perform a cleaning sweep on all the code to ensure we have Python following agreed coding style (#50)
• Get rid of globals on Python code (#60)
• Move embedded HTML in Python code to standalone templates (#41)
• Make sure which Python this works on (#43)
• Remove all PO and TMX files that are not used for testing purposes (#44)
• Move the commands to a bin/ directory (not having a bunch of .py files in src/)
• Make it clear which one is the original files directory, the PO directory, the TMX directory... for the importing process.
• Discuss about maybe separating in two different repositories the translations retrieval (including the translations retrieval and the TMX and PO compendia creation) from the web query server (including index creation and code for the server).

I suggest using the Pootle HTML coding style: http://docs.translatehouse.org/projects/pootle/en/latest/developers/styleguide.html#html

This allows to quickly install all the dependencies using pip. It might be necessary to have a directory with several files, each for different scenarios.

This means provide search results using only a JSON API. The web page holding the search form then can retrieve the results from the API and append the results in the very search page. That way the search page can be served as static HTML and we already have an API capable of feeding CAT tools.

Pushstate is essential. It must be possible to be able to share specific search URLs with other people.

Depends on #23

Like http://www.microsoft.com/Language/en-US/Search.aspx This is very long term because we depend not only on having a working TM search tool, but a terminology search tool.

I suggest using the Pootle JavaScript coding style: http://docs.translatehouse.org/projects/pootle/en/latest/developers/styleguide.html#javascript

This includes all the steps to set up the development environment, and also how to send contributions, commit messages, where are the issues,...

• Have better visual design (using CSS)
• Split some pages (#26)

In order to:

• Prioritize the returned results
• Allow to just query by default the TMs with higher quality (in some special scenarios it still is necessary to query all the existing TMs)