talend / tdi-studio-se Goto Github PK

Java 98.93% CSS 0.01% HTML 0.01% XSLT 0.13% Batchfile 0.01% Shell 0.01% PowerShell 0.01% ASP.NET 0.92%

tdi-studio-se's Introduction

Talend Open Studio for Data Integration

http://www.talend.com

Contents

This repository contains the source files for Talend Open Studio for Data Integration. These files must be used together with the common code contained in tcommon-studio-se.

Repository Structure

All Talend Studio repositories follow the same file structure:


  |_ main           Main Eclipse plugins and features
    |_ features
    |_ plugins
  |_ test           Eclipse plugins and features for unit tests.
      |_ features
      |_ plugins
  |_ i18n           Internationalization plugins and features.
      |_ features
      |_ plugins

Download

You can download this product from the Talend website.

Usage and Documentation

Documentation is available on Talend Help Center.

Support

You can ask for help on our Forum.

Contributing

We welcome contributions of all kinds from anyone.

Using the bug tracker Talend bugtracker is the best channel for bug reports, feature requests and submitting pull requests.

Feel free to share your Talend components on Talend Exchange.

Contributors

See the CONTRIBUTORS.md file for details.

License

Licensed under the Apache V2 License

tdi-studio-se's People

Contributors

Stargazers

Watchers

Forkers

gaozone-talend dakalka i-melnik kirktarou plv-talend brazabr deki fxprunayre amaumont hwang-talend csuntalendbj xhkong-talend amarouni massdevil tuntunmo rubikloud bguillon-td 47lining htyin pxmiao-talend ggu-talend dmitry-savina njoit rdubois glzhou indiantonike rlugojr hanwenbiao jlolling hywang1985 joclaude talend-glorieux bigluster yrabl apoltavtsev velanati pavantyagi dmuliyil scottma marciuc ashokkumards jmcgill-public guol-talend talender lbourgeois scorpioks coresoft2 tmg-mhayhurst yjacolin benzei zafkirtalend xldai qcui huskers2012 lil1ian3 yu-talend yanbasic samked sureshsamikannu mtuhai chenmingyi1 diegoqueiroz 195858 bthompson-nd dsergent zizou84 achursin-talend chaitanyatejagolla overpackaging wangliu-talendbj jinsp ttao-talendbj vbogatyrov zz38 jiezhang-tlnd qiongli izzyalanis ookk3388 yashkumarthakur avinash-servian avinash2fly rmannibucau bhc1983 lawrence-xf cooshal jding-tlnd rafikbenleulmi chernykhsg 180ukinnver coheigea xiaofeng-zhang linjiebelfast lperdereau ediboko1980 batje bgunics-talend cor277 pkjmesra qpandevops tomohiro-shrkm-r

tdi-studio-se's Issues

Security Policy violation Dangerous Workflow

This issue was automatically created by Allstar.

Security Policy Violation
Project is out of compliance with Dangerous Workflow policy: no workflows found

Rule Description
Dangerous Workflows are GitHub Action workflows that exhibit dangerous patterns that could render them vulnerable to attack. A vulnerable workflow is susceptible to leaking repository secrets, or allowing an attacker write access using the GITHUB_TOKEN. For more information about the particular patterns that are detected see the Security Scorecards Documentation for Dangerous Workflow.

Remediation Steps
Avoid the dangerous workflow patterns. See this post for information on avoiding untrusted code checkouts. See this document for information on avoiding and mitigating the risk of script injections.

Dangerous Patterns Found

Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores a project's adherence to security best practices. You may wish to run a Scorecards scan directly on this repository for more details.

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Supported versions of MongoDb and MySql

Hi,

I'm trying to identify the supported versions of MongoDb and MySql but am having trouble separating the documentation from the Enterprise and Open versions of the software.

What is the latest version supported for:
a) MongoDb
b) MySql

Is the version supplied on the Talend site sourced from this repository and is it regularly updated. I only ask as the downloaded file has a 2021 version timestamp in the filename.

Any information is gratefully received.

Thanks,
Andrew

Talend Open Studio version upgrade issue

Hello,
We have been experiencing a Talend Open Studio version upgrade issue since we updated the software from version 5 to version 8.
Indeed, the use of our Jobs with temporary data causes bugs in the tMap components due to the volume of temporary files generated.
We cannot use memory files instead because they saturate our server.

We have discovered that the problem is known to the Talend community:
https://community.talend.com/s/question/0D55b0000779OfqCAE/tos-801-tmap-temp-space-javaioeofexception?language=en_US

But it doesn't solve our problem.
Here are our error logs when we run a Job:

2023-11-10 15:31:55|Voies_Codes: Etape 3/6 : Jointure des codes pour chaque voie
2023-11-10 15:32:01|java.lang.RuntimeException:java.io.EOFException: Read past end of file
2023-11-10 15:32:01| * Fin du sous-job Voies_Codes
Exception in component tRunJob_19 (Publication)
java.lang.RuntimeException: Child job returns 1. It doesn't terminate normally.
Exception in component tMap_4_TMAP_IN (Voies_Codes)
java.lang.RuntimeException: java.io.EOFException: Read past end of file
        at orasgit.voies_codes_2_0.Voies_Codes$RivolStruct.readValuesData(Voies_Codes.java:60116)
        at org.talend.designer.components.lookup.persistent.AbstractOrderedBeanLookup.loadDataValues(AbstractOrderedBeanLookup.java:262)
        at org.talend.designer.components.lookup.persistent.OrderedBeanLookupMatchFirst.next(OrderedBeanLookupMatchFirst.java:180)
        at org.talend.designer.components.lookup.persistent.PersistentSortedLookupManager.next(PersistentSortedLookupManager.java:600)
        at orasgit.voies_codes_2_0.Voies_Codes.tFileInputDelimited_1Process(Voies_Codes.java:28415)
        at orasgit.voies_codes_2_0.Voies_Codes.tWarn_4Process(Voies_Codes.java:25637)
        at orasgit.voies_codes_2_0.Voies_Codes.tFileInputDelimited_4Process(Voies_Codes.java:25002)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_3Process(Voies_Codes.java:13824)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_2Process(Voies_Codes.java:13621)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_1Process(Voies_Codes.java:13406)
        at orasgit.voies_codes_2_0.Voies_Codes.tWarn_3Process(Voies_Codes.java:13191)
        at orasgit.voies_codes_2_0.Voies_Codes.tMysqlInput_2Process(Voies_Codes.java:12887)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_4Process(Voies_Codes.java:11095)
        at orasgit.voies_codes_2_0.Voies_Codes.tWarn_12Process(Voies_Codes.java:10880)
        at orasgit.voies_codes_2_0.Voies_Codes.runJobInTOS(Voies_Codes.java:734)
        at orasgit.voies_codes_2_0.Voies_Codes.main(Voies_Codes.java:167)
Caused by: java.io.EOFException: Read past end of file
        at org.jboss.marshalling.SimpleDataInput.eofOnRead(SimpleDataInput.java:151)
        at org.jboss.marshalling.SimpleDataInput.readUnsignedByteDirect(SimpleDataInput.java:294)
        at org.jboss.marshalling.SimpleDataInput.readIntDirect(SimpleDataInput.java:343)
        at org.jboss.marshalling.SimpleDataInput.readInt(SimpleDataInput.java:316)
        at orasgit.voies_codes_2_0.Voies_Codes$RivolStruct.readString(Voies_Codes.java:59965)
        at orasgit.voies_codes_2_0.Voies_Codes$RivolStruct.readValuesData(Voies_Codes.java:60107)
        ... 15 more

        at orasgit.publication_2_1.Publication.tRunJob_19Process(Publication.java:27158)
        at orasgit.publication_2_1.Publication.tRunJob_5Process(Publication.java:26568)
        at orasgit.publication_2_1.Publication.tWarn_12Process(Publication.java:25889)
        at orasgit.publication_2_1.Publication.tRunJob_4Process(Publication.java:24741)
        at orasgit.publication_2_1.Publication.tFixedFlowInput_1Process(Publication.java:24014)
        at orasgit.publication_2_1.Publication.tWarn_19Process(Publication.java:23233)
        at orasgit.publication_2_1.Publication.tFixedFlowInput_13Process(Publication.java:22932)
        at orasgit.publication_2_1.Publication.tWarn_9Process(Publication.java:21917)
        at orasgit.publication_2_1.Publication.tMysqlInput_8Process(Publication.java:21592)
        at orasgit.publication_2_1.Publication.tMysqlInput_2Process(Publication.java:16021)
        at orasgit.publication_2_1.Publication.tCreateTemporaryFile_1Process(Publication.java:10153)
        at orasgit.publication_2_1.Publication.tFixedFlowInput_3Process(Publication.java:9892)
        at orasgit.publication_2_1.Publication.tWarn_4Process(Publication.java:8675)
        at orasgit.publication_2_1.Publication.tMysqlInput_1Process(Publication.java:8364)
        at orasgit.publication_2_1.Publication.tWarn_2Process(Publication.java:6226)
        at orasgit.publication_2_1.Publication.tMysqlConnection_1Process(Publication.java:6009)
        at orasgit.publication_2_1.Publication.tWarn_5Process(Publication.java:5768)
        at orasgit.publication_2_1.Publication.tWarn_1Process(Publication.java:5551)
        at orasgit.publication_2_1.Publication.tFileInputDelimited_4Process(Publication.java:5310)
        at orasgit.publication_2_1.Publication.tFixedFlowInput_12Process(Publication.java:4310)
        at orasgit.publication_2_1.Publication.tFileInputDelimited_2Process(Publication.java:3374)
        at orasgit.publication_2_1.Publication.tPrejob_1Process(Publication.java:1936)
        at orasgit.publication_2_1.Publication.runJobInTOS(Publication.java:39257)
        at orasgit.publication_2_1.Publication.main(Publication.java:38689)
2023-11-10 15:32:01|java.lang.RuntimeException:Child job returns 1. It doesn't terminate normally.
Exception in component tMap_4_TMAP_IN (Voies_Codes)
java.lang.RuntimeException: java.io.EOFException: Read past end of file
        at orasgit.voies_codes_2_0.Voies_Codes$RivolStruct.readValuesData(Voies_Codes.java:60116)
        at org.talend.designer.components.lookup.persistent.AbstractOrderedBeanLookup.loadDataValues(AbstractOrderedBeanLookup.java:262)
        at org.talend.designer.components.lookup.persistent.OrderedBeanLookupMatchFirst.next(OrderedBeanLookupMatchFirst.java:180)
        at org.talend.designer.components.lookup.persistent.PersistentSortedLookupManager.next(PersistentSortedLookupManager.java:600)
        at orasgit.voies_codes_2_0.Voies_Codes.tFileInputDelimited_1Process(Voies_Codes.java:28415)
        at orasgit.voies_codes_2_0.Voies_Codes.tWarn_4Process(Voies_Codes.java:25637)
        at orasgit.voies_codes_2_0.Voies_Codes.tFileInputDelimited_4Process(Voies_Codes.java:25002)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_3Process(Voies_Codes.java:13824)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_2Process(Voies_Codes.java:13621)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_1Process(Voies_Codes.java:13406)
        at orasgit.voies_codes_2_0.Voies_Codes.tWarn_3Process(Voies_Codes.java:13191)
        at orasgit.voies_codes_2_0.Voies_Codes.tMysqlInput_2Process(Voies_Codes.java:12887)
        at orasgit.voies_codes_2_0.Voies_Codes.tCreateTemporaryFile_4Process(Voies_Codes.java:11095)
        at orasgit.voies_codes_2_0.Voies_Codes.tWarn_12Process(Voies_Codes.java:10880)
        at orasgit.voies_codes_2_0.Voies_Codes.runJobInTOS(Voies_Codes.java:734)
        at orasgit.voies_codes_2_0.Voies_Codes.main(Voies_Codes.java:167)
Caused by: java.io.EOFException: Read past end of file
        at org.jboss.marshalling.SimpleDataInput.eofOnRead(SimpleDataInput.java:151)
        at org.jboss.marshalling.SimpleDataInput.readUnsignedByteDirect(SimpleDataInput.java:294)
        at org.jboss.marshalling.SimpleDataInput.readIntDirect(SimpleDataInput.java:343)
        at org.jboss.marshalling.SimpleDataInput.readInt(SimpleDataInput.java:316)
        at orasgit.voies_codes_2_0.Voies_Codes$RivolStruct.readString(Voies_Codes.java:59965)
        at orasgit.voies_codes_2_0.Voies_Codes$RivolStruct.readValuesData(Voies_Codes.java:60107)
        ... 15 more

The -Xmx and -Xms of the Job concerned are set to 12 GB. We cannot allocate more memory to the Job because the volume of temporary data is extremely large (more than 100 GB).

The JDK used is version 11, which is compatible with version 8 of Talend Open Studio.

Have you ever faced this problem?
Do you have any leads?

Thanks in advance.

Security Policy violation SECURITY.md

Allstar has detected that this repository’s SECURITY.md security policy is out of compliance. Status:
Security policy not enabled.
A SECURITY.md file can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible. Examples of secure reporting methods include using an issue tracker with private issue support, or encrypted email with a published key.

To fix this, add a SECURITY.md file that explains how to handle vulnerabilities found in your repository. Go to https://github.com/Talend/tdi-studio-se/security/policy to enable.

For more information, see https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository.

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Impossible to build: method retrieveSchema(ComponentProperties) is undefined for the type ComponentService

Hi,

After various tentativs to buid Talend Open Studio I open an issue here.

Compilation is made with Java 11

java version "11.0.16" 2022-07-19 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.16+11-LTS-199)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.16+11-LTS-199, mixed mode)

The building process failed because the method retrieveSchema(ComponentProperties) is undefined for the type ComponentService in the org.talend.repository.generic module.

[INFO] --- tycho-compiler-plugin:1.7.0:compile (default-compile) @ org.talend.repository.generic ---
[INFO] Compiling 54 source files to /home/ccharly/talend-sources/tdi-studio-se/main/plugins/org.talend.repository.generic/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] tcommon-studio-se .................................. SUCCESS [  0.202 s]
[INFO] org.talend.libraries.apache.common ................. SUCCESS [  0.826 s]
[INFO] org.talend.libraries.apache ........................ SUCCESS [  0.470 s]
[INFO] org.talend.utils ................................... SUCCESS [  1.282 s]
...
...
[INFO] org.talend.repository.feature ...................... SUCCESS [  0.873 s]
[INFO] org.talend.repository.ftp .......................... SUCCESS [  1.023 s]
[INFO] org.talend.repository.generic ...................... FAILURE [  0.524 s]
[INFO] org.talend.libraries.json .......................... SKIPPED
[INFO] org.talend.repository.json ......................... SKIPPED
[INFO] org.talend.repository.mdm .......................... SKIPPED
[INFO] org.talend.sdk.component.studio-integration ........ SKIPPED
...
...
[INFO] org.talend.mdm.ws.jars ............................. SKIPPED
[INFO] org.talend.tom.feature ............................. SKIPPED
[INFO] tos 8.8.8-SNAPSHOT ................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:42 min
[INFO] Finished at: 2023-06-02T15:35:21+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:1.7.0:compile (default-compile) on project org.talend.repository.generic: Compilation failure: Compilation failure: 
[ERROR] /home/ccharly/talend-sources/tdi-studio-se/main/plugins/org.talend.repository.generic/src/main/java/org/talend/repository/generic/ui/GenericSchemaForm.java:[265] 
[ERROR] 	Schema schemaObj = compService.retrieveSchema(compProperties);
[ERROR] 	                               ^^^^^^^^^^^^^^
[ERROR] The method retrieveSchema(ComponentProperties) is undefined for the type ComponentService
[ERROR] 1 problem (1 error)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :org.talend.repository.generic
\n--------------------------------------------------------------------

An enhancement was introduced recently:

fix(TUP-37395):Schema changes made on DB side not being reflected in Studio 8.0. #8367 -> https://github.com/Talend/tdi-studio-se/pull/8367/files.
fix(TDI-49151): Need API to get snowflake schema changes (#2192) #2205 -> https://github.com/Talend/components/pull/2205/files

Does the instanced ComponentService object get an implemented retrieveSchema(compProperties) method?

Kind regards.

Project merge

How do I merge the tdi-studio source code with the tcommon source code and execute them ?

Security Policy violation Outside Collaborators

Allstar has detected that this repository’s Outside Collaborators security policy is out of compliance. Status:
Did not find any owners of this repository
This policy requires all repositories to have an organization member or team assigned as an administrator. Either there are no administrators, or all administrators are outside collaborators. A responsible party is required by organization policy to respond to security events and organization requests.

To add an administrator From the main page of the repository, go to Settings -> Manage Access.
(For more information, see https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories)

Alternately, if this repository does not have any maintainers, archive or delete it.

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Security Policy violation Binary Artifacts

Allstar has detected that this repository’s Binary Artifacts security policy is out of compliance. Status:
Scorecard Check Binary Artifacts failed: binaries present in source code
Please run scorecard directly for details: https://github.com/ossf/scorecard

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Talend using empty trust manager by default

Hello, I've been hitting my head against this for some time now, I cannot understand why Truster references a non-existing certStore by default (talendcecerts) instead of the accepted default value: System.getProperty("javax.net.ssl.keyStore", System.getProperty("java.home") + "/lib/security/cacerts").

tdi-studio-se/main/plugins/org.talend.designer.components.libs/libs_src/talendssl/src/main/java/talend/ssl/Truster.java

Lines 38 to 43 in 9699972

 if (certStorePath == null) { 

 isSaveCA = false; 

 certStore = "talendcecerts"; 

 } else { 

 certStore = certStorePath; 

 }

Does anyone know the reasoning behind this? If not, might I suggest you change the default behavior?

	if (certStorePath == null) {
	isSaveCA = false;
	certStore = "talendcecerts";
	} else {
	certStore = certStorePath;
	}

talend / tdi-studio-se Goto Github PK

tdi-studio-se's Introduction

Talend Open Studio for Data Integration

Repository Structure

Download

Usage and Documentation

Support

Contributing

Contributors

License

tdi-studio-se's People

Contributors

Stargazers

Watchers

Forkers

tdi-studio-se's Issues

Recommend Projects

Recommend Topics

Recommend Org