acrosman / salesforce2sql Goto Github PK
View Code? Open in Web Editor NEWElectron app to convert a Salesforce org's schema into a similarly shaped database.
License: MIT License
Electron app to convert a Salesforce org's schema into a similarly shaped database.
License: MIT License
Summary
Add an event subscriber on console messages to capture messages within the primary interface.
Solution
Electron's webContents emits an event when messages are sent the browser console. Since we have a logging console for system messages we should try to subscribe to those events and log those messages in the UI.
Describe the bug
Currency datatype is Decimal (8,2) regardless of SF settings
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Desktop (please complete the following information):
Additional context
Noticed the problem because I was getting out of range issues on several currency fields.
Increasingly the SF_calls.js module has several other things in it that has nothing to do with calling Salesforce. The code should be refactored to rename that file to be more accurate and potentially separate the actual SF calls into their own file again or some other similar pattern.
Summary
Create a message to display after the database generation completes to show status of each table.
Please describe the problem you are trying to solve.
Currently nothing appears in the Last Response area to let you know it completed.
Proposed Solution
A popup message that announces the results, and a table-by-table break down that can be displayed in the Last Response area.
Summary
Right now we push all SF decimal field types to float
. They should really all be pushed to decimal
Code to update is here.
Right now it uses the default electron Icon, a proper custom icon is needed.
Summary
With the merge of #49 which closed #48, we now have some test coverage. Bu there is only a little test coverage at the moment. Over time we'll work toward more.
This issue is for generally reminding of the need to include more tests. Additional sub-issues and related issues will be added for specific requests. If you'd like to add tests generically, you may want to look at the functions in src/sf_calls.js
and src/preferences.js
that do not callback to the main window directly (the project already includes rewire to make it easy to access non-exported features of the modules).
There are also no tests currently covering the render JS files, which are fairly extensive as well.
Specific Testing Issues
Iโd like to be able to save the output of a schema.
@acrosman what other detail would be helpful to build this out?
Add a Select All/ Deselect All checkbox for object selection.
After fetching all objects having a checkbox to Select All/ Deselect All to make it easier to grab only needed objects faster.
Describe the bug
When you log out of a Salesforce org, the org is still listed on the set of active collections.
To Reproduce
Expected behavior
The disconnected org should be removed.
Summary
Allow users to select from the present themes
Please describe the problem you are trying to solve.
Bootswatch is used to provide the dark theme, but there are several other themes in the package, might as well let people access them.
Proposed Solution
Add a preferences pane that allows users to select the theme from the list of options that are installed anyway.
Summary
There are lots of changes in the main branch, and pending, that justify a new release. There are several details that should get wrapped up before that happens.
Summary
The ReadMe would be more useful and appealing with a few screenshots of the interface.
Summary
I'd like to be able to say "we use case teams" or "we use pricebooks" or other common features and have the tool check more objects for my schema
Please describe the problem you are trying to solve.
I wish I didn't have to try to remember which objects are included within a particular Salesforce feature.
Proposed Solution
Show a list of features to add or remove using checkboxes. Use the checkboxes drive the selected objects for the schema. This could be a "step 1" where "step 2" is customizing with more granularity (the existing detailed selection).
Summary
Bootstrap 5 has been released and it would be great to update to it (along with Bootswatch) and deal with legacy issues.
.pre-scollable
-bs
additions to tag names.Update version string and anything else that needs correction.
Summary
Add the ability to update a schema based on changes in source org or to add tables.
Please describe the problem you are trying to solve.
Currently the schema save/load process just allows you to regenerate the SQL not update the fields based on changes in Salesforce. It would be a better solution if you were able to tell it to refresh the org objects and fields based on connected org.
Proposed Solution
Once the schema loads from file it should take the user to the Object list, and present a button refresh the objects from Salesforce. When loading and proposing the schema itself the user should be able to indicate (via checkbox?) if they want existing objects refreshed or just to use what was already loaded from file.
To do this well the interface should really be rebuilt so you can easily move forward and back through the process, with controls only showing on the sections that are active.
The project either needs a mock that supports the needed elements of electron to allow for testing of dialogs, or needs a library to provide Jest support for electron other ways. This issue should use the same testing approach as #65.
Summary
Adding settings to change some behaviors and assumptions about how to translate Salesforce fields.
Please describe the problem you are trying to solve.
Depending on the use case some standard SF fields might be useful in a couple different forms. Having settings to controls those could be helpful. For example, the main record ID field might be more useful in some settings as an auto number, or plain integer than as an SFID. Another example would be RecordTypeIDs being set as the record type name on the assumption it would be translated during later processing.
Proposed Solution
Add the following settings to the preferences:
Additionally:
PRs for any individual piece happily considered. Please do not comment that prs that fix a piece will fix or close this issue.
Describe the bug
Currently handleLogin
in app/render.js
pulls the submitted user name from the input form instead of a value from the response. Ideally this would use a value from the main process's response.
Describe the bug
When logging into an org the tool can fail without any response in the interface.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
An error to appear. Something to tell you it failed.
Desktop (please complete the following information):
Describe the bug
After generating SQL into a database the Save SQL button should be active to allow that same SQL to be dropped to disk. But under some conditions it is not.
May be tied to there being an error during the SQL processing with the database. Myabe just be a regression in v0.4.0
The various IPC calls block testing of several important codes. The project either needs a mock that supports the needed elements for those calls, or needs a library to provide Jest support for electron other ways.
The buttons and the tabs in the rebuilt interface are too similar and are likely to cause confusion. The buttons in particular need to be more button-like to separate them from the tabs clearly.
Summary
sf_calls.js has several undocumented functions that could benefit from good doc comments.
Summary
Currently we use a series of object describes, which for large orgs is very slow. A query of FieldDefinitions should allow for pulling of all, or at least most, field descriptions in one query and only 4 or 5 cycles to Salesforce API (instead of hundreds).
Summary
JSForce supports OAuth connections, they should be used.
Is your feature request related to a problem? Please describe.
Using the old security token system is not ideal. OAuth connections would be a better choice for most users.
Solution
Setup Salesforce2Sql to use OAuth connections to Salesforce.
Existing tests of buildSchema send two objects each with two fields. There are only two types of field covered, and none of the preferences options are tested. Please add to the imported schema file a few more fields on each object using additional Salesforce field types (you can get a list of supported field types in app/sf_calls.js), and augment the tests to validate those new fields. Contributions that include coverage of additional details of that feature are encouraged.
My default values aren't being set properly. Defaults for Varchar(255)s are set without being enclosed in single quotes, so they aren't quite working in practice.
Originally posted by @allisonletts in #30 (comment)
Summary
Being able to search the long lists of objects of a specific value is quite useful.
Please describe the problem you are trying to solve.
While the tool does a reasonable job of guessing what objects you want selected, it's useful to be able to quickly jump to the right place to select additional objects or removed selections. So a find in page would be helpful.
Proposed Solution
Something like the approach here or here.
Currently the src/preferences.js
has no test coverage. It would be helpful to have a test class that at least loads the module and validates that the module structure matches the expected. The first two tests covering sf_calls.js provide a pattern to work from.
Any additional test coverage is also welcome and appreciated.
Describe the bug
Knex auto generates index names, meaning that long field names can lead to even longer index names. Salesforce field names can be quite long. This tool then records an error in the log. Fortunately the indexes are added as table alters so the actual table is created just fine, Indexes on the table after the failure are lost.
To Reproduce
This can easily come on up Namespaced tables like the following from Blackthorn:
Error 1059(ER_TOO_LONG_IDENT) creating table: alter table `conference360__Table__c` add index `conference360__table__c_conference360__main_contact_attendee__c_index`(`conference360__Main_Contact_Attendee__c`) - ER_TOO_LONG_IDENT: Identifier name 'conference360__table__c_conference360__main_contact_attendee__c_index' is too long. Full statement: alter table `conference360__Table__c` add index `conference360__table__c_conference360__main_contact_attendee__c_index`(`conference360__Main_Contact_Attendee__c`)
Proposed Solutions
Minimal: Catch the error so all other indexes can complete.
Better: Find a way to replace the generated index name from Knex with something shorter.
Setup
Windows 10
MySQL
Describe the bug
Picklist values with ' in them cause errors.
To Reproduce
'
ER_PARSE_ERROR: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near
Expected behavior
Table to be created successfully.
Describe the bug
Objects with lots of fields may violate the table row size in the database. We should detect the that behavior and protect against it.
To Reproduce
Create a Salesforce object with a large number of fields, particularly large text fields (not text areas or rich text), and then attempt to generate a table from that object. The log message may contain a failure based on this error:
SQL Error (1118): Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs
Suggested Fix
Catch the error during processing and replace all varchar fields with text fields. That generally will adjust the size as needed and the importance of a varchar over text is rare for most use cases that want to use a tool like this.
Summary
If I'm creating a schema to plan a migration, I want to ignore the R/O fields.
Please describe the problem you are trying to solve.
It's hard to tell which fields shouldn't be mapped to, even though it'll throw an error in SF.
Proposed Solution
Options:
Currently the main render.js file has no test coverage. Testing at least the helper functions near the top of the file would be quite useful.
Summary
displayObjectList
has a large block of duplication code, plus other features that should be moved to main.
Details
displayObjectList
in render.js
has a pair of loops over the Org's object list to sort likely selected objects to the top of the list. They should be refactored to at least contain a helper function to reduce duplicate code.
Summary
It would be great to minimize to jquery use in the interface to make the code more consistent.
Proposed Solution
Only the json data viewer requires direct use of jQuery. All other instances of $()
should be replaced with native JS.
Summary
There should be a status update during SF API calls cycles to get object list and field details.
Please describe the problem you are trying to solve.
SF calls are slow, and for a big project there are lots of them. The interface is largely useless while they are running and nothing provides an indication it's running.
Proposed Solution
Initially at least greying the screen and switching to a throbber would be better than current state. But really a progress display that provided updates about what phase the API process was in would be much better.
Summary
Add a feature to save the generated SQL file for editing, and reuse.
Please describe the problem you are trying to solve.
It could be useful to save the SQL statements to disk not just run them.
Proposed Solution
In the database modal add an option to save the SQL statements to a file (needs a save dialog call like the one being built for #13). Knex can do this without an active database connection so that might be an add-on feature as well.
Describe the bug
Some issues with data structures that have come up in practice:
Desktop (please complete the following information):
Summary
We need automated testing. Since Jest is SFDC's library of choice seems like the right choice here.
Summary
Move SF Org type detection to main thread and handle when describe calls come back.
Details
In render.js
there is a snifOrgType
function that determines what kind of Salesforce Org you have to help select the right set of default objects. This really should be in the processing context not the UI. Details about what org type it is and recommended default object selection can be added to the response. Those are currently in selectStandardObjects
which should also be relocated.
When loading the schema from a file the recently redesigned interface should advance from the object screen to the schema page on load but does not.
Currently buildFields()
is only tested via the buildSchema() tests. To help find and fix errors as new features are added to each stage more details coverage of the helpers would be useful. Following the pattern of testing for buildSchema, we should add tests to cover buildFields in a similar fashion.
The knex library is used for database abstraction. To test those features we need a good mock or library that projects jest support for knex.
Summary
Since we're pulling and displaying data from a remote source ideally Electron's Sandbox option would be enabled on the main window. The current protections, and the source of the remote data being a fairly secure source itself, mean there is low risk having this disabled. IPC setup changes may be required to make this work smoothly.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.