Description

Enhance the documentation for curl-ier.js.

Details

• Provide detailed explanations of each command-line argument.
• Include examples covering a range of use cases to assist new users.

Expected Outcome

• Improved clarity and comprehensiveness of the documentation.
• Easier adoption and usage by new users.

Description

Integrate sophisticated rate-limiting logic in curl-ier.js.

Details

• Consider scenarios where the script interacts with APIs having rate limits.
• Implement rate-limiting logic to prevent hitting these limits.

Expected Outcome

• Prevention of rate limit breaches when interacting with APIs.
• More reliable and consistent script behavior under rate-limited conditions.

Description

Refactor the sendRequest function in curl-ier.js to enhance code maintainability and extend functionality.

Details

• Split sendRequest into two functions, one for GET and another for POST requests, to achieve better separation of concerns.
• Introduce a parameter or a configuration object to handle different HTTP methods beyond GET and POST.

Expected Outcome

• Improved code readability and maintainability.
• Easier extension and support for additional HTTP methods in the future.

Description

Implement a more advanced logging mechanism in curl-ier.js.

Details

• Explore the integration of logging libraries like winston or pino.
• Aim for greater control and flexibility in logging levels and outputs.

Expected Outcome

• Improved logging capabilities.
• Enhanced control over log management and analysis.

Description

Improve header parsing in curl-ier.js to handle edge cases more effectively.

Details

• Modify the header parsing logic to ensure it correctly handles cases where the header string might not have a colon or contains multiple colons.

Expected Outcome

• More robust and error-resistant header parsing.
• Prevention of potential runtime errors due to malformed headers.

Description

Enhance error handling in curl-ier.js, specifically in the shouldRetry function.

Details

• Expand the range of HTTP status codes in shouldRetry to include 500 series (server errors).
• Implement a mechanism for error logging and reporting for better monitoring and debugging, especially for production use.

Expected Outcome

• More robust and reliable error handling.
• Enhanced monitoring and debugging capabilities.

Description:
To improve the organization and tracking of output files, it would be useful to include the current timestamp in the filenames. Currently, the output filenames follow the pattern hgmd-genes.ABCA4.html. Enhancing this to a format like hgmd-genes.ABCA4.2024-02-21.html would make it easier to identify when each file was generated, especially in cases of multiple runs or data updates.

Acceptance Criteria:

1. The output filename should include the current timestamp in a readable format.
2. The timestamp should be placed between the main filename and the file extension.
3. The format of the timestamp should be standardized, e.g., YYYY-MM-DD.

Suggested Implementation:

• Update the filename construction logic in the saveResponse function to include a formatted timestamp.
• Ensure the timestamp is added before the file extension in the filename.

Description

Currently, curl-ier.js requires users to login for certain functionalities. This feature request aims to make the login process optional, allowing users to utilize the script without mandatory authentication when not needed.

Motivation:

• Flexibility: Making login optional enhances the script's flexibility, accommodating a wider range of use cases.
• User Convenience: Users who only require basic functionalities can skip the login process, simplifying their workflow.
• Adaptability: The script becomes more adaptable to different scenarios where authentication may or may not be required.

Proposed Change:

• Introduce a flag or parameter that enables or disables the login process.
• Modify the script logic to bypass login-related functions when this option is active.
• Ensure that existing functionalities remain intact and fully operational when the login is bypassed.

Possible Implementation:

• A command-line argument (e.g., --no-login) could be added to disable the login requirement.
• Conditional checks within the script to determine whether to execute login-related code based on this argument.

Impact:

• This feature will not affect existing functionalities for users who require the login process.
• It will provide a more streamlined experience for users who do not need authentication.

Requests with special characters like "*" can break the pipeline because the file name cant be saved.

Description:
The curl-ier application currently saves all output files with an .html extension. To enhance flexibility and accommodate various data formats, it would be beneficial to allow users to set custom file extensions, such as .json, .xml, etc. This feature would enable users to specify the format of the data they are expecting in the response, making the tool more versatile.

Acceptance Criteria:

1. Users should be able to specify the file extension when running the script.
2. The script should save the output file with the provided extension.
3. If no extension is specified, the script can default to .html.

Suggested Implementation:

• Introduce a new command-line option, e.g., --extension or -e, for users to specify the desired file extension.
• Modify the saveResponse function to use this user-provided extension when constructing the filename.

Description

Incorporate the use of environment variables in curl-ier.js for managing sensitive or environment-specific data.

Details

• Utilize environment variables or a configuration file for API endpoints, keys, etc.
• Ensure sensitive information is securely managed and easily configurable per environment.

Expected Outcome

• Enhanced security and flexibility in configuration management.
• Easier adaptation to different environments.