Chunky is a command-line tool designed to automate the process of proofreading and editing large text files. By leveraging the GPT-4 API, it breaks down the input text into manageable chunks, submits them for analysis, and compiles the results. This tool is invaluable for writers, editors, and anyone looking to enhance the quality of their text with the help of advanced AI.

See some similar small tools by the same author, to see if they have anything useful to learn from or borrow:

• pixie - sends local image files to GPT-4 Vision API
• sylph - was intended to grow into a command-line chat interface to OpenAI, never grew very far :-)

• Text Segmentation: Splits large text files into smaller, paragraph-based segments. (Currently about 100 words long; should be made longer.)
• Automated Proofreading: Uses a customizable prompt to check text for typos, spelling errors, duplicated words, and more.
• Flexible Output: Accumulates AI's suggestions and corrections, writing them to a specified output file for easy review.

• Python 3.x
• A virtual environment (recommended for Python package management)

Clone the repository to your local machine using:

git clone https://github.com/peterkaminski/chunky

Navigate to the cloned directory and create a virtual environment:

python3 -m venv venv

Activate the virtual environment.

On Windows:

.\venv\Scripts\activate

On Unix or MacOS:

source venv/bin/activate

Install the necessary Python packages:

pip install -r requirements.txt

Sign up for an OpenAI developer account, and generate an API key in settings. See Where do I find my API Key? in the OpenAI Help Center. OpenAI provides a document about Best Practices for API Key Safety, which you should review.

Then on Unix or MacOS, copy the env.sh-template file to env.sh, then edit env.sh to insert your OpenAI API key. Add your API key to the shell environment:

source env.sh

On Windows, the process will be similar, but we do not have documentation for the process yet.

In any case, make sure you do not push any file with your API key back to your upstream Git host, such as GitHub.

Run Chunky with the following command structure:

./chunky.py -i <input_file.txt> -o <output_file.txt> -p "Your custom prompt"

• -i or --input: Specifies the path to the input file.
• -o or --output: (Optional) Specifies the path to the output file. Defaults to output.txt.
• -p or --prompt: (Optional) Custom prompt for the AI to use when analyzing text. Defaults to checking for typos and grammatical errors.

To process document.txt with a focus on identifying typos and grammatical errors:

./chunky.py -i document.txt -p "Check for typos and grammatical errors."

This command reads document.txt, processes it through GPT-4 with the specified prompt, and writes the corrected text to output.txt.

The output will be a text file (default output.txt) containing the AI-processed segments. Each segment is based on the input text but revised according to the AI's response to the prompt, making it easier to review and make further edits.

• The software is provided "as is", without warranty of any kind. See the LICENSE file for more details.
• This is a "0.1" version of the script, so some features may be missing, more error checking may be needed, etc. Please provide feedback!
• You are responsible for providing your OpenAI API key to operate the software. Your API key is not part of this software. You are responsible for keeping your API key secure.
• The text analysis is generated by GPT-4; if you get a surprising written response, or a refusal to analyze a text, it's due to OpenAI and GPT-4. Try providing a different prompt with --prompt.
• see the [[Roadmap.md]] file for plans and ideas for future development.

This project is licensed under the MIT License. See the LICENSE file for details.

Contributions to this script are welcome. Feel free to fork, modify, and send a pull request.

The first version of Chunky was written by ChatGPT (GPT-4), from a prompt by Peter Kaminski. See [[Chunky Initial Version.md]] for more information.

For any issues or questions, please open an issue in the repository or contact Peter Kaminski.