normal-computing / branches Goto Github PK

Create a new branch local-dev with modified endpoints to support local development. Add corresponding instructions in the README.

As we move towards a general tool for prototyping LLM reasoning and planning algorithms, we should decouple the notions of:

• algorithm being prototyped
• benchmark / problem set being visualized

Implement a series of features to enhance the interaction with nodes within BrainBranch:

• Enable the editing of a node's value. Ensure that any changes in the value of a node automatically update the status (pass/fail/flag) of the node and its entire subtree.
• Introduce a "Regenerate Subtree" button that triggers a refresh of the subtree following an edit.
• Allow users to view and edit the prompt that generates children nodes. Include an option to view this prompt easily.
• Log all edits to provide data that can be used for future fine-tuning of the model.
• Design a new card UI that allows users to edit the prompt for generating children nodes and execute the generation with a "Generate Children Nodes" button

Address UI-related issues and introduce new customization settings:

• Resolve the unexpected color change in the sidebar that transitions from blue to white, then to orange.
• Add a setting to allow users to customize the tree fanout for the Game of 24.
• Develop a UI element for displaying and editing AI evaluations and scores in the code generation visualization, similar to the approach used for Game of 24.
• Create a scaffold for AI evaluations to be displayed as editable cards, which would include the prompt involved in generating the evaluation.

Address critical bugs affecting branches functionality:

• Investigate and fix the timeout issue with gpt-3.5-turbo, particularly for HumanEval 51, where gpt-4 seems to function correctly.
• Troubleshoot and fix the issue with gpt-instruct models not working through the OpenAI API, taking cues from how Flux handles instruct models.
• Address the UI bug where the answer node in Game of 24 incorrectly displays a loading animation (spinning).

Implement these new features:

• Implement a feature to reorder outputs after steps in the Game of 24 logic.
• Establish a dictionary map of model costs to streamline the computational expense tracking.
• Conduct more prompt engineering to improve the default pre-loaded prompts.

Assuming that we want to open source this repo, we should update the docs and pick a name for the library to represent what we're trying to do here

We also need to make sure to properly cite flux, as we are borrowing a lot from this repo despite the distinct purpose