Right now in optimizer.py on line 238

completion = self.call_small(prompt = dynamic_prompt.strip(), model=model, **kwargs)

we call the small model (accessing the generate function) with the model parameter. Right now this does not do anything, we need to be able to provide a .pt file to that parameter and then call load_model (also defined in onsite_llm.py) in order to load that .pt file as the model to use. This needs to be done in the finetune function as well.

Add example tests:

1. Remove the RLHF censorship from a model
2. Only fine-tune on an example if the output leads to a desired result

Please brainstorm 2 other possible uses!

currently, finetuning doesn't persist models after they've been fine tuned

the current main branch has no issues from mypy's perspective, would that be a helpful lint for contributors?

We should use the larger LLM to synthesize data for training the small LLM with in the optimizing api

Outputs should be documented in the READMEs of each agents.

Should have a flag that downloads the model from one of the OSS repos.

Only Small_Local_Neo successfully fine-tunes. This logic needs to be extended to the other models in the file.

By setting up python-dotenv, the environment variables can be loaded from a .env file, which is in the root. And, I would like to work on this.

All HuggingFace arguments need to work on the optimizer.complete endpoint as well. For the complete endpoint this means the all arguments for the Huggingface .generate function must be able to be added to the optimize.complete function as well. Currently, this not something I was able to do with the max_new_tokens parameter, for example.

@itsmeashutosh43 and @VictorOdede I would love your feedback and Ideas on this

prompted by thinking about what we want out of regex/grammar constrained LLM inference, we've realized that we should just embrace having a more generic interface which those would be examples of.

I'm very much not familiar with python best practice, but heres my attempt at specifying it from a Types and stuff perspective thats slightly simplified.

"this is crudely our current api"
Class LLM(ABC):
   model : AbstractHF_Model
   tokenizer : Abstract_HF_Tokenizer
   genToken : List[Token] -> Map( Token_ID ,Log_Probability )

   generate_simple : ... bunch of stuff we pass to hf generate -> Async( String) 
"""generate_simple is our current generate """

   generate_with_Constraints : TokenConstraint -> ... same as above -> Async( String )

""" So lets talk about TokenConstraint!
"""
class TokenConstraint(ABC):

     constraint_type : type // the type of constraints we wanna have
     state_type : type //  this should be frozen/immutable, but i'll describe stuff in a way that maybe kinda works either way
     copy_state : 
     def is_valid (constraint : constraint_type ,  prefix : List of Tokens) ->  Bool : 

     def allowed_transitions(constraint: constraint_type, prefix : List of Token_id, tk : Tokenizer) -> Set(Token_id) : 

     def construct_state(constraint: constraint_type, prefix : List of Token_id, tk : Tokenizer) -> Union[None, state_type]
     // returns a state that *may* allow faster enumeration/checking of what tokens are allowed transitions next

     construct_crude_filter_set (constraint: constraint_type,tk : tokenizer,)-> Set [TokenID]
    // in eg a regex, this might be something like 
    // "the set of tokens that include only the character classes reference in the regexp"
   // it might be 

   // if we have anything complicated in constructing the current state, this should be *much* 
   // faster for filtering tokens 
   allowed_transitions_from_state (constraint : constraint_type, tk : tokenizer, st : state_type ) -> Set[token id]

    def copy_state(state : state_type)-> state_type 
   // if the state is immutable/frozen this should be the identity function 

something like this could maybe be the generic interface for constrained inference perhaps? i'm glossing over a lot of details like we're actually gonna be zeroing out the set of token indices that aren't in these sets in the log probability vectors, and all the mapping to and from tokenid <-> strings, and also the fact that we can do token constraint computation in paralell with running the gen_next_token_inference step

also in principle, things should be such that we can have instances of this interface that correspond with
logically And or Or or Xor or Nandtogether any two constraint languages I think?

Example:

> r = complete(prompt="how many eyes do cats have, and how many eyes do spiders have?", output_filter_regex="\{ 'cats': [0-9]*, 'spiders': [0-9]* \}"
> print(r)
{ 'cats': 2, 'spiders': 8 }

Excuse my regex though, I'm a bit rusty.

Like https://lmql.ai/ but with easier languages to use

We need documentation for the entire project, and each sub-folder, and part. And documentation for each function in the code itself, and a standard/programatic way to display documentation

open bounty for demo applications that work to add to a curated example gallery

System requirements list RAM, but I'd expect an LLM client to be using VRAM. Can it use either? How do I configure the client to use VRAM if I have a GPU for example?

List of open-source LLMs:

• Huggingface
• Wiki
• Awesome-LLM(GitHub)

• make sure that only people who've had successful tickets can get assigned
• soft assignments
• create deadlines such that people get unassigned

the first version uses JSON, which can often be malformed, and theres no good error recovery in that case, need to identify and switch to a more "error tolerant" self aligning format. (meaning we can skip a bad pair and recover useful outputs)

Current onsite LLM class uses full parameter fine-tuning which costly. LoRA fine-tuning will require less memory and prevent overfitting by freezing the pretrained weights.

its as simple as changing
print(s) to print(s,file=sys.stderr)

at least,. i think that is a pretty reasonable change that is good engineering to do

Tasks for end of June and through July 2023

LLM-VM tasks

Error handling

• Can cleanup in progress fine tunings when ctrl+c abort the local version of LLM-VM #23

• Can retry data synthesis when given malformed json results #23

• Maybe later should consider a more robust encoding rather than json, where mismatched brackets from unquoted responses #29

Agents

• Get REBEL agent onto main branch LLM-VM/main #2
• Get Backward Chaining agent onto main branch LLM-VM/main #2
• Get all agents to use the optimizer endpoint, as opposed to the openai api #19
• Get Flat agent on LLM-VM/main #2
• Agent test suite #20

Documentation

• Write documentation for LLM-VM on docusaurus. #22

Data Synthesis #3

• Add all the parameters
• choose a more robust encoding for question answer response sets #29
• K-shot support
• Prompt variation aka support alternative prompts, possibly with different defaults for each supported model
• Track number of actual responses vs requested
• Dedup handling
  • • First exact repeat parameters
  • • then semantic vector comparisons?
• Parameterize the diverse ways you might want to define exact matches or vector comparisions

Parameter Management

• Documenting all the parameters and making sure they’re sane #25

Instrumentation / Statistics

• Keep track of detailed information about inference, data synthesis, response sizes, latency, quality #26

Inference Determinism (more long term question,not immediate prioriity)

• What LLM flavors have explicit rng Seed being surfaces as optional input or something that can be part of the response metadata for reproducibility? #27

misc

• Maybe also something about managing/tracking fine tunings you’ve generated #28

The GPT3 and Chat_GPT class models do not allow for a way to store and load a fine-tuned model into the completion pipeline. Adjust class attributes and methods to allow fine-tuned model with c_ids to be accessed in the openai call.

Currently data_synthesis.py asks GPT to generate all the datapoints in one call. This leads to many issues with many many incorrect datapoints being made and not enough datapoints being made. I fixed this in the experimentation_finetuning branch, and that approach of data synthesis needs to be brought to main and made robust (my current implementation is for research not deployment).

One thing that needs to be done is semantic similarity checking the way that the current data_synthesis.py does. Also my implementation in experimentation_finetuning also does not support lists as inputs, and this needs to be added as well.

This will require re-running of tests. It should be possible to turn off this behavior at a high level. Blocked by both #2 and #18

Add linting and auto-formatting to keep code clean and improve efficiency

• Add linting and auto-formatting to requirements.txt (ex. pylint, black)
• Add pre-commit to run linting and auto-formatting
• Fix errors raised by linter

currently that config setup doesn't check that settings make sense

https://docusaurus.io/docs

Create a Small_Local_Bert class capable of managing the bert LLM similar to other code in the file.

 File "quickstart_finetune.py", line 5, in <module>
    from llm_vm.config import settings
  File "C:\Users\Abhigya Sodani\Anaconda3\lib\site-packages\llm_vm\config.py", line 55, in <module>
    if settings.big_model not in MODELS_AVAILABLE:
  File "C:\Users\Abhigya Sodani\Anaconda3\lib\site-packages\dynaconf\base.py", line 144, in __getattr__
    value = getattr(self._wrapped, name)
  File "C:\Users\Abhigya Sodani\Anaconda3\lib\site-packages\dynaconf\base.py", line 309, in __getattribute__
    return super().__getattribute__(name)
AttributeError: 'Settings' object has no attribute 'BIG_MODEL'

We need to replicate and fix these issues with dynaconf. I have this issue when I clone the repo and run pip run . .

currently we have the library depends on the web server, when theres no good reason to require it

QLoRA (an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance).
Converting LMMs to 4-bit quantized with QLoRA will reduce the required resources, speed up the fine-tuning and inference parts, and be much cheaper.

Refer to the following link for your reference

• https://github.com/artidoro/qlora

• https://huggingface.co/blog/4bit-transformers-bitsandbytes

Check entry points for consistency
Rework app directory structure into src/llm_vm and rename the _ _ init _ _.py file

Each agent should have a research-quality readme including test results and individualized usage and explanations.

• These need a standardized interface (so they take tool descriptions of the same format),
• They should also be updated to use non-api tools (just closures).

Separate into "prompt template" (a GPT-3 recognized word) and variables. Then ask the gpt-3 to change specific parameters for the template.

Prompt template: What is the [variable] of the [object]?

             - variable-object pairs = (currency, Myanmar), (price, Bitcoin)

Now, for generation we might specify differences for either prompt templates or variable/object. Based on following parameters(not comprehensive list):

   i. Style: Keep the meaning of the question same but ask in different styles.
               eg: introduce spelling mistakes, code mixing, cultural tone difference etc.

   ii. Semantic diversity: The task itself should change.

Add the above nlp metrics in code and set thresholds, keep generating till the thresholds are satisfied.

Reference:

1. https://arxiv.org/pdf/2005.04118.pdf (behavioral testing of the nlp)
   2.https://arxiv.org/abs/2111.09963 (behavioral testing of recommender system)
   Essentially we will framing the task as NLP task or recommendation task with behavioral metrics satisfied.

eg: keep generating similar sentences with cosine_similarity 0.5 and diversity 0.4 till "n" sentences reached or "x" attempts satisfied.

perhaps https://www.dynaconf.com/, plus XDG support https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

We need to control how many examples are generated, how dissimilar/similar we want them to be, confirming that we comply with the amount of similarity we want between generated data points

localhost because that easily becomes a remote interface

we're starting to add lots of parameters for configuration of LLM invocation for data synthesis and the agents, these need to be documented and sanity checked

quickstart_finetune.py currently demonstrates a successful finetuning of a local model.
While finetune is set to true in each completion call(lines 8, 14, 20), only the third call (line 20) results in finetuning and saving of the local model.

It is currently not obvious why we need 2 prior calls to completion before fine tuning successfully start.
If this is because we need prior examples this should be detailed in the documents and reflected in the feedback from the call itself in some way.

Hello, I downloaded the project (no installation) as I wanted to edit it for the bounties.

As soon as I tried to compile it gave me an error like this:
Code: from llm_vm.utils.keys import *
Error: No module named 'llm_vm'

If I had installed it there´s no doubt it would´ve worked. However, when doing imports from the same project I think the usual way to navigate through folders is using dots, like:
from ...utils.tools import *

Is there another way to go about this?

should probably surface the right hooks for this, though hobbyists dont really need or care about this